He was quizzed about his relationship with the Simpson family while appearing on "Watch What Happens Live" on Tuesday with his bandmate and brother, Drew.
Lachey, who is now married to Vanessa Minnillo, divorced Simpson in 2005 and admitted he hasn't had contact with the singer/actress for a long time, saying, "It's been years. Years. Honestly, probably six years. It was like another lifetime ago."
Host Andy Cohen quizzed Lachey about his ex-father-in-law, Joe Simpson, who is in the midst of a divorce from Jessica's mom Tina after 34 years of marriage. Rumors started circulating about Joe's sexuality last year.
When Lachey was asked what the best thing was about cutting ties with the former minister, Lachey blurted out, "I don't have to play grab a** under the table on Easter Sunday!"
Cohen laughed and said, "That's very funny," while Lachey quickly realized he may have said the wrong thing. He replied: "It may not be (funny) tomorrow, but it is tonight."