Simon Cowell has kept quiet about reports that he got pal Andrew Silverman's wife Lauren pregnant about 10 weeks ago, dodging questions and refusing to comment on the allegations following Wednesday night's taping of The X Factor.

But there's more to this story than previously believed.

The Silvermans are still technically married, but according to Us Weekly, which broke the pregnancy story, the two "have been unhappy in their marriage for some time and their divorce has been in the works for a while."

"As their marriage deteriorated, she and Simon became close," a source told the magazine.

But The New York Post's Page Six now reports that when Andrew filed for divorce two weeks ago (July 15) he cited adultery as the reason for the filing, and he also included Cowell's name as a co-respondent in the papers.

Rumors of Cowell and Silverman's affair broke in May, and Simon's lawyer Martin Singer denied the story in a letter to the New York Post.

"Any implication or statement that my client is engaged in adultery and has allegedly broken up the marriage of Ms. Silverman is not only false but reckless," Singer wrote.

"Stating my client’s relationship with Lauren Silverman has become more serious such that she is now getting a divorce so that Simon and she can be together... is false and defamatory.”

Sources are now telling the Post that the Silvermans "weren't estranged" and that Andrew recently laid out some serious cash to buy his wife a new 10-karat diamond ring.

One source close to the Silvermans said that Lauren "is a total gold digger," while another implied that Lauren got pregnant on purpose to reel in Cowell and his millions.