On Monday, Cowell confirmed he would be one of the three judges when the hit British talent show takes over TV later this year, but he's coy about who'll join him.
However, he did let some trade secrets slip in a new TV interview with ex-girlfriend Terri Seymour.
He told the reporter for news show Extra he and his production team are "talking to" Scherzinger and Cole and admits his former American Idol judging pal Paula Abdul is "in the mix" with the other potentials.
But Pussycat Dolls star Scherzinger appears to be a favorite. Simon says, "She is gorgeous and she does know what she's talking about."
He also dismissed reports George Michael might be on the panel and Cowell has also ruled out Christina Aguilera after her national anthem mess at the Super Bowl on Sunday.