Talk about digging around in old wounds... OK, so we get hi being jarred by the "crazy hair" and the unfortunate outfit choice, but did he have to mention the weight gain and the confidence issues?
Speaking about his decision to axe her from the show, he said: "There was so much expectation on the show in America, and I definitely wasn't myself. I was uptight. When I saw her I didn't recognise her. Literally. The hair and the outfit were crazy. Just totally crazy."
The 54-year-old Sherlock reckons Cheryl's recent split from her then-husband Ashley Cole played a massive role in her lack of confidence.
Simon continued: "I asked her manager, 'What's she wearing, what's happened to her hair?' He was defensive about it. I haven't a clue why she turned up like that. We then did the first day of auditions, and she was nervous, shy, not in her comfort zone. She'd put on a bit of weight, but not loads... That wasn't the issue. The only way I can describe it is that it wasn't Cheryl sitting in that seat. She'd lost her confidence."
And despite previously claiming Cheryl - who is now married to French restaurateur Jean-Bernard Fernandez-Versini - just wasn't right for the US show, Simon's now confessed it was the programme itself that didn't work across the pond.
He told the Radio Times magazine: "There was something not quite right about that show in America."
Um, because it already existed under the guise of American Idol, which is obviously so much better cause it has "American" and "Idol" in the title.