It seems even The X Factor USA are also struggling to hold onto their judges with the latest announcement that Demi Lovato is set to leave the talent show.
Demi, who joined the show in 2012, has also appeared in Glee and began her career in Barney and Friends (always good to know those kids turned out alright).
The singer has said she is now going to focus all her attention on her music, touring and making a new album.
Simon Cowell tweeted : "She and I have had our ups and downs but now consider her a true friend. The door will always be open. And thank you Demi. Simon.x"
So who will be the next in line for a judges throne? No doubt Simon will have something up his sleeve, lets just hope it's not another Cheryl Cole scenario...