The show's September debut earned 12.5 million viewers which was less than the premiere of the comedy "Modern Family", the leader of the night with 14.5 million viewers. "The X Factor" held those numbers for the second episode. In week two, ratings hovered around 12 million viewers for each episode.
"I'm not going to lie. I wanted 20 million when we launched," says Cowell, "but now I'm kind of back in the real world and I'm seeing this grow naturally. I'm as happy as I've ever been."
The show is Cowell's baby. Besides being a judge on its panel, he is its creator and an executive producer.
"X Factor" has a lot of competition not only from other TV shows but in convincing viewers to invest in another TV talent competition like Fox's "American Idol" and NBC's "The Voice," which averaged 12.6 million viewers an episode in its first season earlier this year.
Cowell says he "got a buzz" when he heard about "The Voice." "We're not in Russia where you have one show. This is life. If you're competitive and it spurs you on, I kind of get excited by it."
This week on "The X Factor," contestants will advance to the boot camp round of the series. Singers travel to Los Angeles to once again sing for the judges and a live audience as they are critiqued on various aspects of performing.
"The X Factor" airs Wednesdays and Thursdays on Fox at 8:00 p.m. ET/PT.