Ferguson, host of "The Late Late Show" since 2005, told his studio audience during Monday's taping that he will step down at the end of the year. Ferguson's show airs after Letterman's, at 12:35 a.m. on weekdays.
The move was no surprise after CBS announced that Stephen Colbert will replace Letterman sometime next year. There was a time that Ferguson, whose show won a Peabody Award in 2009, was considered a strong contender for that job.
But "The Late Late Show" has faded in the ratings, particularly with the arrival of Seth Meyers in February as competition in the same time slot.
"CBS and I are not getting divorced, we are 'consciously uncoupling,'" Ferguson said. "But we will still spend holidays together and share custody of the fake horse and robot skeleton, both of whom we love very much."
CBS Entertainment Chairwoman NIna Tassler said Ferguson "infused the broadcast with tremendous energy, unique comedy, insightful interviews and some of the most heartfelt monologues seen on television."
The Scottish-born Ferguson became a U.S. citizen during his tenure on the show.
He already has a new job lined up, as host of "Celebrity Name Game," a syndicated game show set to debut later this year.
It's been an unusually busy period of personnel changes in the late-night television arena. Jimmy Fallon took over the "Tonight" show on NBC from Jay Leno in February and was an instant sensation, ascending to the top of the ratings against Letterman and ABC's Jimmy Kimmel. Letterman announced that he would be leaving CBS sometime next year after more than three decades in late-night TV.
Chelsea Handler has also said she will be leaving her late-night show on E!
CBS plans to continue the show, and will be searching for another host, the network said. There's another opening at Comedy Central, which is looking to replace "The Colbert Report" when it ends at the end of the year.
Associated Press Television Writer Lynn Elber in Los Angeles contributed to this report.