Clearly in the Christmas spirit, at the first mention of old holiday specials, Colbert launches into renditions of Williams' "Little Altar Boy" and Crosby and David Bowie's "Little Drummer Boy."
The latter was the inspiration for a duet between Colbert and Willie Nelson, who appears - in one of the more bizarre numbers - as a tiny wise man in a miniature nativity scene.
"This is just some good fun to watch during your eggnog-induced dementia," Colbert joked in an interview Wednesday.
The special finds Colbert far from his "Colbert Report" set in an obviously made-for-TV room of a mountain cabin dressed for Christmas. Hanging by the fireplace are two stockings, one labeled "Stephen," the other "Colbert."
While he's snowed in and a bear lurks outside, Colbert is visited by Nelson, Toby Keith, Jon Stewart, John Legend, Feist and Elvis Costello to sing Christmas songs that were penned by "Daily Show" executive producer David Javerbaum and composed by Adam Schlesinger.
"I had a clear, clear command to everyone involved: 'No cynicism,"' said Colbert. "We're not mocking Christmas specials. We're doing MY Christmas special. And that was the aesthetic we tried to bring into it. Like, we're really doing this. I want people to see this every year."
The special was originally planned for last Christmas but was delayed a year when Colbert became swamped during his brief run for president in the South Carolina primary. Instead, the special was taped mostly over a three-week period in August.
The 44-year-old comedian, who lives in New Jersey with his wife and three children, is a practicing Catholic who has taught Sunday school at his church. The special concludes on a positive note, with Colbert and Costello singing that "there are much worse things" than believing in Christmas.
Costume pieces from the special are being auctioned to benefit Feeding America, and a percentage of the DVD proceeds will also go to the charity.
Conservative pundits, of course, were the basis of Colbert's character - and there is some allusion to the "war on Christmas" that various commentators have waged in recent years.
But while Colbert still remains in character, the special is ultimately mostly free of politics. During the nonstop campaign, Colbert looked forward to the special like a "gift box," completely removed from the election.
"See, no politics," said Colbert proudly, as if proving his versatility.
Though fodder from the campaign was a boon to "The Report," Colbert says he feels greater freedom now that the election is over.
"I've actually had a better time than I've had in a long time," he said about the last few weeks. "I was strapped to someone else's galloping horse. There was no escaping how fast the news was changing. We were completely in a responsive comedy."
But there is nothing reactive about the unique "A Colbert Christmas."
"I'm so proud that we made something that is sincerely strange," said Colbert, "but also strangely sincere."