The band played the Who's "I Can't Explain" as McCain walked onstage at the Ed Sullivan Theater. After he sat down, Letterman asked, "Can you stay?"
"Depends on how bad it gets," McCain answered.
Letterman had replaced McCain with the GOP hopeful's persistent critic, MSNBC's Keith Olbermann, on Sept. 24. Olbermann was waiting in the wings Thursday - and McCain had a pained expression when he noticed that.
Although Letterman said he was "willing to put this behind us," he came after McCain hard with questions. He asked whether Palin was his first choice as vice president.
"Absolutely," McCain answered.
He said he didn't know her well before choosing her, but that he was impressed by her reputation as a reformer.
Letterman repeatedly pressed McCain on her qualifications, asking if he was confident she could lead the country in a time of crisis.
"In all due respect, one of the people I admired most was an obscure governor of a southern state called Arkansas and he turned out to be a fairly successful president," McCain said, complimenting Bill Clinton. "Ronald Reagan was a cowboy, no experience in international affairs. I think she has shown leadership."
As Letterman pressed on, McCain asked, "Have we pretty well exhausted this?"
"No, no," Letterman said. "I'm just getting started."
Letterman questioned him about Palin's claim that Democratic presidential candidate Barack Obama "palled around with terrorists," and McCain backed her up, saying his opponent need to better explain his relationship with former Weather Underground activist William Ayers.
"Did you not have a relationship with Gordon Liddy?" Letterman asked about Watergate burglar G. Gordon Liddy.
McCain said he knew him. Then, after a commercial break, McCain said, "I know Gordon Liddy. He paid his debt, he went to prison ... I'm not in any was embarrassed to know Gordon Liddy."
"You understand the same case could be made of your relationship with him as is being made with William Ayers?" Letterman said.
McCain said he has been completely open about his relationship with Liddy.
Letterman appeared to ridicule McCain about the implication that Obama and Ayers had a relationship.
"Are they double-dating, are they going to dinner, what are they doing?" Letterman asked. "Are they driving across country?"
"Maybe going to Denny's," McCain said.
Letterman said that Obama was 8 when Ayers was 29, and McCain appeared exasperated. "There's millions of words said in a campaign. C'mon, Dave," he said.
McCain said he thought Palin would appear on NBC's "Saturday Night Live," where Tina Fey has been doing a dead-on impersonation of her. "Probably get more of an audience than our debate did," he said.
Although Letterman had said he felt like an "ugly date" after McCain's initial cancellation, representatives for the two men never stopped talking about a return date.
While McCain risked a rough appearance - "I haven't had so much fun since my last interrogation," he said - it gave him the chance to show courage in the face of fire. Letterman reaches about 4 million people a night, a number sure to increase with McCain as guest. With clips on the Internet and Friday morning news, countless more people will undoubtedly learn about their encounter.
McCain did offer one campaign promise that he was probably more likely to keep after he left the stage.
"It's not the time to raise anybody's taxes - except yours," he said to Letterman. "I guarantee you if I become president, I'll do it. First executive order."