At least for now though, the longtime radio personality says of the idea, "My plate is full."
Stern addressed the late-night TV talk show host's retirement announcement while arriving in New York on Friday for "America's Got Talent" auditions. Stern is one of the judges on the show.
The 60-year-old stressed Letterman's impact on late-night TV, saying he's sad to see Letterman leave. Stern said he enjoyed his appearances on the "Late Show with David Letterman."
"I've been doing the show for years and years and I've had so many funny, great times with him that, you know, it'll be weird not to have him there."
Stern also said there are few people in show biz whose opinion matters to him, and Letterman is one.
One of Stern's fellow "America's Got Talent" judges, Heidi Klum, supports the idea of Stern taking over for Letterman, saying he'd be perfect.
"Howard is very special, you know? He gets right to the chase. He's not scared of anything, and I like that about him. He gets things out of people. Sometimes people don't even want to like say certain things, and he gets it out of them."
Letterman announced Thursday he plans to retire next year.
"America's Got Talent" returns in May.
