Hip-hop mogul Russell Simmons and the imam who had led an effort to build an Islamic center near the World Trade Center site were among those who addressed the crowd.
"Our real enemy is not Islam or Muslims," said the imam, Feisal Abdul Rauf. "The enemy is extremism and radicalism and radical ideology."
The chairman of the House Homeland Security Committee, Rep. Peter King, has said that affiliates of al-Qaida are radicalizing some American Muslims. He's planned hearings starting Thursday on the threat he says they pose.
King, a Republican from New York's Long Island, told CNN's "State of the Union" on Sunday that he sees an international movement with elements in the United States of Muslims becoming more radical and identifying with terrorists.
Speakers at the cold and drizzly Times Square rally said King was targeting Muslims unfairly.
"American Muslims are as fully American as any other faith community," said Rabbi Marc Schneier, founder of The Foundation for Ethnic Understanding. Singling out Muslim Americans "as the source of homegrown terrorism" is an injustice, he said.
Democratic Congressman Andre Carson of Indiana, one of two Muslims in Congress, said he wanted to say "to the Peter Kings of the world: We will not take your xenophobic behavior."
Imam Shamsi Ali, the leader of the Islamic Cultural Center on Manhattan's Upper East Side, said, "We are here today because we love this country. We are here today because we want to see America remain the most powerful and the most beautiful country in the world."
Simmons promised "to make sure that this rally is taken to the next generation and to a new age" by enlisting entertainers and sports figures to tweet about it, including Kim Kardashian, who tweeted Sunday that she stood with Simmons in "promoting love and compassion."
A smaller group rallied a few blocks away in support of King's hearings.
Beth Gilinsky of the Jewish Action Alliance heaped scorn on the Times Square rally's slogan, "Today I am a Muslim too," and referenced Rauf, who was given a reduced role in the Islamic center project this year because he had other commitments.
"I want to tell Imam Rauf and Imam Shamsi Ali and all of the rest of them up there that I am not a Muslim today," Gilinsky said. "Yesterday I wasn't a Muslim. Today I'm not a Muslim. I'm not going to be a Muslim for even 24 hours, Imam Rauf, and I'm not going to be a Muslim tomorrow. You will not convert me."