They include millions of "our fellow citizens in this country" who are our friends, neighbors and co-workers and attend school with our children, Obama said.
Obama asked what it would say "when we start acting as if their religion is somehow offensive."
He said it's important to be clear on who the enemy is - al-Qaida and extremists, not Islam or the majority of Muslims.
Obama's comments at a White House news conference came as a Florida pastor threatened to burn Qurans on Saturday, the anniversary of the Sept. 11 attacks. Debate is also raging about an Islamic center planned near ground zero.