"We let the boss down," Army Gen. Martin Dempsey, chairman of the Joint Chiefs of Staff, told a Pentagon news conference. He said he regretted that the scandal, which also involved 11 Secret Service agents accused of cavorting with prostitutes at the hotel, diverted attention from Obama's diplomacy at a Latin America summit.
"I can speak for myself and my fellow chiefs: We're embarrassed by what occurred in Colombia, though we're not sure exactly what it is," Dempsey added.
Pentagon officials said earlier Monday that the number of military members involved in the scandal appears to be greater than the five originally cited. One senior defense official said that at least 10 military members may have been involved. The official spoke on condition of anonymity because the matter is under investigation.
Pentagon press secretary George Little said that military members who are being investigated were assigned to support the Secret Service in preparation for Obama's official visit to Cartagena. He said they were not directly involved in presidential security.
The Secret Service sent 11 of its members, a group including agents and uniformed officers, home from Colombia amid allegations that they had hired prostitutes at a Cartagena hotel. The military members being investigated were staying at the same hotel, Little said.
The Secret Service personnel were placed on administrative leave and on Monday the Secret Service announced that the agency had also revoked thier security clearances. That means they cannot enter Secret Service facilities or facilities guarded by the Secret Service, said agency spokesman Brian Leary.
Though on leave, the agents and officers will continue to be paid.
Appearing with Dempsey at the Pentagon, Defense Secretary Leon Panetta said U.S. troops are expected to abide by "the highest standards" of behavior whether they are at home or abroad. He noted that a military investigation is under way and promised that if wrongdoing is confirmed, "these individuals will be held accountable."
Army Col. Scott Malcom, chief spokesman for U.S. Southern Command, which organized the military team that was assigned to support the Secret Service's mission in Cartagena, declined to say how many additional service members are under investigation. He also would not say which branch of the military they were from.
"We are still putting together all the facts," Malcom said.
A defense official in Washington said at least some of those under investigation are members of the Army. The official spoke on condition of anonymity because the matter is under active investigation.
Malcom said a colonel from the Southern Command staff, whom he would not identify by name, had been sent to Cartagena to gather facts. He said at least five military members under investigation were being flown to Miami on Monday.
The U.S. Southern Command had announced on Saturday that five service members assigned to the presidential mission in Colombia had violated curfew and may have been involved in "inappropriate conduct."
In a statement Saturday, Air Force Gen. Douglas Fraser, commander of Southern Command, said he was "disappointed by the entire incident" and that "this behavior is not in keeping with the professional standards expected of members of the United States military."
The Secret Service placed the 11 agents on leave while the agency reviews what happened.
"I expect that investigation to be thorough, and I expect it to be rigorous," Obama said Sunday. "If it turns out that some of the allegations that have been made in the press are confirmed, then of course I'll be angry. ... We are representing the people of the United States, and when we travel to another country, I expect us to observe the highest standards."
California Rep. Darrell Issa, chairman of a House investigative panel, said he wasn't certain whether Congress would hold hearings on the alleged misconduct. But lawmakers will be looking "over the shoulder" of the Secret Service, he said, to make sure that the agency's methods for training and screening agents aren't endangering the nation's VIPs.
Associated Press writers Pauline Jelinek and Jim Kuhnhenn contributed to this report.