The chairman of the House Homeland Security Committee, Rep. Bennie Thompson, said the country is fortunate the affair didn't end in a "night of horror."
Meantime in an interview at the White House, Obama said that even though "the system didn't work the way it was supposed to" last week, he still feels safe in the mansion and trust the agency fully to protect not only him but his wife and children.
"I could not have more confidence in the Secret Service," Obama told USA TODAY's Richard Wolf and Justin Hyde of the Detroit Free Press.
Appearing before Thompson's panel for questioning, Sullivan acknowledged his officers made mistakes and that the Secret Service must have a "100 percent" performance record.
Thompson, D-Miss., also said that Congress needs to talk not only to Tareq and Michaele Salahi, the couple who got in without invitations, but also to White House social secretary Desiree Rogers. All three have declined to appear. Thompson said he is likely to authorize a subpoena for the Salahis to testify.
Rep. Pete King, the panel's ranking Republican, accused the White House of "stonewalling" in not permitting Rogers to appear. The New York congressman said he thinks the White House is either afraid of something or doesn't want to take any heat for last week's incident.
Thompson said at midday that the Salahis could be cited for contempt of Congress if they continue to shun the committee's request that they proceed. He said he has asked staff to prepare subpoenas for the pair, and said "my door remains open." King indicated he would continue to press for Rogers to appear as well. The two appeared to differ on whether Rogers should be subpoenaed.
Thompson said at the outset: "This hearing is not about crashing a party at the White House. Nor is it about wannabe celebrities." He said the purpose is to better protect the president.
The Salahis have been trying to land a part on a Bravo reality show, "The Real Housewives of D.C.," and were filmed by the TV show around town as they prepared for the White House dinner.
"We're not concerned about agency embarrassment," Thompson said. "We're all fortunate that this diplomatic celebration did not become a night of horror. ... We must dissect every fact ... and after we do these things, we need to give thanks that no lives were lost," he said.
Said Sullivan: "In our judgment, a mistake was made. In our line of work, we cannot afford even one mistake."
"I fully acknowledge that the proper procedures were not followed," he said. " ... This flaw has not changed our agency's standard, which is to be right 100 percent of the time."
Thompson asked Sullivan what went wrong.
"Pure and simple, this was human error" in which normal security protocols were not followed, Sullivan said. The breach was not caused by poor screening technology, he added.
The Secret Service chief said the investigation so far has found three people from the agency's uniformed officer division responsible for the security breach and all three have been put on administrative leave. He added that the agency is still reviewing what security protocols weren't followed.
"What we find is if the protocols are followed, we would not run into this situation," Sullivan said.
Asked whether there was a risk posed to people attending the dinner for the visiting prime minister of India, Sullivan said he was confident there wasn't.
Sullivan said there was no threat to Obama, noting that "last week we took him to a basketball game, and there was 5,000 people sitting around the president."
In response to a question from Del. Eleanor Holmes Norton, D-D.C., he said Obama had not had an extraordinary number of threats against his life, contrary to her assertion, and said that Obama had received no more such threats at this point in his term than his two predecessors.
White House press secretary Robert Gibbs earlier this week described both Obama and his wife, Michelle, as angered by the incident.
Attending a White House event shouldn't be like "going to a big box retailer the day after Thanksgiving," Rep. Jane Harman, D-Calif., told Sullivan.
Asked by King if the pair would have been able to penetrate the White House if a representative of the White House had indeed been present for clearance assistance, the Secret Service chief replied, "It would have helped."
From now on, the White House has said, someone from the social office will be present to help the Secret Service if questions arise.
On the eve of the hearing, the Salahis' publicist, Mahogany Jones, issued a statement addressing why the couple would not appear before Congress.
The Salahis believe "there is nothing further that they can do to assist Congress in its inquiry regarding White House protocol and certain security procedures," the statement said. "They therefore respectfully decline to testify."
Jones said the couple's information makes clear they broke no laws, that White House protocol at the dinner "was either deficient or mismanaged" and that "there were honest misunderstandings and mistakes made by all parties involved."
The White House also took some responsibility for the foul-up. "After reviewing our actions, it is clear that the White House did not do everything we could have done to assist the United States Secret Service in ensuring that only invited guests enter the complex," Jim Messina, deputy chief of staff, wrote in a memo to staff Wednesday.
Still, White House spokesman Robert Gibbs cited the separation of powers and a history of White House staff not testifying before Congress in explaining why Rogers, herself a guest at the dinner, wouldn't be coming.
A senior White House aide, Valerie Jarrett, defended Rogers' refusal to appear, telling a network news show Thursday morning that executive staff members have been allowed to testify to Congress only in rare circumstances in the past.
Jarrett said on ABC's "Good Morning America" that there was no need for Rogers to attend the hearing and answer questions because "we think we've really answered the questions fully."
Copies of e-mails between the Salahis and a Pentagon official have clouded the couple's claims that they were invited to the state dinner honoring the visiting Indian prime minister.
RELATED LINK: Read the text of the emails