At a Senate Judiciary Committee hearing, the attorney general said that the probe was flawed in concept as well as in execution, never should have happened and "it must never happen again."
Holder's comments came as the panel's ranking Republican, Sen. Charles Grassley of Iowa, said the operation represented an "utter failure" by federal law enforcement officials to enforce existing gun laws.
Holder said that he wants to know why and how firearms that should have been under surveillance could wind up in the hands of Mexican drug cartels.
Grassley asked who the attorney general "plans to hold accountable" for the arms trafficking probe.
In August, the Justice Department replaced three officials who played critical roles in the arms trafficking probe - the acting director of ATF, the U.S. attorney in Arizona and a prosecutor who worked on the arms trafficking probe. The department's inspector general is still investigating the case, at Holder's request.
Grassley pointed to a Justice Department letter the senator received last February saying that federal agents make "every effort" to intercept weapons that have been purchased illegally and prevent their transportation to Mexico.
Holder said that the letter was based on incorrect information and that "I regret that." The attorney general said the incorrect information was supplied by the U.S. Attorney's office in Phoenix and the Bureau of Alcohol, Tobacco, Firearms and Explosives
Regarding the fallout from Fast and Furious, Holder said that "unfortunately, we will feel its effects for years to come as guns that were lost during this operation continue to show up at crime scenes both here and in Mexico."
Several ATF agents have testified that they were ordered by superiors to let suspected straw buyers walk away from Phoenix-area gun shops with AK-47s and other weapons believed headed for Mexican drug cartels, rather than arrest the buyers and seize the guns there.
The goal was to track the guns to trafficking ring leaders who long had escaped prosecution. ATF lost track of some 1,400 of the more than 2,000 weapons whose purchases attracted the suspicion of the Fast and Furious investigators.
Grassley's investigation brought problems in Operation Fast and Furious to light early this year. Sen. John Cornyn, R-Texas, was expected to question Holder on whether federal agents in Texas adopted the same controversial tactic called gun-walking used in Arizona in Operation Fast and Furious.
Holder also may run into questioning by Democrats on the panel over new FBI rules on intelligence collection activities, an issue of importance to civil liberties groups concerned that in a post-Sept. 11 world, the government is loosening restrictions on investigative tactics.
In the years since 9/11, Congress and the Justice Department have granted the FBI "ever-greater powers to investigate Americans with less basis for suspicion and less oversight," said Elizabeth Goitein, co-director of the Liberty and National Security Program at the Brennan Center for Justice.
Holder, who says he learned of problems in Fast and Furious early this year, has become a focal point for criticism in a congressional investigation by the House Oversight and Government Reform Committee chaired by Rep. Darrell Issa, R-Calif. Republican critics have suggested Holder was informed of the problems as early as July 2010 when the operation's name turned up repeatedly in weekly departmental reports. The reports provided updates on dozens of investigations, including Fast and Furious, but do not mention the gun-walking tactic.
Holder used the hearing as an opportunity to urge support for ATF.
The attorney general cited congressional testimony by some of the ATF agents in the probe who said they lack effective enforcement tools. They have sought clearer legal authority to arrest straw purchasers and tougher prison sentences for them. Holder asked Congress to "fully fund our request for teams of agents to fight gun trafficking."
On Monday, the committee's chairman, Sen. Patrick Leahy, D-Vt., asked whether the Justice Department's inspector general has expanded its probe of Operation Fast and Furious to include earlier Bush-era arms trafficking probes that relied on gun-walking.
The Associated Press reported on Friday that a briefing paper prepared for then-Attorney General Michael Mukasey during the Bush administration in 2007 outlined failed attempts by federal agents to track illicitly purchased guns across the border into Mexico. Those failed attempts involved an earlier gun-walking probe run out of the same ATF office in Phoenix that later handled Operation Fast and Furious.
A month ago, the AP also disclosed that several hundred weapons wound up in the hands of arms traffickers in a second Bush-era gun-walking probe beginning in 2006. It was called Operation Wide Receiver and was run out of the ATF's office in Tucson, Ariz.
The IG's office says in a semiannual report that it is reviewing Operation Fast and Furious "and other investigations with similar objectives, methods and strategies." A spokesman for the IG's office, Jay Lerner, declined to comment on whether the investigation has been expanded to cover Wide Receiver and the probe that the briefing paper to Mukasey referenced.