The Administrative Office of Pennsylvania Courts said in a news release that the four Centre County Common Pleas Court judges bowed out "to avoid any appearance of conflict of interest due to real or perceived connections" to Sandusky, the university or the charity for at-risk children Sandusky founded.
John M. Cleland, a senior judge from McKean County, was appointed to take over the case, although another judge, Kathy A. Morrow, was named to handle matters until he can assume jurisdiction.
The court system said neither Cleland nor Morrow, who sits in Perry and Juniata counties, have any known connections to Sandusky, the university or the charity.
Cleland chaired the Interbranch Commission on Juvenile Justice, established in the wake of the "kids-for-cash" courthouse scandal in which Luzerne County judges were accused of sending children to private detention centers for kickbacks.
Sandusky has maintained he is innocent of the 40 criminal counts against him, which accuse him of sexually abusing eight boys over a 15-year period.
On Tuesday, a preliminary hearing was rescheduled for Dec. 13 at the Centre County Courthouse in Bellefonte. It also will be handled by an out-of-county jurist, Westmoreland County Senior District Judge Robert E. Scott.
Scott replaced the district judge who set bail when Sandusky was arrested Nov. 5, Leslie Dutchcot of State College, who had ties to The Second Mile, Sandusky's charity. The court order said the change was designed to avoid any appearance of impropriety.
Sandusky was released on $100,000 unsecured bail, meaning he didn't have to post any collateral to be freed, but his status could change if additional accusers surface and police file new charges, as his lawyer fears.
Criminal defense lawyers said Sandusky could then be hit with a bail so high he would not be able to afford to pay it.
Prosecutors "don't have to start all over," said Lemoyne defense attorney Bill Costopoulos. "The additional counts would result in another arrest, another bail piece, another preliminary hearing date being set."
Sandusky's lawyer, Joe Amendola, told ABC's "Good Morning America" on Tuesday he was worried there may soon be new criminal allegations against his client.
"My concern is, if they bring new charges based upon new people coming forward, that bail's going to be set and he's going to wind up in jail," said Amendola, who has not returned multiple phone messages from The Associated Press seeking comment.
Scott would be making new decisions about bail in the context of public outrage over the allegations, which include charges that Sandusky found victims among boys being helped by The Second Mile.
"The more charges, the more serious it becomes, and of course I've heard public outcry that his bail is unsecured and it's too low," said Tunkhannock defense attorney Gerald Grimaud. "Any new judge or district magistrate is not tone deaf. I'm sure they're reading things in the news media and watching things on TV like everybody else."
Until the preliminary hearing, prosecutors can seek to have bail modified by the district judge, said Lehigh County District Attorney Jim Martin. After that hearing, bail changes would have to be pursued by a county court petition, he said.
Martin said criminal complaints can also be amended prior to a preliminary hearing, but afterward the defendant would have to be re-arrested, and then the prosecution and defense would argue over whether to consolidate the two sets of charges for trial.
An attorney general's office spokesman declined to comment on the Centre County judges' recusal or about potential new charges against Sandusky.