The court agreed with prosecutors that U.S. District Judge Ronald Buckwalter did not explain why he sentenced Vincent Fumo far below federal sentencing guidelines.
The court also upheld 68-year-old Fumo's conviction and ordered a new sentencing of an aide convicted at trial with him.
A jury in 2009 convicted Fumo of defrauding the state Senate, a museum and a South Philadelphia nonprofit of millions. The Philadelphia Democrat had been a wealthy power broker during his 30-year state Senate career. He remains incarcerated at a federal prison in Kentucky.
Prosecutors had sought at least a 15-year term for Fumo, who was convicted of all 137 fraud and obstruction counts after a five-month trial, and put his guideline sentencing range at 21 to 27 years.
In their appeal, they said "breathtaking" corruption had been exposed in the case and said Fumo used his control of a senate committee and the nonprofit to amass political power and live in grandeur.
Prosecutors said the fraud topped $4 million, twice the total assessed by the judge. They also complained that Buckwalter failed to explain how he calculated the fraud total or sentencing range, which they called "unreasonable" and "unduly lenient."
Defense attorneys argued that Buckwalter was on solid legal ground in imposing the sentence. They said the amounts had been calculated fairly and the decision to credit Fumo for "good works" was well grounded.
Aide Ruth Arnao was also convicted on all counts and received a one-year prison term, which will now also be revisited.
Fumo's sentence created a firestorm in Philadelphia and Harrisburg, especially after Buckwalter praised Fumo for his "extraordinary" public service.
By comparison, U.S. District Judge William H. Yohn Jr. called the conspiracy to destroy email evidence in the case "egregious" and sentenced a low-level Fumo aide to 2 ½ years.