Philadelphia District Attorney Seth Williams made the announcement regarding Rev. Robert Brennan Wednesday, surrounded by the family of Brennan's accuser.
Williams said the family of Sean McIlmail wanted his name and the alleged abuse against him made public.
McIlmail, a former altar boy, came forward earlier this year to accuse Brennan of having molested him over a period of three years, beginning in 1998 when McIlmail was 11.
In withdrawing the charges, Williams said prosecutors determined that there was not enough evidence apart from McIlmail's testimony to try Brennan.
The announcement came a week and a half after McIlmail died at the age of 26.
"He had really straightened out his life. He was motivated to pursue justice. This was his goal, so this was shocking to all of us," lawyer Marci Hamilton said after the death. "He made it through college. He was battling demons all the way through."
Brennan, 75, was described as an alleged serial abuser in a 2005 grand jury report on priest abuse in the Roman Catholic Archdiocese of Philadelphia, but he was never charged until this year because of legal time limits.
After McIlmail's death, prosecutors had to decide whether the rape case could go forward without the accuser's trial testimony. Two other men had filed civil suits against Brennan, and could potentially offer supporting testimony.
Defense lawyer Trevan Borum said he did not know what the other evidence might be, but said "obviously the central part of their case would have been the accuser's testimony."
Brennan was freed on $50,000 bail two weeks ago. He was suspended by the church after the 2005 grand jury report, and had been living in Perryville, Md.
The Associated Press contributed to this report.