Picower's wife, Barbara, discovered his body and pulled him from the water with help from a housekeeper, authorities said. He was pronounced dead at Good Samaritan Medical Center at about 1:30 p.m.
Palm Beach police are investigating the death as a drowning, but have not ruled out anything on the cause of death.
Picower suffered from Parkinson's disease and had "heart-related issues," said family attorney William D. Zabel. He described Picower's health as "poor."
Picower's body showed no visible injuries, said Joseph Sekula, spokesman for the Palm Beach Fire Department.
"There wasn't anything noted as far as trauma or anything to the body," he said, adding that "it did appear that he was swimming because he was wearing swimming trunks."
Detectives were still at the home more than six hours after the initial 911 call. The iron gate to his long driveway was open and several Palm Beach police cars were parked near the mansion. The home and property is worth more than $33 million, according to the county property appraiser's records.
Picower had been accused by jilted investors of being the biggest beneficiary of Madoff's schemes.
In a lawsuit to recover Madoff's assets, trustee Irving Picard demanded Picower return more than $7 billion in bogus profits. In an e-mailed statement Sunday, Picard said only that "litigation will continue."
Zabel, the Picower family attorney, said in a statement that "there was progress towards a settlement with the trustee."
Picower and his wife started the Picower Foundation in 1989, which has given millions to the Massachusetts Institute of Technology, Human Rights First and the New York Public Library. It also funded diabetes research at Harvard Medical School.
The foundation, whose assets were managed by Madoff, said in its 2007 tax return its investment portfolio was valued at nearly $1 billion.
After the Madoff scandal broke in December, the Picower foundation said it would have to cease grant-making and would be forced to close.
But the trustee's lawyer said Picower's claims that he was a victim "ring hollow" because he withdrew more of other investors' money than anyone else during three decades and should have noticed signs of fraud.
According to the lawyers, Picower's accounts were "riddled with blatant and obvious fraud," and he should have recognized that because he was a sophisticated investor.
Picower had asked that the lawsuit be dismissed, saying it was unsupported by the facts.
Madoff is serving a 150-year prison sentence after he admitted losing billions of dollars for thousands of clients over a half-century career that saw him rise to be a Nasdaq chairman. Madoff's attorney, Ira Sorkin, did not respond to a request for comment.
Jonathan Landers, an attorney representing a large group of victims, said in an e-mail that it was impossible to tell what effect Picower's death would have on efforts to recover funds lost in Madoff's massive Ponzi scheme.
"While there are allegations regarding his knowledge of the Madoff fraud and his possible liability to investors, none have been proven," he wrote. Landers added that even if such facts could be proved, Picower's "death could make it easier or more difficult to obtain and collect on claims."
"It may cause those who have control of his assets to fight harder because there is no longer any personal dignity or desire to settle and move on," he wrote.
Jerry Reisman, another attorney representing about 26 victims, said that Picower's death does make it more difficult for the trustee to recoup some of the money for the victims.
"We won't be able to hear from his own words whether he was complicit," Reisman said.
Associated Press writers Lisa Orkin Emmanuel in Miami and Cristian Salazar in New York City contributed to this report.