Doctors say it's another sign that people should be careful about using vitamins and other supplements.
"People tend to think of vitamins as innocuous substances, almost like chicken soup - take a little and it can't hurt," said lead author Dr. Eric Klein of the Cleveland Clinic. The study shows that is not true.
"If you have normal levels, the vitamin is probably of no benefit, and if you take too much, you can be harmed," Klein said.
Men randomly assigned to take a 400-unit capsule of vitamin E every day for about five years were 17 percent more likely to get prostate cancer than those given dummy pills. That dose, commonly found in over-the-counter supplements, is almost 20 times higher than the recommended adult amount, which is about 23 units daily.
The results mean for every 1,000 men who took vitamin E, there were 11 additional cases of prostate cancer, compared with men taking dummy pills.
The study was actually launched to try to confirm less rigorous research suggesting vitamin E might protect against prostate cancer. Overall, about 160 of every 1,000 U.S. men will develop prostate cancer in their lifetime.
Risks increase as men age. Detection can be tricky since symptoms such as frequent urination can also be caused by harmless conditions, and doctors disagree about the benefits of screening tests. Treatment is also complex since some slow-growing prostate cancers are not deadly but some procedures, including surgery, can damage sexual function and cause incontinence.
Just last week, a government-convened panel of experts recommended against routine PSA screening for prostate cancer. That draft advice is open for public comment.
What should vitamin E users do, given the new study results? About 13 percent of American men take it, according to a supplement trade group.
Dr. Otis Brawley, chief medical officer for the American Cancer Society, says they should stop taking large doses and talk to their doctors about risks and benefits from prostate cancer screening. Smaller doses, typically found in multivitamins, are probably fine, said Brawley, who was not involved in the research.
Vitamin E is found in foods such as nuts, seeds and vegetable oils. The nutrient helps nerves, muscles, blood vessels and the immune system function.
Vitamin E supplements have long been promoted for disease prevention, but scientific research has disproven many claims and suggested they might increase risks for some conditions, including heart failure.
Brawley noted that the study echoes previous thinking on beta-carotene, which once was thought to protect against cancer but more recently has been linked with increased risks for lung cancer, especially in smokers.
"There should be a global warning that ... excessive use of vitamins has not been proven to be beneficial and may be the opposite," Brawley said.
Experts generally agree that foods are the best sources for vitamins.
The new research appears in Wednesday's Journal of the American Medical Association. The National Cancer Institute and National Center for Complementary and Alternative Medicine paid for the multimillion-dollar study.
Joe Latina, a cabinet shop owner in Aurora, Ohio, was among study participants. He said researchers gave him pills they said "might slow down prostate cancer." Now 71 and cancer-free, he says he doesn't know whether he was given vitamins or dummy pills.
Latina said he "was kind of surprised" by the study results, but is not stressing out over the possibility that he still might get cancer.
"I'm a positive thinker," he said. "I'm not walking around saying, `Oh my God, the other shoe is going to drop. I don't think I have any way to control it."
The study involved more than 35,000 healthy men aged 50 and older, from the United States, Canada and Puerto Rico. They were randomly assigned to take daily vitamin E or selenium supplements, both pills or dummy pills. The study was halted after about five years when there were signs of no benefit and a possible increased risk for prostate cancer in vitamin E users. The researchers continued tracking the men even after they stopped taking pills.
The follow-up found that a potential link between selenium and diabetes was a false alarm, but it confirmed signs of a vitamin E-prostate cancer link. Over a total of about seven years, there were 76 cases of prostate cancer diagnosed per 1,000 men, versus 65 cases in men given dummy pills.
"The implications of our observations are substantial," the study authors said.
The results suggest that extra risks associated with taking relatively high doses of vitamin E continue even after supplements are stopped. The researchers said it is unclear how vitamin E would harm the prostate.
There was no increased risk for men who took both vitamin E and selenium, suggesting that selenium might somehow counter the harmful effects of vitamin E, the study authors said.
Duffy MacKay of the Council for Responsible Nutrition, a supplement makers' trade group, said the study shouldn't be interpreted as questioning the benefits of vitamin E as an essential nutrient, and he said there is evidence that many Americans don't get enough.