Another major concern among the boomers: losing their financial independence.
The struggling economy, a longer life expectancy, ever-increasing health care costs and challenges facing Social Security are putting added pressure on the boomers, those born between 1946 and 1964.
According to the Associated Press-LifeGoesStrong.com poll, 43 percent of boomers polled said they were "very" or "extremely" worried about being able to pay for their medical costs, including long-term care. Almost the same number, 41 percent, said losing their financial independence was a big concern.
"I always say I am going to work until I'm in the ground," said Ellie Krall of Manalapan, N.J., one of the boomers polled. "I don't see myself being able to fully retire like people were able to do years ago."
Krall, 50, manager of an orthopedic office and mother of an 18-year-old son in college, said she's worried about paying for health care costs down the road and isn't banking on Social Security.
The oldest boomers are turning 65 this year, but it's the younger ones like Krall who might be feeling more apprehension.
"Boomers are not all created equal," says Olivia Mitchell, professor at the Wharton School of the University of Pennsylvania and executive director of the Pension Research Council.
Many older boomers still have a defined benefit pension plan, probably some decent retiree medical insurance and Social Security, said Mitchell, a boomer herself who has studied pensions and retirement extensively.
"The youngest boomers - the people who were born in the 60s - face more uncertainty about their pensions, their Social Security, their housing and their medical care," Mitchell said.
Her advice: "Push your retirement back two or three or five years if you can. As long as you are still working then you're not drawing down on your nest egg," Mitchell said in an AP interview. "What most people don't realize is how expensive it is to live in retirement."
Many people in their late 60s, and some into their 70s, are still working. According to the Bureau of Labor Statistics, 29.1 percent of people aged 65 to 69 worked at least part-time last year. And almost 7 percent of people aged 75 or older were employed in 2010.
One significant cost facing new retirees is health care. A study by Fidelity Investments estimated that a 65-year-old couple retiring this year with Medicare coverage would need $230,000, on average, to cover medical expenses in retirement. The estimate factors in the federal program's premiums, co-payments and deductibles, as well as out-of-pocket prescription costs.
The projection does not factor in long-term care, such as the cost of living in a nursing home - something most boomers in the Associated Press-LifeGoesStrong.com poll haven't planned for yet.
Some 83 percent of boomers polled said they do not have long-term care insurance, a private policy that helps pay for nursing homes or in-home care costs not covered by Medicare. Larry Plotkin, 60, of Wheeling, Ill., doesn't have the insurance but says he and his wife have thought about it.
"The problem with it is that as you get older, the cost goes up," said Plotkin. "At a certain point, it might not be worth it."
Costs for long-term care insurance can range from $1,000 to $8,000 a year, depending on age, health conditions, policy term and other factors.
"It's a tough sell," says Paul Fronstin, director of health research and education at the nonprofit Employee Benefit Research Institute. "Even someone in their 60s might look at it and say it's going to be 20 years before I need long-term care, so why buy it now."
Boomers also didn't fare so well in other later-life planning, such as legal wills and health care proxies.
Forty-percent of the boomers polled said they had a legal will to spell out how their possessions should be distributed after death.
Fewer boomers - 34 percent - had health care proxies and living wills. The living will allows people to document their wishes concerning medical treatment, and the proxy is a medical power of attorney that allows for the appointment of a trusted person to make medical decisions in case an individual is unable to do so.
The AP-LifeGoesStrong.com poll was conducted from June 3-12 by Knowledge Networks of Palo Alto, Calif., and involved online interviews with 1,416 adults, including 1,078 baby boomers born between 1946 and 1964. The margin of sampling error for results from the full sample is plus or minus 4.4 percentage points; for the boomers, it is plus or minus 3.3 percentage points.
Knowledge Networks used traditional telephone and mail sampling methods to randomly recruit respondents. People selected who had no Internet access were given it for free.