The North Wildwood fire chief said the man became stuck between two rocks, making it tricky and dangerous to rescue him.
"The green slime and barnacles are very, very slippery," said Chief Dominick McClain.
Firefighters took precautions and used climbing gear and ropes after getting 911 calls at about 12:30 p.m.
"He slipped off of there and rolled down and struck his head. So, he had some contusions on his head," said McClain.
The victim, a man in his 50s, also had broken ribs. A ladder truck raised him to safety.
McClain said the equipment is new and was purchased with the seawall in mind.
"People run on (the seawall), they walk on it. It's flat, it's nice and clear. You just have to be careful of going off the edge because there are no railings on that. It's open for the view," said McClain.
"I saw the guy, he was stuck in between the rocks," said William Joe, who says he was fishing Monday when the accident happened.
"Sometimes it's scary, especially when they're wet, but that guy? He just fell in," said Joe.
He says he and his friends fish on the rocks almost every day.
"Yeah, they're slippery. You have to stay off the green stuff," said Timothy Everett, who was fishing on the rocks Tuesday.
He and his friends say they feel safe about it.
"First of all, you have to make sure your equilibrium is OK because if you're staring at the water too long, you can get dizzy," he said.
Others say they stick to the wall.
"All them fisherman walking down there on all the slimy rocks, I'm surprised it hasn't happened sooner," said Gerry Mullin, of North Wildwood.
Safety signs on the seawall prohibit things bikes and alcohol but don't warn against staying off the rocks. Firefighters say the fall victim will be OK and is recovering in the hospital.