Several fire companies responded to the scene and over 100 firefighters battled the blaze.
The flames were reported around 12 noon Saturday at the Valley Forge Beef and Ale restaurant in Audubon.
It took four hours to bring the blaze under control.
"There was an explosion in the basement, looked like the boiler room.," said Brian Flaxman.
Flaxman, the owner of the restaurant, was one of 23 people inside when the fire broke.
The flames started in the basement. An employee reported hearing an explosion and seeing black smoke and fire coming upstairs.
The fire eventually made its way from the basement to the third floor and out to the roof.
"I just renovated, just did the men's room, redid the bar. New floor in the bar. new ladies room, new pizza oven - been doing a lot of renovations over the last few years," said Flaxman.
Firefighters say it's all the renovations over the years that made the fire so tricky to fight.It took hours to bring the blaze under control.
The original part of the building was built in the late 1800s and there have been four additions.
The fire moved in between those additions, making it difficult for firefighters to get the upper hand.
Regulars like Scott Cannon, came to see what happened to their favorite neighborhood hangout.
They will have to find somewhere else to meet for now, but say they will come back.
"We want to support him. People will definitely support him coming back," said Cannon.
"We're going to try to rebuild, clean it out and see where it's at," said Flaxman.
Flaxman says he has about 25 employees that will also have to find at temporary jobs somewhere else.
Meanwhile the cause of the blaze remains under investigation.