The fire was reported around 6:05 p.m. at the former New Jersey Avenue school in the unit block of New Jersey Avenue.
Flames spread to all three floors in the center of the building, which was built in 1925.
Raw video of the school building fire in Atlantic City, New Jersey
According to a post on the school district's website, the school was being demolished to make way for a new district administration building.
It had been closed for more than a decade, closing its doors in 2012 when the Pennsylvania Avenue School opened.
Later that same year, the building was severely damaged by flooding during Superstorm Sandy, leaving it unusable, the district said.
"We are grateful to the Atlantic City Fire Department and first responders who are working diligently to contain the blaze and ensure the safety of the surrounding neighborhood," said Superintendent La'Quetta Small in a statement.
Photos taken by Action News viewers showed the black smoke that could be seen for miles along the Jersey Shore.
There was no immediate word on the cause of this fire.
Officials say the building was unoccupied at the time of the fire. No injuries have been reported.
Discolored Water Warning
Atlantic City officials warned residents that they may temporarily experience discolored water in their homes.
Residents are asked to run their water until it clears, adding that the discoloration may continue while firefighting efforts are underway.
Anyone who continues to experience discolored water after firefighting operations end can contact the city for assistance.