The fire started around 2:30 a.m. on the 100 block of Hickory Lane near Hartford Road in the Woodridge neighborhood of Medford Township.
Four adults lived there, at least two of them use wheelchairs with the help from caregivers who stayed at the house all day.
Medford Township firefighters said that one of the employees helped firefighters rescue the residents from the burning home.
Action News spoke with Deidra Cohne who said she used to work at the group home.
"This is horrible, and I just need to know the conditions of the clients," said Cohen.
Cohen said some of the residents are hard to move. She was relieved in finding out they were able to be rescued considering the extent of the damage.
Firefighters had to shut down Hickory Lane and Hartford Road while trying to put the flames out.
Officials said four of the residents were taken to the hospital for evaluation and another is being treated for smoke inhalation.
Cohen now worries about where they'd go from here.
"This is all they have. Half of them don't even have families that recognize them," Cohen said.
Officials said the fire is under investigation by the Medford Township Fire Marshal's Office and the Medford Police Criminal Investigations Bureau.