The blaze broke out at around 4:00 p.m. at the Colonial Park Apartments.
Evacuated residents, many of them needing assistance, were slowly ushered into the first floor of an unaffected building.
Firefighters arrived to heavy fire from two apartments on the fourth floor.
Emergency operators received several calls reporting some residents were trapped.
"I don't know [what happened.] I was asleep and I woke up and I saw these blue lights and whatnot and then my nieces started calling me saying your building's on fire; I go to shaking," resident Mary Green said.
"I was just worried about going down the steps because of my legs and my back, other than that, I wasn't afraid," resident Charlotte Love said.
The Colonia Park Apartment complex is a low income elderly facility subsidized by the federal government.
As firefighters put out the flames, relatives made their way to the facility to check on loved ones.
The cause of the fire is still under investigation.
Two residents were taken to local hospitals after suffering minor injuries. One received minor burns and the other was suffering from smoke inhalation. Both are expected to be OK.
Officials are accessing the damage to determine the exact number of displaced residents.
