The family says a 5-year-old Ty-Yon and 7-year-old Michael passed away at the hospital after the blaze early Saturday morning on the unit block of West Washington Street.
The children were initially taken to Crozer Hospital's burn unit and then transferred to St. Christopher's Hospital for Children, after firefighters pulled them out.
Tekeema Moore, a woman who said she was the brothers' aunt, spoke with Action News.
"My niece is broken, she has to live right now without her boys," said Moore. "She didn't expect for her boys to go this soon, they're only 7 and 5, and now this is what she has to live with. It's terrible. I wouldn't wish this on no one."
Moore said the boys' mother, 25-year-old, Ty-Yana Pierson, loved to make dance videos with her boys, who were her only children.
The others inside the home survived. A man inside was able to get his mom and a baby out in time.
The cause of the fire remains under investigation.
Tisha Holloway said her stepmother lives in the home. She was asleep at the time, and so were her grandchild, son, and two unrelated children she was babysitting.
"My little brother didn't know that the other two kids were there. He just knew that his mom and his nephew were in there," said Holloway.
She said her brother was able to get his mom and the baby out, but the fire quickly engulfed the house, making it too dangerous to get to the other two children.
Mayor Gary Stevenson said police arrived at the scene initially and were told there were children in the burning home.
Police officers tried to gain entry into the house but were driven back by the flames and smoke.
Firefighters responded within five minutes, the mayor said.
Crews put up ladders and were able to rescue two children from the second floor.
Neighbors and family stood outside the home for hours Saturday in disbelief this could happen to the woman who lived there.
"She is the neighborhood mom, like she's here for everybody. She feeds everybody; summertime we have barbeques. She turns nobody away," said Holloway.
The Red Cross is helping the family with living arrangements. The community is also rallying to raise money for the beloved woman.
Two women from the community stopped by the home Monday to lay flowers by the front stoop.
"We're going through times, bad times right now, but this pulls the community together," said Cindy Trout, from Paulsboro.
"Losing two children, I mean what can you say when it comes to that, it's, I mean, I can't even imagine, and I have children," said Marilyn Martinez, from Paulsboro.
The fire marshal and prosecutor's office are investigating the cause of the fire.