It was 5:45 a.m. when officers arrived at the Oakland Park Apartments, located on the 200 block of Coolidge Avenue.
The woman told authorities that she managed to escape with a 1-year-old child, but that the man who was allegedly holding her remained inside with another child.
New Jersey State Police were then called in to assist in negotiating with the man, and by 9:50 a.m. the man finally surrendered in the company of a 3-year-old girl.
It's unclear if the man was armed during the incident.
It is believed that the man and woman are in a relationship.
Police are continuing their investigation. So far, no word on charges.