Investigators said it happened around 1:30 a.m. Thursday at the intersection of Frankford Avenue and Harrison Street in the Frankford section of Philadelphia.
The mother told police she left her car running with the two small children in the back while she went in to pick up food from the New China Chinese & Ameican Food restaurant, but the suspect stole the car with the newborn and toddler still inside.
"At this we don't believe that the driver knew the children were in the backseat," said Chief Inspector Scott Small of the Philadelphia Police Department.
A 15th district officer spotted the car 15 minutes later at Cheltenham Avenue and Hegerman Street in the city's Wissinoming section, which is about a five-minute drive from where the car was stolen.
Investigators said they arrested the 25-year-old male driver after a struggle.
The children were unharmed and sleeping the entire time, according to police.
"She is really, really lucky. She did the right thing, she notified police right away. She gave us a good description of the vehicle and police from the 15th district did a great job stopping the vehicle, making an apprehension and the two children are fine and reunited with their mother," said Small.
Small said officers spoke with the mother about the importance of not making that mistake again.
The suspect is being held at the Special Victims Unit where he's being questioned by investigators.