Michael Phillips was supposed to be arraigned late Wednesday night, but the judge did not proceed after observing his condition.
Officers then escorted Phillips out of the Coatsville Police Department and took him to Chester County Prison.
Authorities say Phillips allegedly stabbed his infant son in the stomach and then threw him into the snow.
Late Wednesday morning, officers responded to what officials described as an altercation between a mother and a father. Investigators say the mother was trying to stop Phillips from stabbing their son.
According to an affidavit, Phillips argued with officers who first arrived at the scene, while another officer ran toward an ambulance carrying the 3-month-old baby wrapped in a blanket.
The affidavit states that blood was on Phillips' shirt and that body camera footage later captured him saying, "I did it, God, I did it." He also allegedly made statements saying, "This was all part of God's plan."
Neighbor Danielle Gray said she was in shock after learning the disturbing details.
"I called my husband, and I said you're not going to believe what happened," Gray said.
Neighbors told Action News that the Phillips family has lived in the Millview Apartments for years.
The affidavit also states that the baby's mother told officials Phillips made comments about sacrificing the baby before he stabbed him. Investigators say that is when she told her 9-year-old child to run and get help.
Investigators say Phillips took the baby and threw him into the snow, and the mother then laid on top of the child in an effort to protect her son.
"It's shocking. It's truly shocking, and to find out who it was, heartbreaking. I've known them for years," Gray said.
Phillips is being held at Chester County Prison after being denied bail during his arraignment on Thursday morning.
He is facing several charges, including attempted criminal homicide, aggravated assault with intent to cause severe bodily injury and endangering the welfare of a child.
The child was flown to Children's Hospital of Philadelphia and underwent emergency surgery. He is in critical but stable condition.