Frank Breneman, 71, of Lititz, Pennsylvania, was charged on Wednesday with rape of a child and sexual assault.
He is accused of sexually abusing at least two victims when they were between the ages of 4 and 11 years old, according to police.
Police say the crimes occurred between 2004 and 2016.
Pennsylvania State Police say they are worried there are more victims.
Officials say some of the victims who have come forward were former foster children of Breneman's and were abused while living in his home.
Breneman was arraigned and remanded to the Lancaster County Prison in lieu of bail.
Anyone with information regarding the case or conduct of Breneman is asked to call Pennsylvania State Police at 717-299-7650.