"I'm excited for this next chapter of my career," said Girardi. "The Phillies have a strong commitment to winning from the owners to the front office to the players to the fans. It's something that I've seen up close for the last 30 years of my baseball career.
Girardi, 55, also interviewed with the New York Mets and Chicago Cubs for their openings.
Girardi replaces former manager Gabe Kapler who was fired on October 10 after two seasons at the helm.
"On behalf of everyone at the Phillies, I want to congratulate Joe and welcome him to our baseball family," said Klentak. "Joe brings high character and a tremendous work ethic to his position, and he is a proven winner. I look forward to working with him and I believe that he is the right manager to lead our team to the next level."
Phillies managing partner John Middleton is hoping new leadership can push the Phillies into the postseason next season.
The Phillies were 20-36 in two Septembers under Kapler. They were only two games behind in the NL wild-card standings after beating Atlanta on Sept. 18 this season but lost eight of the next nine, including a five-game sweep at Washington. They finished eight games behind Milwaukee for the second wild card and 16 behind the first-place Braves in the NL East.
Last year, the Phillies had a one-game lead in the division on Aug. 11 before going 15-31 the rest of the way.
"Matt did a great job running the search, culminating with the three exceptional candidates we interviewed," said Phillies managing partner, John Middleton. "Ultimately, we all agree that Joe is the right person to lead our team, and I am excited to welcome him to the Phillies."
While with the Yankees, Girardi managed the club to a postseason appearance in six of his 10 seasons and won the franchise's 27th World Series in 2009 in a six-game series against the Phillies.
Girardi will wear No. 25.
ESPN contributed to this report.