Lidge bobbled a sacrifice by Matt Diaz before making a wild throw to first that bounced into foul territory in right field, allowing Anderson to score the tying run as Diaz advanced to third.
Lidge then issued an intentional walk to Adam LaRoche and walked pinch-hitter Greg Norton to load the bases with no outs. Ryan Church struck out before Infante's single to left.
Brian McCann hit a two-run homer off Cole Hamels in the third to give Atlanta a 2-0 lead. Peter Moylan (5-4) pitched a perfect ninth for the win.
Jayson Werth gave the Phillies a 3-2 lead in the seventh with his 24th homer, matching his career high.
Hamels gave up two runs and three hits, walked four and struck out two in six innings. He did not allow a hit in his final three innings.
Six of the Phillies' seven hits off Kenshin Kawakami were for extra bases. Kawakami gave up three runs in 6 1-3 innings.
The Phillies scored in the fourth and fifth to tie it at 2.
Shane Victorino tripled and scored on Ryan Howard's grounder to second baseman Martin Prado, who threw out Howard from shallow right field. Raul Ibanez led off the fifth with a double to right and scored on a triple to left-center by Pedro Feliz.
Victorino caught two deep fly balls with his back against the center-field wall - by Yunel Escobar following McCann's homer in the third and by Chipper Jones in the fifth.
The Phillies stranded a runner on third in the fifth, seventh and eighth innings.
NOTES: Braves CF Nate McLouth (left hamstring) and Prado (heat) left the game early. ... Braves closer Rafael Soriano, who gave up Howard's go-ahead homer in the ninth inning on Friday night, was unavailable due to soreness behind his right shoulder. ... Phillies RHP Brett Myers was scratched from his scheduled rehab appearance with Class A Clearwater due to swelling around his eye. He said he fell while getting out of his truck on Friday night. He initially told general manager Ruben Amaro he suffered the injury while playing catch with his 4-year-old son.