Hobbled Josh Hamilton put Texas ahead with an RBI double in the first inning. Then Mike Napoli broke it open with a three-run homer in the sixth on the first pitch from reliever Mitchell Boggs, who came in after loser Edwin Jackson walked his seventh batter.
Holland walked Rafael Furcal with one out in the ninth, and Neftali Feliz got two outs, including Albert Pujols with two runners on.
Game 5 is Monday night at Rangers Ballpark, a rematch of the opener, when Cardinals ace Chris Carpenter topped C.J. Wilson.
A night after the Cardinals won a 16-7 slugfest as Pujols tied a record with three home runs, Holland restored some of the pitching dominance of the first two games.
Holland, 2-0 in 10 career postseason appearances, allowed only four baserunners: Lance Berkman doubled in the second and singled in the fifth, and Nick Punto and Furcal in the sixth and ninth.
Holland struck out seven. He got assistance from third baseman Adrian Beltre, who made a leaping catch on Furcal's liner starting the game.
Jackson struggled with his control after left after 5 1-3 innings and seven walks - three short of Bill Bevens' Series record from 1947. Jackson went to three-ball counts on four of his first 10 batters but mostly escaped trouble with the help of some long outs - four flyouts at or just in front of the warning track.
He walked Nelson Cruz and David Murphy with one out in the sixth, and Napoli greeted Boggs by sending a 95 mph fastball just inside the left-field foul pole, about 10 rows deep. In their first-row seats, former President George W. Bush and Rangers CEO Nolan Ryan exchanged a high-five. On the mound, Boggs kicked and grimaced.
Jackson gave up three runs and three hits, striking out three, and threw just 59 of 109 pitches for strikes. He didn't allow a hit after the second inning.
Following his 5-for-6 performance with six RBIs in Game 3, Pujols grounded out and fouled out in his first two at-bats, coming up with no runners on base.
After getting pounded Saturday night, Texas was hoping to become the first team to tie the Series 2-2 since the 2003 Florida Marlins against the New York Yankees. The Rangers had not lost consecutive games since Aug. 23-25 against Boston.
Texas was ahead after 10 pitches from Jackson, ending the Cardinals' streak of scoring first in 10 straight postseason games, one short of the record set by Detroit from 1972-84.
Elvis Andrus singled sharply to left with one out and Hamilton, just 1 for 12 (.083) coming in, doubled down the right field line. Two walks loaded the bases with two outs, and Murphy flied out to Matt Holliday a couple steps in front of the left field warning track, ending a 25-pitching inning.
After his throwing error at first base Saturday night led to two runs, Napoli moved behind the plate and Mitch Moreland started at first. Napoli was dropped to eighth in the batting order, between Murphy and Moreland, to split the righties.
Moreland wound up helping save a run in the second. Berkman doubled to the right-center gap with one out and, after David Freese struck out, Yadier Molina hit a grounder off the front of the mound. Ian Kinsler ranged to the shortstop side of the mound, gloved the ball and made an off-balance throw to first, where Moreland scooped it.
Kinsler singled sharply with two outs in the second, then got caught off first by Molina and was tagged out in a rundown for a caught stealing.
Jackson walked two more with two outs in the fourth, then struck out Moreland.
For the second time in three years, baseball and NFL teams from the same cities met on the day of a World Series game, with the Dallas Cowboys defeating the St. Louis Rams 34-7 across the street. In 2009, the Yankees won 7-4 at the Phillies as the Eagles routed the Giants 40-17 across the street in Philadelphia.
In a rematch of the opener, won by the Cardinals 3-2, Chris Carpenter starts Game 5 for St. Louis on Saturday night and C.J. Wilson goes for Texas.
Thirty-six of 42 teams to win Game 4 for a 3-1 Series lead have gone on to win the title. Twenty-two of 40 teams to win Game 4 and tied the Series 2-2 have gone on the championships.
NOTES: Bush - a former Rangers owner - threw the ceremonial first pitch to Ryan, the Hall of Fame pitcher. Bush stood in front of the rubber, and his ball went over the right-handed batter's box, bounced off Ryan's catcher's mitt and glanced off a photographer. Rangers manager Ron Washington and players heckled Ryan.