Jakub Voracek scored the deciding goal for Philadelphia, and Brian Elliott made 33 saves. Ivan Provorov had two assists.
"We want to keep pushing here, there is no give up from our group," Elliott said. "Tomorrow is another big test and we just got to respond the right way and comeback with another gutsy effort."
Brendan Smith and Artemi Panarin scored New York, and Igor Shesterkin made 33 saves. The Rangers fell for the second straight game after four consecutive victories against the New Jersey Devils.
The teams will finish the eight-game season series Friday night in New York.
"We know what's at stake every game," Rangers defenseman Adam Fox said. "At this point there are only so many left, but we're going to keep fighting until we're officially maybe out of it. Boston winning, us losing, it hurts, but we've got a resilient group in there and I think we'll just keep fighting and take it game by game."
Former Rangers forward Kevin Hayes won an offensive-zone faceoff prior to van Riemsdyk deflecting Provorov's shot from the point at 10:29 of the third period to give the Flyers a 2-1 lead. It was the second time in six chances Philadelphia converted on the man advantage.
"Most of his goals are second opportunities in front of the net, tip ins, screens," Flyers coach Alain Vigneault said of van Riemsdyk. "That's where you score the goals. You got to get in front of that net, near that blue paint. He certainly understands that. When he is on top of his game, he is doing that extremely well."
Voracek extended the Flyers' lead to two goals with a wrist shot that sailed past the glove of Shesterkin at 14:07. Sean Couturier and Shayne Gostisbehere assisted.
Philadelphia evened it at 1 late in the second period with an odd deflection off the face of van Riemsdyk.
Travis Konecny's shot deflected off Rangers defenseman K'Andre Miller's stick before hitting van Riemsdyk and trickling into the back of the net. The power forward immediately went to the locker room following the play but would return early in the third period.
"Ultimately, I know that's my bread and butter, being in front of the net especially on the power play and being a productive player there," van Riemsdyk said. "But, it's that fine line of just standing in front of the goalie and also just playing hockey and being a part of making more plays."
Smith opened the scoring with a slap shot from the top of the right circle shortly after New York's power play expired at 9:55 of the first period. Miller and Filip Chytil assisted on the play.
Zac Jones picked up 9:45 of ice time in his NHL debut Thursday. Jones recently turned pro after leading the University of Massachusetts to the college championship this season. New York selected Jones in the third round of the 2019 NHL draft.
"I thought he got better and better as the game went on," Rangers coach David Quinn said of Jones. "Not an easy situation to be put in and I thought he handled himself well."
Elliott's best save of the game came midway through the second period to keep the Flyers deficit at one. Ryan Strome slid the puck from the slot over to Panarin before Elliott made a lunging glove save.
"There is no doubt when a goaltender does that for you it brings a tremendous amount of confidence to the rest of the group," said Vigneault. "When your goaltender is making the saves, it permits you to go out there and feel that there are some plays that you might be able to make and play a better game."
Joel Farabee had an opportunity to give the Flyers a one-goal advantage late in the second period when he was awarded a penalty shot after being taken down on a breakaway. Farabee tried a series of dekes that resulted in a backhand chance that Shesterkin calmly turned away.
Panarin trimmed the Flyers' lead to 3-2 at 18:00 of the third period with a slap shot from the left point.
The Flyers had their first three-day break in game action in over two months since a pause in their season in mid-February due to COVID-19 prior to the victory against the Rangers. This was the third time this season they have had three days or more between games.
The Blueshirts are the only NHL team that has five players who have 40 or more points this season (Pavel Buchnevich, Fox, Panarin, Ryan Strome and Mika Zibanejad). In addition, the Rangers are one of only two NHL teams that have four players who have 16 or more goals in 2020-21 (along with Florida; Chris Kreider, Buchnevich, Mika Zibanejad, Panarin).
The Rangers are currently in the midst of a 17-game stretch where they solely play in the New York Metropolitan area. The stretch began April 6 and is scheduled to end May 6 when the the Rangers travel to Boston.
The Associated Press contributed to this report.