Kelly scored midway through the first. Alfredsson put the Senators up by two with his 18th goal 18 seconds into the third.
The Senators strengthened their hold on fifth place overall in the Eastern Conference with 83 points, four ahead of Philadelphia and Montreal.
Flyers left winger Simon Gagne got 19 minutes in penalties in the wake of Alfredsson's goal when he jumped Anton Volchenkov after the Ottawa defenseman checked him into the boards from behind earlier in the play.
Gagne got a double minor for instigating a fight with a visor, a major for fighting and a 10-minute misconduct. Volchenkov was given a roughing minor. The Senators failed to score on the ensuing 7-minute power play.
Brian Boucher made 24 saves in his fourth straight start for the Flyers, who lost their third in a row.
Philadelphia, tied with Montreal for sixth place in the Eastern Conference with 79 points, lost leading scorer Jeff Carter to a broken foot in Sunday's 3-1 loss to Atlanta. Carter is out for three to four weeks after he was struck by a shot. The injury ended Carter's streak of 286 consecutive games played - the third-longest in team history.
Ottawa, which went 0 for 7 with the man advantage, had already failed to convert three power-play opportunities when Kelly opened the scoring midway through the first. The Senators center held onto the puck on a 2-on-1 rush with Andy Sutton and fired a wrist shot past Boucher from the right side 10:34 in for his 14th goal.
Boucher made a strong glove save on Nick Foligno in the second just seconds after the Flyers killed Ottawa's fourth power play.
Kelly was called for hooking Gagne 9:06 into the second just as Volchenkov drove Daniel Carcillo into the boards when the Flyers left winger carried the puck along the right side.
Philadelphia failed to score despite enjoying a 5-on-3 advantage for a full 2 minutes when Sutton was sent off for roughing Carcillo at 15:26. A delayed penalty had already been signaled against Erik Karlsson, who tripped Carcillo in retaliation for a hit in the corner.
Gagne had a great scoring opportunity late in the period, but shot high after Elliott slipped and fell.
NOTES: Referee Bill McCreary immediately waved his arms to signal no goal when the puck ended up in the net 2:23 in after Nick Foligno swatted it past Boucher with his glove. The Flyers also failed to get credit for a goal in the third period after a lengthy video review. ... It was Elliott's sixth career shutout.
