The change is aimed at "getting fresh thinking, fresh eyes on the problem," Gates told a Pentagon news conference.
The move comes as more than 21,000 additional U.S. forces begin to arrive in Afghanistan, dispatched by Obama to confront the Taliban more forcefully this spring and summer.
Replacing McKiernan will be Lt. Gen. Stanley McChrystal, who has had a top administrative job at the Joint Chiefs of Staff for less than a year. He is a former commander of the Joint Special Operations Command.
McKiernan held the command in Afghanistan for about 11 months.
Gates recommended to Obama that McChrystal be nominated for the top job and that Lt. Gen. David M. Rodriguez be McChrystal's deputy. He urged the Senate to confirm the two quickly.
Obama had approved 17,000 additional combat forces for Afghanistan this year, plus 4,000 trainers and other non-combat troops. By year's end, the United States will have more than 68,000 troops in the sprawling country - about double the total at the end of Bush's presidency but still far fewer than the approximately 130,000 still in Iraq.
McKiernan and other U.S. commanders have said resources they need in Afghanistan are tied up in Iraq.
Asked if McKiernan's resignation ends his military career, Gates said, "Probably."
Get Action News on your website