Historians say hundreds of slaves helped build both the White House and the Capitol.
Their owners were paid $5 a month by the government.
The slaves worked as quarrymen, sawyers, brickmakers and carpenters.
And let's not forget that eight presidents owned slaves while they lived in the White House, including the likes of Jefferson, Madison, and Jackson.
Slaves also contributed mightily toward building the Capitol, where Obama will be sworn in Tuesday.
Professor Charles Blockson of Temple University knows all about this, and he explained that it was a slave by the name of Philip Reid who fitted the legendary statue of freedom on the dome of the capitol.
Barack Obama is not a descendant of African slaves, but his wife Michelle is.
Her great-great-grandfather was a slave at what was called the "Friendship Plantation" in South Carolina.
When Michelle moves into the White House, it will represents a journey that millions of Americans, black and white, will see as a fulfillment of the American ideal.
For Professor Blockson at temple, he put it this way.
"It's an exciting change to witness, to be able to see, to walk the same earth that Obama's walking. I'm honored to be alive," Blockson said.