So when that famously changed to Democrat, his identity was gone, and soon after, so was his seat in the Senate.
Now, Citizen Specter is sounding off in this new book. And in case you're wondering where he stands now, I asked him.
"I'd say dead center," Specter told me.
And that was precisely the problem. In 2010, facing the fight of his electoral life, Arlen Specter was a man without a political home.
Isolated by Republicans for his vote in favor of the economic stimulus package, he joined the Democrats, and made no bones about why.
"I couldn't run in a Republican Primary and win," he said. "I wanted to stay in the Senate."
It was a high stakes gamble that didn't pay off. Specter lost the Democratic Primary to retired Admiral Joe Sestak, who in turn lost to Conservative Republican Pat Toomey.
Was Specter devastated by the loss? No. Disappointed by how it happened? Absolutely.
Hence the title of his new book: 'Life Among the Cannibals.'
"In a sense they devoured me," he said.
The cannibals to which he refers are the Tea Party: in his words, the extreme conservatives destroying not only the Republican party, but the political process along with it.
And if he's pessimistic about their influence, Specter may be as doubtful about the next president's ability to stop it.
"I've not been happy with President Obama," he said. "A lot of statements are made for political expediency, and I think Romney is loaded with them. You take the lesser of the evils. At the moment I'm not happy with anybody, but I will vote, and I'll choose the better of the nominees in my judgment."
The book has every bit the candor that sometimes landed Senator Specter in trouble.
For instance, there is one passage in which he writes about meeting Sarah Palin, and recalls that she "radiated sensuality." But it's the ugly side of politics he really sets out to detail.
"I don't care if people buy this book, although you can get it at amazon.com and Barnes and Noble," he said, laughing. "I want people to read it. I want to change the political system in America."
You can catch more of the senator's candid thoughts after life in Washington, as he joins Matt O'Donnell on Inside Story. That airs Sunday March 25 at 11:30 a.m. on 6abc.
