Castle is also a former two-term governor and the longest serving congressman in state history. He lost the GOP primary earlier this month in a stunning upset to tea party favorite Christine O'Donnell.
He issued a statement Wednesday night saying he had ruled out a write-in campaign, which many supporters had asked him to consider. The decision brings an end to a political career stretching over four decades.
Castle had until Thursday afternoon to declare his candidacy in writing.
His decision leaves O'Donnell to face Democrat Chris Coons in November for the Senate seat long held by Joe Biden before he became vice president.