Sherrod and Agriculture Secretary Tom Vilsack said that she may work with the agency in a consulting capacity in the future to help it improve its outreach to minorities. She told reporters she did not think she could say yes to a job "at this point, with all that has happened."
"I look forward to some type of relationship with the department in the future," she said. "We do need to work on the issues of discrimination and race in this country."
Vilsack, who apologized to Sherrod for pushing her out, had offered her a new position in the Office of Advocacy and Outreach, which is works on civil rights issues.
"I think I can be helpful to him and the department if I just take a little break and look at how I can be more helpful in the future," Sherrod said.
Vilsack said that "Shirley has unique opportunities here."
Vilsack said he had worked hard to get Sherrod to return.
Formerly the agency's director of rural development in Georgia, Sherrod was forced to resign after a conservative blogger posted snippets of a March speech in which she appeared to make racist remarks. Vilsack and others, including the NAACP, condemned the remarks before grasping the full context of her speech, which was meant as a lesson in racial healing.
The incident proved embarrassing for the Obama administration, and President Barack Obama called her personally to express his regret.
The NAACP also apologized for its reaction.
Sherrod repeated Tuesday that she plans to sue the blogger who posted excerpts of her speech, Andrew Breitbart. But she declined further questions on the subject.
Associated Press writer Ben Evans contributed to this report.