But Berger called Crowder "a prime suspect" in a string of fires intentionally set around the city Sunday.
Firefighters continued to make gains against the flames Thursday, allowing some schools to reopen and some area residents to return.
Investigators recommended Crowder be charged with three counts of intentionally setting fire to public land - each count carrying up to three years in prison.
Crowder told reporters watching as police led him to a transport van that he may have accidentally sparked a fire, and ran because he was driving without a license.
"I believe that I accidentally may have - may have - started by tossing a cigarette out the door," said Crowder.
Kepler Funk, who represented Crowder at a short bond hearing, said he fears the public has rushed to judgment against his client. He said police were still looking for other suspects, and Crowder hadn't been accused of the destructive fires.
"People are losing their homes. It's serious, it's horrible, it's tragic," Funk said. "But it's too quick for anyone to jump to the conclusion that this is the man that is in fact responsible for all that devastation."
Records show Crowder has drug, burglary and automobile theft convictions dating back to 1996.
Berger said investigators on Thursday began interviewing everyone in Brevard County with a history of arson arrests.
Todd Schroeder, spokesman for the state's Division of Forestry, estimated the fires were 60 percent contained, up from 40 percent contained on Wednesday. Winds had died down and humidity was up above 90 percent.
"People are starting to feel a little bit more comfortable again," Palm Bay Councilman Ed Geier said. "There's no big black clouds in the sky. It's beautiful blue up there."
Elsewhere, scattered fires were still burning around the state. A total of almost 26,000 acres - 40 square miles - were ablaze as of Thursday, according to an emergency management report.
Aside from the fires in Palm Bay and Malabar, the majority of the fires were in Glades County. In an area around Lake Okeechobee, roughly 11,000 acres had burned or were still burning, though no structures had been damaged.
In California, crews were trying to make more progress against a stubborn 340-acre wildfire. The blaze on Mount Baldy, 45 miles northeast of downtown Los Angeles, was being fed by winds reaching 40 mph. The fire was 25 percent contained and had moved sluggishly away from populated areas since it began Tuesday.
Firefighters also fought a brush fire in central Los Angeles' Griffith's Park and a grass fire in the San Fernando Valley's Sepulveda Basin Wildlife Reserve.
