The other man, however, told the court he doesn't know what set off the musician and his friends.
"I gave it to him verbally, I said, 'Shut your mouth, man,"' the singer testified in DeKalb County court, adding that while members of his entourage hit Harlen Akins, the musician never landed a punch.
The musician, whose real name is Robert James Ritchie, testified that Akins insulted a woman in Kid Rock's party and asked the entertainer about his ex-wife, former "Baywatch" star Pamela Anderson.
Akins said he was asking the musician's friend Davina Barnes, whom Akins had known for years, to stop calling and texting him about a disagreement they'd had. But he denied insulting her or asking about Anderson and testified that, to this day, he doesn't know why Kid Rock and his friends turned hostile.
"I still don't even know what did I say to him to start this jumping on me," Akins testified, adding he was scared for his life during the fight.
Both men acknowledged that they'd been drinking. Kid Rock said he'd had more than a six pack of beer after a concert in Atlanta before arriving at the restaurant and was celebrating the fact that his latest album had just hit No. 1 on the charts. Akins said he'd smoked marijuana and had two or three shots of cognac at a club.
Akins, an event promoter and caterer, filed suit against Kid Rock and five members of the musician's entourage over the early-morning fight. He is seeking about $6,000 in medical fees and unspecified damages.
Kid Rock pleaded guilty in March 2008 to a misdemeanor charge of simple battery in the case.
The musician testified that he believes Akins was trying to cash in on the fact that he's a big star with a lot of money.
"I feel like I'm a big number in the lawsuit lottery," he said.
As the exchange got more heated, Kid Rock testified, Akins said, "I wish you'd do something to me. I'd get paid."
The physical fight began when Kid Rock's guitarist, Jason Krause, lunged at Akins and hit him when he reached for something on the table of his booth. Krause testified Wednesday that he thought Akins was reaching for his knife.
Akins testified he was reaching for a cell phone to call police.
After a brief scuffle inside, Kid Rock's friends forced Akins out of the restaurant and held the door shut. Akins banged on the window until it shattered. He said he was trying to get someone inside to bring him his cell phones and keys, which were on his table.
"When the window smashes, now I'm freaked out. I'm scared. I'm like, this guy's nuts," Kid Rock testified.
Multiple members of Kid Rock's group testified that they heard Akins say he was going to call his friends to come shoot the place up. Akins denied that.
The musician said he and his friends then went out to get on their tour bus. Akins said they came out and kept beating him before leaving on the bus. The bus was pulled over by police a short distance away.
Kid Rock testified that he and his friends were acting in self-defense and only used as much force as was necessary.
"If you look at me, Bossy, Jason and Bear," he said referring to members of his entourage who are relatively big men, "if we wanted to hurt this man, we could have hurt him very, very bad."
Several doctors testified Wednesday that Akins had injuries that were consistent with having been in a fight. A firefighter who drove him to the hospital testified Tuesday that he didn't note anything about Akins being bloody in his report and found no signs of internal bleeding.
Eric Hertz, one of Akins' lawyers, brought up other fights Kid Rock has been involved in: punching a DJ at a Nashville strip club and punching rocker Tommy Lee, another Anderson ex, at an awards show.