The altercation happened around 10 p.m. Tuesday at the city's Curran-Fromhold Correctional Facility, which has nearly 2,800 inmates.
While investigators weren't sure what caused the fight, it went down inside a building where narcotics had been found during a shakedown earlier in the day, according to Lorenzo North, president of Local 159, which represents the 500 corrections officers at the prison.
The movement of inmates at the prison was being restricted Wednesday, but the prison was not locked down, North said. The search earlier on Tuesday had found homemade weapons and drugs in the same section of the prison where the fight happened.
The 42-year-old inmate who died had been stabbed several times, according to police. A 24-year-old inmate was in critical condition Wednesday, while a 24-year-old inmate was in good condition and a 28-year-old was stable. No officers were hurt.
A prison spokesman had no immediate comment.