The competitor "experienced distress" during a 2.4-mile swim in the Hudson River at the start of the all-day competition, a publicist for the race organizers said. The course ran along the New Jersey shoreline, just north of the George Washington Bridge.
The swimmer was pulled out of the water and taken to a hospital in nearby Englewood Cliffs, N.J., but did not survive. The organizers said the cause of death is unknown. An autopsy is planned.
New York City police said the contestant was a 43-year-old man. His name has not yet been released.
"On behalf of all of us in the triathlon community, we mourn his death and send our condolences to his family and loved ones," organizers said in a statement.
Contestants in the race followed their swim in the Hudson with a 112-mile bicycle ride through the suburbs, and then a 26.2-mile marathon that finished at Manhattan's Riverside Park.
Jordan Rapp, a winner of multiple Ironman titles, won the race in an unofficial time of 8 hours, 11 minutes and 18 seconds.
Earlier in the week, officials had warned of partially treated sewage that was being discharged into the Hudson River while a broken pipe was fixed upriver in suburban Westchester County. Eventually, the discharge was stanched, and tests were done to determine the race course was safe for swimming, organizers said.
Deaths in triathlon competitions have happened with regularity in recent years, almost always during the swim, and the Hudson River has been particularly dangerous.
Two people died during the swim portion of the Nautica New York City Triathlon last summer. A male competitor also died during the New York City Triathlon in 2008.
After a spate of five deaths around the country in two months last summer, a governing body for the sport, USA Triathlon, created a task force to examine the fatalities.
The swim always comes at the beginning of the triathlon, when athletes are most fresh, but the field is often intensely crowded at that stage of the race. Competitors often report feeling panicked as they bump and thrash for space in sometimes frigid water.