The agency said in a statement early Tuesday that one of the jumpers was a construction worker at the trade center site.
The Port Authority, which owns the site, says "the spirit of respect and reverence for this sacred site" was violated.
Four men were arrested Monday in connection with the "base jumping" on Sept. 30. The three daredevils and someone accused of being an accomplice on the ground were arraigned on charges of burglary, reckless endangerment and misdemeanor jumping from a structure.
Police Commissioner William Bratton says "being a thrill-seeker does not give immunity from the law."