The incident happened on the 22nd floor of an apartment building on East 64th Street.
Police say maintenance work was being done, allowing the man to gain access to the empty apartment.
He reportedly climbed over the balcony railing and was threatening to jump when officers managed to grab him and pull him to safety.
He was taken to Cornell Medical Center.