Speaking at the scene in the Kowloon district, Hong Kong leader Donald Tsang told reporters the men fell after the platform they were working on collapsed in the elevator shaft.
All six workers died, police spokesman Michael Kwan said. The workers were believed to have fallen from around the 30th floor to the 10th floor, he said.
The building's developer, major Hong Kong property company Sun Hung Kai, has agreed to pay each of the victims' families 1 million Hong Kong dollars ($128,000) on top of normal compensation payments, Hong Kong Labor Secretary Matthew Cheung told local media.
Sun Hung Kai did not immediately respond to a request for comment.
Construction accidents in Hong Kong declined by about 76 percent from 1998 to 2008, as the government worked to enhance safety amid a continuous push by developers to build the soaring office and residential towers that have become a hallmark of the city.
Even so, some workers are killed every year, with falls a major cause of death. Between 2000 and 2004, nearly 44 percent of fatal construction accidents involved falls.
There were 16 fatalities in 2006, according to the government.
