The $250 million apartment building was planned for the north end of the arena.
Officials said the building was slated to have nearly 400 apartments with a portion designated for affordable housing.
But Councilmember Mark Squilla said some members of the Chinatown community did not see this as a development that was affordable or benefited them.
The plan to eliminate the tower came at Squilla's request.
The Save Chinatown Coalition issued a statement saying that the city is "letting rich developers off the hook despite housing being a central mission of the city."
According to the agreement, the Sixers will finance the entire $1.3 billion project. No city funding is slated to be involved.
READ MORE: More details revealed about Parker's Philadelphia 76ers' arena deal | What you should know
Last month, Mayor Cherelle Parker, who has thrown support behind the plans, detailed the proposal in front of a packed Pennsylvania Convention Center.
Demolition is expected to start in 2026 if the plan is approved by City Council. Construction would then start in 2028 followed by an opening in 2031.
Read the full statement released by Save Chinatown Coalition on plans to scrap the apartment building:
"The billionaires behind the arena don't care about affordable housing and the tower was always a PR sham. Councilman Squilla gave fuel to their lies and is pushing the Mayor's disgraceful deal that secures exactly $0 for the housing trust fund and affordable housing. They're letting rich developers off the hook despite housing being a central mission of the city and a stated priority for this Mayor. The city's getting taken for a ride, and Mayor Parker and Councilman Squilla are helping push the cart, lowering the bar for every development yet to come."