But it turns out that what fell to the ground was nothing compared to what fell backwards onto the building's rooftop.
Most of a wall, called a parapet, collapsed harmlessly onto the roof. However, some debris rained down on the sidewalk.
"Kids or anybody could have been walking past. Bricks could have fallen on them. That's dangerous," said Lauren Savage of West Philadelphia.
A witness claimed it narrowly missed at least one pedestrian.
This type of thing has happened before in Center City.
In June, 2008, ornate terra cotta smashed down on cars at 23rd and Walnut. Last January, bricks fell from the 15th floor of the Bellevue. Two months ago, at 1619 Walnut, a granite facade plunged five stories.
Some cities require owners to have periodic structural inspections of older buildings. But not here, says George Folkman. His restoration firm deals with collapses.
"Philadelphia, unlike Chicago and New York, has no inspections. Just like your car inspections, if not told to take your car for inspection you will just correct things when they have to be corrected, or after they fail," Folkman said.
In Philadelphia, it is up to building owners to inspect. There was talk of requiring inspections in the wake of a 1997 collapse that killed a man, but nothing changed.
Now, L and I says it's something they will revisit.
With four collapses in 14 months, some watching Wednesday's cleanup wondered what officials are waiting for.
"Someone to get hurt seriously or killed to fix the problem. A little to late then, right?" said James Harris of West Philadelphia.