Neighbors describe the abandoned mansion as a headache and safety hazard. The home is now covered in graffiti from taggers, and squatters have been coming and going for more than two years.
One neighbor said 911 has been called several times just in the past week.
"Every time we call, the police will come and they'll clear it out, and then within an hour or two, five more people are there and staying the night," said the neighbor, who didn't want to be identified.
The neighbor said things at the property have gotten violent in the past.
ALSO SEE: Woman charged in case of deed to $4M home switched without homeowner's knowledge
"Apparently, some squatters came and actually attacked one of the neighbors with steel rebar and beer bottle," the neighbor said.
Our sister station KABC captured what appeared to be three taggers leaving the home Thursday evening, with private security stepping in to get them out.
The Los Angeles Police Department says officers have been called to the home six times this month for trespassing and vandalism. Police removed 10 people on Wednesday and arrested one person on a warrant.
"The police told us they are tied because it's still owned by the homeowner, so it's stuck in limbo land," the neighbor said.
According to public records, the owner is John Powers Middleton, the son of Phillies owner John S. Middleton.
The Phillies confirmed the property belongs to the team owner's son and told Action News that no other members of the Middleton family have ownership or involvement in these properties.
"The California properties are owned by John Powers Middleton. No other members of the family have ownership, investment, control or involvement in these properties," the Phillies said in a statement.
Neighbors also say they've reached out to L.A. Councilwoman Nithya Raman's office but haven't had much luck with a resolution.
KABC reached out to Raman's office but has not heard back. Calls and text messages were sent to numbers associated with the mansion's owner, but there has been no answer.
Neighbors say they will continue fighting for a fix, adding that they have even tossed around the idea of possibly coming together as a group to buy the property.