The stone with the copper plaque honoring so-called "comfort women" was placed in front of the Palisades Park library in 2010. More than half of the town's 20,000 residents are of Korean descent.
Mayor James Rotundo tells The Record newspaper (http://bit.ly/Ml3JOF ) a group of Japanese officials in May visited and asked that the monument be removed.
The mayor says key officials from South Korea visited days later and supported the town for refusing to make any changes.
Petitions for and against the monument have appeared on a White House "We the People" website.
The comfort women issue remains a sticking point between Japanese and South Korean officials.