Murphy, a Democrat, announced the change during a news conference on COVID-19. He said the change depends on there not being any "dramatic" deterioration in coronavirus trends.
Students were required by state guidance to wear masks in their schools, except in some cases. Murphy faced pressure from some parents who opposed the mandate in recent months and announced school districts had the ability to relax mandates.
"After a school year where the majority of our students spent significant time learning remotely, the upcoming school year will see a return to normal," the governor said.
The new school year will also mark the return to full-time in-person education. Many schools started the year remotely but then returned to some in-person education.
Murphy has long said he planned for the fall term to begin fully in-person.
The relaxed mandate comes as New Jersey continues to outpace much of the country in vaccinations, with 67% of adults fully vaccinated. Fifty-seven percent of adults nationwide are fully vaccinated, according to the Centers for Disease Control and Prevention.