The dog, described as a pit bull or pit bull mix, bit Nephi Selu of Dixon on top of his head around 11:30 a.m. Monday at his grandparent's home in Union City, police said.
A relative pulled the dog off the boy, Union City Police Cmdr. Ben Horner said. Emergency crews treated the boy at the scene and took him to Lucile Packard Children's Hospital in Palo Alto.
The boy later died from his injuries around 4 p.m. Monday at the hospital.
Family members told investigators the boy was playing with the dog and may have attempted to climb on its back when he was attacked, Horner said.
The dog was taken to a local animal shelter for a routine, 10-day quarantine, Horner said. The dog was in compliance with his anti-rabies shots and licensing requirements, Horner added.
A "Beware of Dog" sign was posted on a fence next to the two-story, beige house, but neighbors who have seen the dogs out and about said they were always on a leash and gave no hints that they might be vicious.
"It's such a shame," neighbor Arthur Kouns, 41, told the Bay Area News Group. "The dogs always appeared very friendly."
The San Francisco Chronicle reports that a man who answered the phone at the home declined to comment.