The store's owner said the thief gave back the mandolins after surveillance video of the crime went viral.
The theft happened at Lark Street Music in Teaneck, Bergen County.
Owner Buzzy Levine said security cameras captured a man taking two mandolins from the wall and tucking them into his jacket.
After calling police, Levine posted the video online.
His children told him the post racked up more than four million views.
Four days later on Dec. 26, Levine said a quiet visitor slipped into the store and left something behind.
"I noticed someone opening the door, bending down and then splitting. So I walked over - in the shopping bags were the two mandolins. I couldn't believe it," Levine said.
A handwritten apology note was left with the instruments. It read: "Sorry, I been drunk. Merry Christmas," and "You are good man."