His monumental piece commissioned specifically for the fair - a mural depicting mug shots of the New York Police Department's 13 most-wanted criminals - was deemed too edgy for the family friendly event and was painted over just before opening day.
Now, 50 years later, the work is the focus of a museum exhibition being staged on the very fairgrounds where the pop-art provocateur was censored.
"There's no question Warhol was not interested in the notion of a family friendly fair," said Larissa Harris, the exhibition's curator. "It's possible that he understood the concept very clearly, but he did this absolutely intentionally."
The exhibition, "13 Most Wanted Men: Andy Warhol and the 1964 World's Fair," opens April 27 at the Queens Museum in Flushing Meadows Corona Park. It includes never-before-shown archival documents and materials, including the artist's letter agreeing to the paint-over and the NYPD mug shot booklet.
Warhol was one of 10 artists commissioned by famed architect Philip Johnson to create 20-foot-by-20-foot artworks for the outside of the New York State Pavilion's Circarama theater.
Gov. Nelson Rockefeller, who was seeking the Republican presidential nomination at the time, gave the order to paint over Warhol's mural. The stated reason, according to Johnson and others, was because seven of the 13 criminals were Italians and he didn't want to risk alienating his Italian constituencies, Harris said.
"It does seem like an incredibly bold step," said Nicholas Chambers, a curator at The Andy Warhol Museum in Pittsburgh, which is collaborating on the exhibition. "In retrospect, it seems that was the only possible outcome - that it had to be censored."
According to his autobiography, Warhol believed the work was whitewashed "because of some political thing I never understood."
Warhol offered another work as a replacement - 25 identical portraits arranged in a grid of the fair's controversial head Robert Moses - but it was rejected by Johnson as inappropriate.
Warhol never made another public work.
The documents show that Warhol agreed to have the mug shot mural painted over in silver. (He was already producing paintings that had large silver blank panels.) Removing it wasn't an option because it would have ruined Johnson's vision for the building's exterior that called for a combination of black-and-white and colored pieces.
Three months later, Warhol took the screens he used for the mural and created "13 Most Wanted Men" on canvas.
Nine of the 13 original canvases - assembled from various collections - are the subject of the exhibition.
The goal of the show is to delve into the mural's creation and the reasons Warhol chose to make "13 Most Wanted Men" for the World's Fair.
"Warhol loved experimenting with how far he could go," Harris said. The 36-year-old artist, known for his Coke bottles, soup cans and celebrity portraits, was also creating silkscreens of suicides and car crashes at that time.
"Andy was interested in this darker side to American media culture," Chambers said. "There are these kinds of connections to tragic events even in some of the more iconic celebrity portraits that he was doing at that time. ... The Marilyn Monroe portrait for instance, was produced shortly after her suicide."
The exhibition runs through Sept. 7 and then goes on view at the Warhol museum from Sept. 27 to Jan. 5.