With nine Australians playing in the NBA, including rookie superstar Ben Simmons, Philadelphia 76ers coach Brown, who was involved in three Olympics with the Boomers, insists the future is bright.
"I think the next Olympic Games will be the best chance to win a medal," Brown told AAP in London ahead of his side's NBA clash with Boston Celtics.
"There are some great kids coming out of high schools in the United States that are Aussies.
"And if you look at the WNBA and the volume of women that play, there are over 160 NCAA Australian athletes littered throughout the American 1, 2, 3 division system.
"It's a blueprint of how a country should be run basketball-wise. It's a little business that is really well run."
The engaging Brown spent 17 years involved in the Australian game, coaching 278 NBL matches and was Boomers head coach at the 2012 Olympics and an assistant at the 1996 and 2000 Games.
He remains deeply passionate about Australian basketball despite being based back in the US since 2002, helping lay the foundations for San Antonio's four championships between 2003 and 2014.
His current role as head coach of the 76ers will see him build on Simmons' rich potential.
Brown believes his young point guard is the man to lead the Boomers to success having missed out on a bronze after a gut-wrenching one-point loss to Spain in Rio two years ago.
"I believe Ben is going to be the tipping point of those teams that can win a medal and I hope he can," he said.
"I've been to three Olympics and I've seen how important and what a privilege it is to experience and I want him to share in that.
"It's a lifelong memory that is different to NBA basketball.
"The national patriotism, living in the village, the opening ceremony, seeing the fastest man on the planet in the food hall. It matters and I hope he can experience it."
Simmons has enjoyed a strong rookie season after sitting out last year due to a foot problem.
Brown, who also coached Ben's father Dave in the NBL, said it is not only Simmons' on-court play that has stood out but also his maturity.
"What impresses me the most is that he doesn't seem fazed, his Dad was like this, very reserved, not beating his chest," he said.
"He's not a big towel swinger. I think the poise with which he's handled things is incredibly impressive.
"We've experienced teams purposefully fouling him, and he's got to walk to a line in front of national TV and 20,000 people, and he doesn't blink."
Baynes: Why not reach for the gold medal?
Boston Celtics and Boomers big man Aron Baynes is confident about Australia's chances at the 2020 Olympic Games, given their talent currently playing in the NBA.
Simmons: I know I'm the best rookie in the league
Ben Simmons speaks about his aspirations for the rest of the season including making the All-Star team and representing Australia in the Olympics in 2020.