More often than not, the story circulates around Simmons' deficiencies rather than his strengths.
For Brown, that angle has become more than a little tiresome.
"I pinch myself when I say he's 23 years old, I mean really, the stuff that he does -- he's incredibly maligned sometimes to a point you just learn to ignore stuff," Brown said in Milwaukee prior to the Sixers' loss to the league-leading Milwaukee Bucks.
"He's 23 -- he's a two-time All-Star, he was a college 4-man that I made an NBA point guard because he didn't come into a program with Chris Paul. So, it was like 'What can you do ... oh, you're going to be the point guard.'"
Simmons has quickly become the engine of the Philadelphia offence, with his elite passing ability and combination of size and athleticism setting him apart.
Of course, critics will continue to point to Simmons' lack of a reliable jump shot as a reason why the Aussie will never reach his full potential, but Brown likened Simmons' weaponry to that of reigning NBA MVP Giannis Antetokounmpo.
"His ability to find people, his vision line at 6-foot-10 is different than 6-foot-2 guards," Brown said.
"There's nobody in the league with breakaway speed any faster. Giannis and Ben Simmons, you are going to look at two amazing athletes ... that have changed the game that aren't American.
"His better days are completely yet to come once he conquers a little bit of Giannis' path of the shot and so on, his future is just ... there's no ceiling, he can be what he wants to be."
Ranked fourth in the league for transition frequency (30.1%, min. 35 games played), Simmons plays at one speed: breakneck.
And for all the questions on the fit between Simmons and Philadelphia big man Joel Embiid, the loss to Milwaukee was the perfect illustration of the Sixers' reliance on the 2016 No. 1 draft pick.
Having missed the previous game with lower back tightness, Simmons looked sore from the outset and would last only 4:44 of playing time before being ruled out.
Without Simmons, Philadelphia's lack of playmaking was strikingly clear, as the No. 1-ranked Bucks defence suffocated the Sixers into 35% shooting from the field en route to a deflating 119-98 loss that sank the supposed contenders to an abysmal 9-20 on the road.
Milwaukee coach Mike Budenholzer didn't downplay the hole Simmons left when he was forced off the court during the Bucks' win.
"He's just so fast in transition, he gives them speed to the basket, speed to the paint, and then they spread out their shooters around him," he said postgame.
"He creates a ton of shots, creates a ton of 3s and then defensively he deserves a lot of credit for the effort he gives there.
"You could go on and on about him, he's an All-Star, he's Ben Simmons. They miss a ton without him out there."
With Simmons' time on the sideline undetermined, it remains to be seen if Philadelphia will be able to keep its head above water. Without their playmaker, the Sixers are in uncharted territory -- the dynamic guard has missed just seven of a possible 221 regular-season games since making his NBA debut at the beginning of the 2017-18 season.
Teammate Matisse Thybulle said there was no denying the hole Simmons left while sidelined.
"There's only one Ben Simmons," Thybulle told ESPN.
"I've never been on the court with someone who is as physically dominant as he is, someone who just demands so much attention and causes so much havoc on the other team's defence. It takes all five guys to stop him in transition.
"I think we can [win without him] -- we've played without him [before] and been able to figure it out but I think it's just hard ... he plays such a pivotal role in our game plan on both ends of the court."