The outgoing head coach was on a plane to Japan on Saturday night, shortly after the team’s 35-6 loss to the Blues at Eden Park.
It brought to an end Brown’s second Highlanders stint as a coach — and fourth with the team, including his playing days.
A season that featured four wins and 11 losses will not have delighted the highly rated coach.
However, as his attention turned to the Japan national team and his partnership with Jamie Joseph, he admitted he would miss his home team.
"I’ll always miss coaching the Highlanders, playing for the Highlanders," Brown told Sky Sport.
"I’ll be floating around somewhere.
"I’m actually flying out for Japan tonight [Saturday], now. I was hoping to get another week in New Zealand, but I’m on the plane tonight and joining up with Jamie [Joseph]."
Brown said player availability had a significant impact on the Highlanders this year.
Injuries had hurt the team, while a lack of depth meant Covid-19 had a bigger impact on it than most.
That continued right up until Saturday night — Aaron Smith being ruled out late with a groin injury, while Thomas Umaga-Jensen picked up an injury midway through the first half.
"For us at the Highlanders, we’ve just got to get all of our best players ready to go.
"We don’t have the depth of the other teams. But when we get our best 15 on the park, we can compete with everyone.
"That’s the challenge for us going forward. Footy’s footy, it’s not the end of the world.
"We’ll put a team together next year and have another crack."
He admitted the Blues were a quality team, although he said Andrew Makalio’s red card 22 minutes into Saturday’s quarterfinal was a defining moment.
The Highlanders had scrapped well against a superior opponent up until that point.
However, the hooker’s shoulder to the head of Blues lock Tom Robinson had left them reeling and eventually unable to contain the Blues’ firepower in the midfield and out wide.
"As soon as you get a red card, you’ve got to go into defensive mode," Brown said.
"We held on and held on and then one missed touch from a penalty and they scored. It sort of breaks the back a little bit.
"We keep 15 on the park and we keep applying pressure, we’re in with a better shot."
From there the floodgates opened somewhat, as the Blues made use of front-foot ball and the Highlanders’ defence wilted.
Beauden Barrett showed all his class in finding gaps, crossing for two tries.
The midfield was equally threatening and the combination of Roger Tuivasa-Sheck and Rieko Ioane cut the Highlanders’ defence to pieces, Scott Gregory notably missing nine tackles.
"We just gave up three very soft tries off a missed touch, a missed tackle at lineout strike and dropped a high ball. So 21 points.
"I thought the boys put in a massive effort. If we kept 15 on the park, who know what could have happened in that first half? I thought we started the game really well."
The Blues advance to host the Brumbies next week, after the Australian team beat the Hurricanes 35-25 on Saturday night.
The Chiefs will face the Crusaders in the other semifinal.
That came after the Chiefs beat the Waratahs 39-15, having led 27-10 at halftime, their first playoff win in five years.
The Crusaders had earlier beaten the Reds 37-15 on Friday night.
