The Wellingtonian went on to conquer the world, of course, starring in the All Whites' 1982 World Cup campaign and enjoying a successful career which culminated with the 48-year-old named Oceania's footballer of the century.
But it all started in his backyard with practice, practice and more practice.
And that was exactly the message the champion striker was trying to drill home at a football clinic at the Ocean Grove Domain yesterday.
Arguably New Zealand's greatest player, Rufer told a captive audience, of about a dozen 9 to 11-year-olds, talent was no substitute for hard work.
"Kids who work the hardest and put in the most practice end up being the best players," he said.
"It is not rocket science; it is about repetition and hard work."
Rufer was invited by Grants Braes football club to come to Dunedin and lead a three-day clinic but he was keen to do something more substantial.
For the last 13 years, Rufer has run WYNRS Soccer School of Excellence and has been looking at expanding into Dunedin and Christchurch.
WYNRS is well established in Auckland and Wellington.
All Whites Chris Wood and Cole Peverley are just two of the players who have emerged from the programme.
Rufer said with the All Whites qualifying and then competing so admirably at the World Cup, and with the Phoenix doing "quite well" in the A-League, there had been an upsurge in interest in the sport.
The key was to capitalise on that enthusiasm and bring through the next generation.
"Football is on a high. It is the best it has ever been, and that includes the early '80s," he added.
"'They [New Zealand Football] didn't take advantage of the opportunity back then but it is different today.
"'There is already a lot more happening than there was in '82. All they had then was the World Cup opportunity.
"Today there is the Phoenix, the under-17 and under-20 World Cups and the Olympic Games."
Rufer said the Phoenix offered players of a certain calibre a pathway which did not exist during his playing days and was a positive step for the game.
But the New Zealand Football league - the ASB Premiership - was not as strong as it could be.
"It is limited but it is getting better. It is a long way off from where the A-League is at. And the A-League is a long way off from leagues in Europe.
"Europe is where you want to be playing because it is the highest level."
Rufer believes the country has a lot of potential in the women's game.
He has been doing some work with the Football Ferns ahead of the World Cup in Germany this year.
"I have a good relationship with John Herdman, who is the head coach.
"He asked me if I could get involved, so I've been working with them for the past few months and there is some great talent there."
In terms of the men's game, Rufer said expectations shot up following the All Whites' 1-1 draws with Slovakia and Italy in their opening pool matches.
Hopes were high the All Whites could upset Paraguay and progress to the second round of the World Cup.
The game was drawn 0-0.
"We hadn't qualified for the World Cup since '82 and as soon as we did the expectations rose.
"Once you are on the big stage everyone wants to win, which is natural.
"But, to be fair, the team did outstanding, even though on the world stage no-one celebrates being undefeated.
"But it was a good thing for New Zealand and it has worked really well as a marketing tool, which is great."
But as far as playing more attacking football and posing a greater threat, Rufer said the reality was the best player in the team was a defender and the All Whites had played to their strengths.
"Without Ryan Nelsen I don't think they would have got to the World Cup," he said.
