The big English defender moved up to attack a left-wing corner as referee Peter Faithful was about to blow the final whistle, and he hit the jackpot with a powerful header that flew past goalkeeper Matt Scoles.
Most of the derby match was balanced on a knife edge, as both teams showed pace in attack and generated enough goalmouth action to keep the large crowd perched on the edge of their seats.
Despite being several regulars short, Technical coach Mike Fridge adopted a brave attacking formation that had Caversham penned in its own half for the first quarter.
Caversham keeper James Waggett had to make several impressive saves from Blair Scoullar, Aaron Burgess and Justin Flaws, who, along with brother Mike, used pace and width to deliver dangerous crosses into the Caversham penalty area.
But Steve Fleming's side settled and hit back at Technical with equally swift breaks, and piercing crosses that extended central defenders Scoullar, Matt Joy and Dan Ford, and required Scoles to make several critical high takes and brave saves.
Midfield was a battleground, but as both sides were keen to push forward it was not too congested and there was plenty of creative play.
Technical's Andy Coburn had his usual strong forcing game, as did George Milne, who harried Caversham into untypical mistakes and loss of possession.
Striker Burgess came deep to unload some clever first-time lay-offs, then doubled around at pace to get on the end of some well-flighted crosses.
Caversham's Patrick Fleming became more prominent as Darren Overton and Blair Duncan got their passing game going, and Robbie Deeley and Anthony Hancock made late runs that produced shots.
In the 37th minute Caversham scored when a left-wing attack produced a bizarre deflected cross that fell vertically into the goalmouth, and a collision between Scoles and defender Hayden Gunn allowed Hancock to force the ball over the line.
From the restart, Technical showed its mettle, and a couple of penetrating passes released Justin Flaws, who skipped past a despairing challenge by Waggett and rammed in an equaliser for 1-1 at the break.
The second half produced more cut and thrust from both sides with enough chances to score several more goals, but it looked as if the match would be drawn until Schwarz powered in the winner.
"It's a cruel game.
We did enough to at least get a share of the points, and I am proud of Technical's effort," Fridge said.
Fleming commended the opposition for not allowing his side to settle initially.
"It was a good contest, and needed some good individual performances for us to get some controlled moves going," he said.
"It was not a vintage Caversham performance, but the fighting spirit was there, and we have to be happy with three points that keeps us in the hunt for the FPL championship."
Grants Braes confirmed its vast improvement this season by earning a hard-fought 4-3 away win over the Spirit in Invercargill.
Goals to Shannon Cox (two), Newton Rewi and Pete Alcock made coach Al Laidler a happy man.
At Sunnyvale, Roslyn-Wakari beat Green Island 4-2, but the home side signalled improvement to fight back and narrow the score to 3-2 before Roslyn scored the final goal.
In another tight match, Mosgiel stayed ahead of Northern to win 3-2.
Striker Shayne Danielson scored twice for the Plainsmen and Morgan Day also got on the scoresheet.
Northern's goals came from Tom McLean and Tim Smith.
In Queenstown, Sam Jasper scored twice for Rovers, and Johnny Moore clinched the 3-1 result over an under-strength University side.
