The 1-0 win over Brendan Rodgers' side now means City can secure the Premier League title with a victory at Brighton on Sunday.
City are on top of the table with a one point advantage over Liverpool, who host Wolverhampton Wanderers on Sunday and know that their only chance of winning the title is if City stumble on the south coast.
It had been a nervy and frustrating night for the home crowd until Kompany's 25-metre drive sent the Etihad Stadium wild in a mixture of relief and joy after their longest wait this Premier League season for an opening goal at home.
The 33-year-old Belgian defender picked up the ball just over the half-way line with the Leicester defence camped deep, and he unleashed a stunning drive that flew into the top corner of Kasper Schmeichel's goal.
It was the first time that Kompany has scored a goal for City from outside the box and there could not have been a better moment for such unlikely inspiration.
Leicester had been good value in the first half as they tested City's defensive solidity and frustrated their forward movement.
There was a definite sign of nerves impacting on Pep Guardiola's side as they struggled to find their usual fluidity and invention.
Tension spilled
The crowd's tension spilled onto the field and the occasional murmurs suggested that self-doubt was creeping in.
But after the break, City stepped up the tempo and the introduction of Leroy Sane for Phil Foden in the 56th minute allowed them to stretch the Foxes defence and penetrate the penalty area.
But still chances were at a premium and when Sergio Aguero managed a close range poke goalwards in the 69th minute, Schmeichel was up to the task.
Seven minute later Kompany took centre stage but there was still time for a scare as Leicester had an opportunity to level in the 87th minute but substitute Kelechi Iheanacho, a former City player, was unable to find the target from a promising position.
City, who have now won 13 straight league games, have 95 points to Liverpool's 94 and also have a plus four goal difference over Juergen Klopp's side.
Unless Brighton can produce the upset that has so far failed to materialise in the run-in of this epic duel, Liverpool, beaten just once this league campaign, will have to settle for the runners-up spot.
Kompany, who has won three Premier League titles with the club since 2012, was overcome by the moment and was in tears as he took part in the lap of appreciation after the game.
"I'm really happy for the team. This club, it's part of how we have been successful to have goals like this, moments like this. We always believe that we have a chance and can play for the last second," said Kompany.
"We were putting pressure on, upping the tempo but found the last pass difficult. They defended really well and it was going to take something special. Now we have a final to play in Brighton, Nothing is won yet."
