Striker Kgatlana struck two minutes into stoppage time to set up a clash with the Netherlands in Sydney on Sunday and broke the hearts of the Italians, who had only needed a draw to progress.
Italy had appeared to salvage the draw they needed when Arianna Caruso scored from a corner in the 74th minute but were left to rue what could have been after a series of late missed chances in Wellington.
"They fought like warriors," said an emotional coach Desiree Ellis, who was a founding member of the South Africa women's side in 1993.
"They fought like the heroines that we know that they are. They fought to be historically remembered and they've made history not just getting our first win, but going to the round of 16 and that is freaking amazing."
The Italians had opened the scoring when Karabo Dhlamini tripped Italy forward Chiara Beccari just inside the box in the 10th minute and Caruso put the penalty into the bottom left corner of the net.
South Africa equalised in the 32nd minute through an own goal from Benedetta Orsi, who did not check where goalkeeper Francesca Durante was before making a back pass.
Banyana Banyana went ahead for the first time in the 67th minute when Kgatlana's neat inside pass found Hildah Magaia free behind the defensive line and the forward swept it into the net.
Caruso grabbed her second of the match seven minutes later from a corner when the ball caught her hip in a goalmouth scramble and cannoned into the bottom right corner of the net.
It looked as though Italy might hang on for the draw in a frantic finale but Magaia's pass found Kgatlana in the box and the striker smashed the ball into the net to secure second place in the group behind Sweden.
Kgatlana overcame personal tragedy and a brutal injury to deliver South Africa's first Women's World Cup win.
"Over the last three weeks, I've lost three family members. I could have went home, but I chose to stay with my girls," Kgatlana said. "Because that's how much it means."
The forward suffered a torn Achilles tendon in 2022, an injury that kept her sidelined for some 10 months and nearly cost her a trip to her second World Cup.
"I came back from a very, brutal injury to be here, to play for the country, to represent every single girl that wanted to be here, to make history with the girls for South Africa."
Sweden beat them 2-1 with a 90th-minute winner in their opener and the team were forced to settle for a 2-2 draw in their second group match against Argentina, after blowing a two-goal lead.
But Kgatlana said she and her teammates walked into Wednesday's match ready to "dig deep" despite the frustrating previous results.
"Today we just told ourselves, we have to fight. We were the underdogs and I think it worked perfectly for us. When you play that underdog, they come out running for you and we knew that in terms of pace, they weren't like us, and it's something that we could utilise for the victory."
They will face the Netherlands in the last 16 in Sydney on Sunday, after finishing second in Group G behind Sweden.
Other matches
Jamaica 0, Brazil 0
Jamaica earned a draw in Melbourne on Wednesday to advance to the Round of 16 at the Women's World Cup and send Brazil home from the group stage for the first time since 1995.
Three consecutive shutouts propelled Jamaica into the knockout round for the first time.
Jamaica goalkeeper Rebecca Spencer continued her stellar play by keeping Brazil off the board, ending the spectacular World Cup career of Brazilian star Marta, who has a record 17 career World Cup goals.
Playing in the Women's World Cup for only the second time, Jamaica needed a GoFundMe campaign to pay for the team to make it to the event.
The Reggae Girlz eliminated reigning Copa America champs Brazil and advanced as the runner-up to France (seven points) in Group F with five points.
Sweden 2, Argentina 0
Two second-half goals set up Sweden for a Sunday date with the United States in a matchup of group stage winners.
Elin Rubensson sparked the celebration in the 90th minute with a penalty kick in Hamilton on Wednesday. Rebecka Blomqvist opened the scoring, with a header for a tally in the 66th minute for No 3 Sweden.
Sweden beat the Americans in 2021 in Tokyo during the quarterfinals of the 2020 Summer Games.
France 6, Panama 3
France dominated possession and scored six goals to win Group F in a fast-paced and entertaining match against Panama in Sydney on Wednesday.
France fell behind instantly on a free kick by Marta Cox and appeared to be reeling.
But Lea Le Garrec made it 4-1 before the half and France's Kadidiatou Diani scored three times. Panama wouldn't go silently, with goals in the 64th minute on a penalty and Lineth Cedeno's header in the 87th minute made it a one-goal game.
France began the match with captain Wendie Renard (calf injury) and all-time leading scorer Eugenie Le Sommer on the bench but proved to have more than enough firepower, netting six of its 21 total shots and possessing the ball nearly 70 percent of the match.
- Reuters and Field Level Media
