Southland Area Commander Mike Bowman said emergency services leapt into action after they were notified of the capsize on the Riverton bar at about 2.55pm Saturday.
"The boat, a recreational craft, capsized approximately 100 metres off shore with five people on board."
Two people were recovered alive from the water and were brought to shore before being transported to Southland Hospital, where one remains.
A third person was unable to be revived by emergency services after being recovered unresponsive and brought to shore.
The fourth person was located dead in the water.
Southland District mayor Rob Scott said it was "devastating" to learn of the incident and was a "real tragedy" for the small community of Riverton.
"There’s three people that went out on a trip and didn’t come home, it’s going to have a big impact on a lot of people.
"It’s going to be a grieving community."
Mr Scott said the smaller communities inside Southland were "very tight knit" and would be wrapping around each other in support, and he planned to head up to Riverton this afternoon.
Coastguard Riverton skipper Ross McKenzie, who responded to the boat tragedy, said the three who died were locals and were known to the Riverton community.
"Obviously we're a pretty small community and everyone knows everyone.
"It's just heart-wrenching for the families and friends of families of the people involved. They've lived in the area for some time... about 1400 people are resident, so we all know it each other."
Conditions were almost perfect on Saturday afternoon, he said.
"I had a look at the bar probably half an hour before ... the incident and was thinking how flat and calm it was.
"But you can never be too sure on crossing a bar - safety first."
Even the most experienced sailors need to be prepared and wear a life jacket, McKenzie said.
The Riverton and Bluff Coastguard, multiple rescue helicopters, Fire and Emergency New Zealand, Hato Hone St John, Police Search and Rescue and other volunteers were part of the search and rescue effort, Insp Bowman said.
"This is an absolute tragedy and our hearts go out to all those impacted by this devastating incident."
- Additional reporting RNZ