The 23m boat, on a day fishing trip, struck a rock just east of Green Island at 12.30pm.
Torea skipper Peter Leask (46), two crew and 23 passengers were later rescued from Ruapuke Island near Bluff.
Mr Leask, a Bluff boat builder, said he had done the best he could in the worst situation.
"I had to make a decision so I beached it pretty much and we called a mayday.
"It was pouring in ... There was water in the engine room, it started to blast around there, it was like the Titanic."
After hitting the rock, he said they had to travel two to three miles while taking on water before they could safely ground close to Ruapuke Island, 10 nautical miles from Bluff.
Mr Leask said his priority was his passengers, most from Southland and Otago.
They "seemed" calm throughout the ordeal.
"It was a bit of a worry with 20 or so people on board. They all had their life jackets on and were prepared. We were in control the best I could be.
"They were good about it and they were quite keen to come back. For them it was an experience I guess, they'll tell their mates."
Two Coastguard rescue boats from Bluff and Riverton and three winch-capable helicopters from Dunedin and Te Anau were initially launched for the rescue.
Five fishing vessels in the area had also been diverted to provide assistance. When notified all on board were safe, the rescue was scaled back.
The Coastguard, and fishing vessels Takitimu II and Provider were used to bring the passengers, crew and their gear back to Bluff.
Mr Leask was due to return to Torea today to check its condition and speak to his insurer.
"I want to get it salvaged if we can. I feel bloody pissed off and it's a real shame."
Mr Leask has skippered Torea for seven years. He told the Otago Daily Times he experienced a "worse situation" in 1999 when he had to abandon his boat, John A Settree, and was in the water alone for an hour before being rescued.
Mr Leask's wife, Estelle, said when she heard Torea was in trouble, she was "beside herself".
After numerous unanswered calls to her husband's cellphone, she feared the worst.
She would have extra reason to celebrate her birthday today, she said.
"I will give him a great big cuddle and then a slap over the head." Environment Southland communications manager Michele Poole said the boat was listing last night.
There were fears the 3000 litres of diesel on board could contaminate the area.
"There is no sign of diesel leaking, but you can see from aerial photographs there's a sheen on the port side of the boat. We believe it's coming from an air vent in the fuel tank," she said.
Regional on-scene commander Dallas Bradley said there would be talks with the owner and insurance company about salvage plans.
Last night, the boat was resting on the seabed with most of the deck underwater. It was stable and in a reasonably sheltered place given the forecast winds overnight, and its condition would be monitored, Mr Bradley said.
"Our priority is to prevent any diesel spilling from the vessel. This may mean transferring the fuel oil off the vessel first, or blocking the vents."
Environment Southland had also contacted the Department of Conservation and Iwi, including Ruapuke whanau, which was standard procedure in any offshore oil spill response, Mr Bradley said.
The Transport Accident Investigation Commission (TAIC) and Maritime New Zealand are making preliminary inquiries. Torea is owned and operated by Bluff charter company Campbelltown Seafoods.
The New Zealand Companies website lists the directors as William and Isabel Johnson, of Bluff, and Arthur Evans, of Invercargill.
