The beach was cleared after the 3-year-old female sea lion began chasing, barking and baring its teeth at swimmers between the flags about 2.40pm.
St Kilda Lifesaving Club head lifeguard Sam Todd said he did not believe it was going to bite anyone, but as a safety precaution the decision was made to clear the beach.
''I don't think it's a danger to the public.''
The sea lion had visited the beach daily for the past six days, but this was the first time he had seen it act aggressively. The sea lion left soon after the water was cleared. The beach was reopened.
Jesse Farry (26), a Dunedin teacher who was among the people told to leave the water, said the sea lion had been swimming near beach-goers when it started to become ''a bit aggressive''.
The behaviour continued after they were out of the water.
''It came up on to the beach and it was sort of charging at people, being a little bit aggressive, and opening its mouth and barking.
''Everyone just kept moving back and then it just sat down in the middle of the flags, plonked itself on the sand and sat there.''
The sea lion swam away after about five to 10 minutes.
''No-one was scared. Everyone thought it was just interesting.''
New Zealand Sea Lion Trust chairman Steve Broni said the sea lion probably wanted to play.
''His goal is not to bite anyone's leg off or anything, but sea lions do tend to play with their mouths.''
The worst possible outcome would likely be a playful bite, but sea lions' ''razor sharp'' teeth could cause some damage.
''To the best of my knowledge, no-one has been seriously injured by a sea lion on the Otago coast.''
Mr Broni said a lot of people made the mistake of trying to shoo sea lions away like they would with another animal.
''The more gesticulation you make around them, the more they take it as an invitation to play.''
According to the Department of Conservation, the sea lion is a 3-year-old male called Seazar.
It was pictured in Saturday's Otago Daily Times playing with a ball on St Kilda Beach.
On Saturday, a sea lion approached and barked at a swimmer in the surf at nearby Smaills Beach.
