But the man behind the proposal, Dunedin businessman Peter Graham, said his venture would be respectful to both and bring something ``unique'' to the city.
The debate came during a joint Otago Regional Council-Dunedin City Council consent hearing in Dunedin yesterday.
Mr Graham wanted to use a small reserve south of the Vauxhall Yacht Club as the base for his golf challenge, kayak and bicycle hire and coffee and food van business.
Golfers would aim for an 8m by 12m floating pontoon positioned 95m out into the harbour, and stood to win a $10,000 prize if they scored a hole-in-one.
The proposal attracted 25 submissions, including 15 opposed and seven in support.
Among those arguing against the venture yesterday was Craig Latta, a regular recreational paddleboarder and owner of Watercooled Sports, which runs paddleboarding training sessions in the Vauxhall area.
Mr Latta said he regularly paddleboarded in the area and saw first-hand the rubbish already littering the harbour.
Adding 28,000 golf balls a month to the water around the pontoon would only make the problem worse, and he doubted the balls would be able to be retrieved.
``The balls will sink. They will go everywhere. It's just not going to happen.''
Mr Latta also worried about public safety, despite a plan to use a spotter to stop golfers if another water user entered the ``strike zone'' around the pontoon.
The business would attract an unruly element after hours, bringing their own equipment and balls, he predicted.
``Who is responsible if someone gets hit by a golf ball and dies? Because that's a possibility,'' he said.
He supported other aspects of Mr Graham's proposal, but warned other water users would avoid the area if the golf venture proceeded.
John Parnell, a coach at the Otago Rowing Club, also opposed the venture, saying the pontoon would be anchored in one of the few stretches of water suitable for rowers' 1000m interval training.
That would have a ``massive'' impact on the rowing community, forcing them into rougher water further out or to change course three times to go around the structure, he said.
The club did not claim sole rights to the stretch of water but the golf venture would effectively be handed that if Mr Graham's business went ahead, he said.
Other submitters also argued against the golf venture, including Hayley van Leeuwen, a Vauxhall resident whose home overlooked the site.
She said the repetitive sound of groups of golfers striking balls would have a ``significant'' negative impact on the area's residents, including those with young children.
A similar attraction in Taupo stayed open until 11pm and attracted large groups, including stag parties, hen nights and work functions, she said.
Lyndon Weggery said walkers like him, out to enjoy the ``peaceful and serene'' area, could be at risk from stray golf balls.
Otago Peninsula resident Brenda Leigh congratulated Mr Graham on following up his idea, but opposed it, saying it highlighted a growing conflict between the city's environment and potential economic gains.
``We are talking about a commercial operation . . .and the beauty of our environment, and that's the toss-up.''
However, Mr Graham insisted that as a Vauxhall resident, regular sailor and traffic management company director, he wanted a venture that was safe, respectful and encouraged more people to use the harbour.
As well as using a spotter and other measures to control the way golfers behaved, he had agreed not to run the golf venture before 9am to avoid any conflict with rowers, he said.
However, removing the pontoon each night and installing it again in the mornings, as rowing clubs wanted, would ``render the operation unviable'', he said.
The success of the business would also depend on the recovery of golf balls from the sea floor, creating an incentive to do so, he said.
``Some people are talking about them being rubbish, but they are actually the asset for the business and they need to be collected.
``The whole operation depends on getting the balls back.''
Lala Frazer, of Save the Otago Peninsula, also submitted against the proposal.
The hearing was adjourned late yesterday for non-public deliberations. A written decision would follow.