Statistics show the average asking price in the districts is more than $1 million and the area is the least affordable in New Zealand.
Central Otago Mayor Tim Cadogan said he came to the district more than two decades ago and a person in the position he was at the time would not find it possible today.
"I came here as a solo father on what was then the DPB - on the benefit with a two-year-old and a three-year-old, about three years later I bought my first house with a $10,000 deposit that my parents lent me and that was on one income - forget about it. I'm not coming here now - people like I was then just can't afford to survive here," he said.
The situation was even worse than that though as families on two incomes were struggling to get a foot on the property ladder, and an affordability crisis meant companies struggled to attract workers and long-time residents - some multigenerational - were moving away, he said.
"I was talking to a young couple with a baby in Clyde just recently, they are struggling to find somewhere else to go because the landlord wants them out before Christmas. I spoke to a young couple with a three-year-old at the playground just last weekend - they were in the same boat. They just cannot find somewhere to live at an affordable price."
There were a number of factors at play, but the free market had failed young families and seasonal workers in the area.
"I do have voices that will say to me 'Just leave the market, the market will sort it out', I just think that's rubbish," he said.
"If you look at the town I live in - Clyde, a third of the houses in Clyde are unoccupied almost all of the time because they are holiday homes. And look people have worked hard and they've gained those.
"But when you've got businesses that can't get workers because they've got nowhere to live and you've got young people who have got nowhere to live and are leaving the area. And a third of your houses are empty, you've got to ask yourself if the system is bust or not."
Central Otago District Council was working with employers to ensure housing is being built for seasonal workers, as there was expected to be a strain this coming summer as a crackdown on freedom camping around the district began.
The council was also reviewing the district plan. But just because land was freed up for housing that did not mean development necessarily followed, Mr Cadogan said.
"We've only got so many levers we can pull and that is frustrating because at a local level we can see the problems, we can try to do what we can with the problems," he said.
"What a district plan change does is it enables people to do things. It can't force them to do things."
Land-banking and Not-In-My-Back-Yard-ism over higher density housing were obstructions to potential solutions.
"There is no perfect answer but I'm not willing to let seeking perfect get in the way of bloody and that's what we need here is a bloody good solution - we don't have it yet," Mr Cadogan said.
In the neighbouring Queenstown Lakes district, the problem was so bad that house prices were now more than 13 times the average wage.
Queenstown Lakes Community Housing Trust executive officer Julie Scott said she regularly heard of households leaving the district due to affordability.
"It is very sad to see families who have been here a long time, people who have grown up in Queenstown, have the inability to buy a home or just live affordably - it's not just about being able to buy a home when the median rent for a three-bedroom property is $750 a week.
"That's the highest in New Zealand. That's just outrageous."
The waiting list for assistance from the trust was now at an all-time high.
"We're still seeing probably 20 to 30 new registrations online each month from households seeking assistance from us," she said.
"So our waiting list is now at 569 households."
It had doubled from about three years ago, and that did not taken into account those who had already left the district due to unaffordability.
"Often people do let us know they want to be taken off the waiting list because they have left town and they usually cite housing cost as the key reason for that," she said.
A mayoral taskforce formed to tackle affordability in the district set a goal of building 1000 affordable homes in the next decade.
Whether that was enough remained to be seen and might need to be reviewed at some point, Ms Scott said.
Queenstown Lakes Mayor Jim Boult said he was pleased with the progress the council was making for permanent residents in the area, but there was much ground to cover for transient, seasonal workers.
"They're an enormously important part of the district - the town doesn't function without them," he said.
"And despite looking at a number of models to try to encourage worker housing development we've been unsuccessful to date.
"I'm very unhappy with stories about a dozen young people sharing a four-bedroom house. So we want to do something about it."
The numbers seldom stacked up for investors looking at developing workers accommodation, and employers needed to play their part in ensuring workers could affordably live in the area, Mr Boult said.
A visitor levy could also contribute back to council to assist in the development of worker accommodation in tourist hotspots under strain, he said.
