Mr Varsanyi finally said goodbye yesterday to his home and dream of the past 14 years.
With a little help from his friends.
"All these people just turned up to help me move," he said.
"I've got 10 years' of backpackers helping me out.
"I don't know what I'd have done without these guys."
Mr Varsanyi (51) bought the asylum in 1997 and established Asylum Lodge Backpackers four years later.
"It was abandoned and I thought it would make a nice place in the country. But it's been a lot of work. We've put tens of thousands of hours into the place.
"I'll really miss the place. It's so private and quiet and it's been a really good lifestyle. People like it here.
"We had horses, surfing, sea kayaking, fishing ... This job was more fun than most people have on their holidays. It was a great job."
Another drawcard was Mr Varsanyi's collection of more than 50 pre-1965 classic cars, which he has spent the past three weeks relocating.
"I love sculpture and, to me, cars are just another form of sculpture," he said.
The former asylum also served as a recording studio, when Dunedin band The Bats recorded two albums there at Christmas.
Backpacker Aline Walther returned to Dunedin with a group of friends from Germany this week to mark the end of the era.
"I first came here five years ago, when I learned to surf, and I always really wanted to come back.
"So I brought five friends over with me and we've been helping Frank out," she said.
The Seacliff Lunatic Asylum was built in 1884 and was later home to Dunedin writer Janet Frame, who received more than 200 applications of electro-convulsive treatment between 1947 and 1955.
On the night of December 8, 1942, 37 women psychiatric patients perished in a fire.
The land and six remaining buildings have been bought for $525,000 by an unidentified English couple.