The body of the 27-year-old was found near a park bench in the Dunedin Botanic Garden on Sunday, November 4.
Police have referred the death to the coroner, after finding gas canisters near his body.
His Auckland-based parents, Jenny and Bryan Lee, said their son was not an unloved child from a dysfunctional family but a loved young man from a good home.
"He died with so little, and on his own," Mrs Lee said.
His possessions were contained in a small paper bag, and "he had no backpack, no wallet, no socks, no underpants, no toothpaste, no soap, and a 50c piece in his pocket".
Josh had travelled around the world with his parents, fenced competitively, enjoyed rock climbing and had been on an Outward Bound course.
But that "extraordinary life" began to change when the then 17-year-old returned from a trip to the United States, where he had worked as a camp counsellor.
"When he returned to New Zealand he couldn't connect with his friends, who had left school and started careers, and an old school friend showed him a way to relax and chill out and it seemed to push all the right buttons with Josh," Mrs Lee said.
The couple were told huffing was psychologically addictive and difficult to stop, and even first-time users could die from the practice.
Inhalants were cheap and readily available.
The couple had tried everything to keep Josh from his addiction, including using professional anti-addiction services, community alcohol and drug counselling, doctor visits for psychiatric referrals and Salvation Army services and even approaching retailers directly.
"I would go around service stations in our local Pakuranga area with pictures of Josh pleading with them not to sell these products to this guy as he is an addict. They would just shrug their shoulders and say 'I just work here'."
The couple said there was little support available for Josh.
"It was a battle to get professional support when we needed it, rather than waiting weeks for appointments. Every can was potentially lethal for Josh. God help those people who do not have advocates," Mr Lee said.
Available support tended to focus on helping the family rather than treating the source of their anguish - Josh's addiction.
"Once he became an adult we weren't able to become his advocates as much."
The couple believe the longest time Josh spent off inhalants was the 12 weeks he spent at the Auckland-based Higher Ground Drug Rehabilitation Trust, but once he left "there was no going back", Mr Lee said.
"If I could have bricked him up in a house in the back of a garden and pushed food through a slot for him, I would have, but you can't do that."
For several weeks his grandmother drove him twice daily to attend Narcotics Anonymous sessions, but to no avail.
"He was not strong enough to push it away. He was ashamed by what he was doing but he could not help it. He was so angry with himself and became angry at us ... I miss him so much, so much."
Mrs Lee said the huffing addiction had taken a tremendous toll on the family, and while she had feared that final contact from the police informing them he had been found dead - "you hope like hell that won't happen".
Their son wanted to end his addiction and was terrified of the hallucinations he experienced when he was high.
"Huffing is horrible. It is not like having a few drinks too many and curling up in the corner and going to sleep. It is truly horrendous," Mr Lee said.
On Sunday, November 4, three policeman knocked on the door of their home in Auckland to inform them Josh had been found dead in Dunedin.
Josh came to Dunedin several years ago after meeting a girl attending the University of Otago.
While in the city he took a barista course and developed a passion for making coffee.
Last month, he called into a Princes St cafe, where he had worked until early 2011.
"He was pretty much in tears and in a mess. He wanted some sort of job or something and I told him to come back and see me the next day," Mojo Coffee owner Jason Moore said.
"I wanted to give him some positive encouragement and write him a reference."
However, Josh never showed up.
Mr Moore said Josh was well-liked by staff and customers and was passionate about coffee, but had to be let go after "dropping off the grid" for days at a time.
"He made a lot of poor decisions and we gave him a lot of chances, far more than we ever gave anybody else ... It is just a real shame."
Months before his death Josh was found by social services staff sleeping rough in the grounds of First Church.
He became a client of Family Works as his family was in Auckland and he had little support in Dunedin, Presbyterian Support communication and fundraising director Lisa Wells said.
News of his death had affected staff.
Many of the social services were not specifically designed for people of Josh's age and they tended "to fall through the cracks much easier than younger or older people".
During this period Joshua was also referred to the Emergency Psychiatric Service, and spent time at the Dunedin Night Shelter.
Another person contacted by the Otago Daily Times said she had taken Josh to have a psychiatric assessment, after concern for his welfare.
"We realised that things were pretty dire for the boy."
She described Josh as a clean-cut young man who was well-spoken and spoke fondly of his job working in a cafe and his passion for coffee.
Dunedin-based friends of Josh plan to hold a small service for him at the Dunedin Botanic Garden on Sunday.
Meanwhile, his parents had travelled to Dunedin to collect his body, and were now preparing for his Auckland-based service on Thursday.
"It's to remember all things we have lost," his mother said.
