The trust applied for resource consent to demolish four buildings in Gore St that make up the Club Hotel, a category 2 heritage building on Heritage New Zealand Pouhere Taonga's list.
The site is also recorded as being of heritage value in the Invercargill district plan.
Commissioner Peter Constantine said after the hearing last month there was a significant lack of detail in the application to rule out the potential for alternatives.
"I am also persuaded that the proposal will result in significant adverse effects on the environment and that it is not consistent with the outcomes sought by the relevant planning instruments."
Festival trust member Kylie Fowler previously told the Otago Daily Times the trust bought the building in 2014 and had tried "everything possible'' to sell it.
"We are volunteers and we don't have this money.''
