While the inspectors reported the Otago Corrections Facility (OCF) was well-maintained and clean, they also found some prisoners were spending up to 23 hours a day in their cells.
In response, Corrections said it had moved quickly to address the recommendations made by the Chief Ombudsman.
But the department stood by its earlier comments that there was no evidence of a significant increase in violence or threatening behaviour at the prison.
Inspectors, who were authorised by the Chief Ombudsman to visit places of detention under the Crimes of Torture Act, visited OCF to follow up on 16 recommendations made in a report three years ago.
Chief Ombudsman Peter Boshier said of the 2016 recommendations, five had been achieved, five partially achieved and six not achieved.
"There have been some significant improvements to OCF's record keeping and associated processes, particularly relating to the segregation of prisoners and the use of force."
All prisoners were receiving their minimum entitlement of an hour of fresh air each day and OCF continued to be well-maintained and clean, Mr Boshier said in the report.
However, there were areas of concern, including rising threats and violence and unprofessional staff conduct towards prisoners.
Inspectors also found the use of pepper spray at OCF was at a higher level than at other prisons and staff seemed to have a low threshold for its deployment.
"In contrast to the 2016 inspection, my Inspectors observed some prison staff displaying a lack of respect for, and negative attitudes towards, prisoners.
"The use of profane language with prisoners is, in my opinion, unprofessional."
Some voluntarily segregated prisoners also reported spending up to 23 hours a day in their cell.
Mr Boshier said senior prison management reported a significant number of prisoners were from out of the region and their contact with whanau was therefore compromised.
He repeated seven of the previous recommendations, including to investigate alternatives to wait restraints and handcuffs to manage self-harming behaviour, and that CCTV should not cover the toilet area.
He also made two new recommendations.
The first was that arrangements for dispensing and supervising the taking of medicine should be improved.
The second was the Intervention and Support Unit should start implementing interventions to help prisoners with coping strategies and their mental wellbeing, as an alternative to locking them up in their cell.
The Otago Daily Times recently revealed assaults at OCF had reached a record high in the last financial year.
Sources inside the prison and former inmates described rising violence and tension at the facility and some North Island inmates assaulting officers at random in order to be sent to a facility closer to home.
But Corrections yesterday stood by its comments that there was no evidence of a significant increase in overall violence and threatening behaviour at OCF.
In reference to reports of staff showing a lack of respect for prisoners and using vulgar language, chief custodial officer Neil Beales said Corrections did not tolerate abusive behaviour from staff and would be investigating.
Mr Beales acknowledged the report found the use of pepper spray at OCF was higher than at other prisons. Pepper spray was introduced at the prison earlier than other sites, and this contributed to its higher use at OCF, he said.
