Last week, a woman came forward alleging Middleditch had sexually abused her when she was 11 in 1972.
A second victim then came forward and told the Otago Daily Times she was also abused by him.
The women — who did not want to be named — had a meeting with HNZ yesterday about displaying the artworks in the current and new Dunedin hospital.
One of the women said the meeting was "very positive".
"They’ve taken down two out of the three artworks from the hospital foyer and removed the plaque from the large copper water fountain.
"They have taken all of John Middleditch’s artworks off the catalogue of art that’s proposed for the new hospital."
It was a huge relief none of his work would be displayed in the new hospital, she said.
Police confirmed Middleditch was convicted in an unrelated case in December 1976 of indecently assaulting a girl aged between 12 and 16 and of other indecent assaults.
Reports in the Otago Daily Times from the same month show Middleditch was convicted of seven charges of indecently assaulting girls aged between 8 and 11, and one of attempting to indecently assault a girl.
The woman was told by a hospital staff member Middleditch was also found guilty of physical assault on a woman in the 1940s.
In 2012, the woman requested the then-Southern District Health Board remove the large water feature, but when she told the art adviser Middleditch had been convicted of assaults on children in the 1970s, the woman was told she had no proof.
Yesterday, HNZ apologised to her for their response.
"They acknowledged the issue going back to 2012 where my initial request was ignored by the then-DHB, and I’ve asked for some formal acknowledgement of that."
HNZ and the women also had a "lively" discussion on what to do with the artwork going forward.
While it would not all be plain sailing, she felt like they were moving in the right direction.
