Junior Togatuki, who had mental health illnesses, died in solitary confinement in Goulburn's Supermax prison.
The 23-year-old's sentence for robbery and assault had ended in August this year, but he was being held ahead of his deportation to New Zealand - the country he had left aged four.
Togatuki had written to Australian Immigration Minister Peter Dutton, begging to be allowed to stay in Australia.
New Zealand Prime Minister John Key plans to bring up his concerns about the nearly 200 Kiwis being held in detention centres - some on isolated Christmas Island - with Australia's Prime Minister Malcolm Turnbull.
However, pressure is building for a faster response. This afternoon, Andrew Little said an urgent explanation was needed in the case of Togatuki.
"Australians are not treated like this in New Zealand. The Government must do something about this now," Mr Little said.
"It is time our Government stopped sitting back and watching this appalling treatment of New Zealand citizens and sought an urgent clarification of Australia's policy."
Togatuki died after writing farewell messages to his family in his cell, the Sydney Morning Herald reported. Authorities in Australia say he killed himself.
He suffered from schizophrenia and anxiety.
"The death of Junior Togatuki was a preventable tragedy. The Government must seek an urgent explanation about why the 23-year-old - who had a recognised mental illness - was left unsupervised in solitary confinement," Mr Little said.
From last November, Australia has toughened up its rules, and making non-citizens who served more than a year in prison liable for deportation.
That meant there were some people facing deportation who had lived in Australia for many years and who committed relatively minor offences.
Concerns have been mounting about people, including New Zealanders, held in detention centres, with a member of the Australian Lawyers Alliance saying their treatment could amount to torture.
Today, New Zealand's Humans Rights Commission said Kiwis with concerns about loved ones in Australian immigration centres should contact the Australian Human Rights Commission.
"We support our Australian colleagues' longstanding calls to shut down the Australian Government's detention centres," human rights commissioner David Rutherford said.
"We support this position and encourage our Government to continue to advocate on behalf of New Zealanders held in Australian immigration detention centres."
As well as opposition parties such as Labour and the Green Party, the Government's support partner United Future has condemned the new Australian policy as damaging to the Trans-Tasman relationship.
"These people are being denied basic rights like access to medical care and lawyers. Their families cannot contact them and they have no idea how long they are going to be there for," Mr Little said.
"It has the potential to damage the special relationship between Australia and New Zealand - a relationship described by Australian leaders as 'family', and fostered by fighting side by side, even today in Iraq."
Mr Key, who is currently in New York for the United Nations General Assembly, on Friday said he was concerned about the deportation policy.
He believed New Zealand still had a special relationship with Australia, and thought it was likely Kiwis were "collateral damage" in a broader policy shift.
- By Nicholas Jones of the New Zealand Herald
