The call is in response to increasing use of sophisticated new data tools and predictive analytics by state agencies - sometimes without the knowledge of those whose information is being used.
Immigration NZ has been piloting a profiling system to find over-stayers and public insurer ACC has been using a model to predict how long clients will be on its books.
The approach also came under the spotlight when former social development minister Anne Tolley angrily blocked a Ministry of Social Development (MSD) trial that proposed to test its model by risk-rating tens of thousands of newborns - and waiting for two years to see if its predictions proved accurate.
A group of University of Otago academics who have been investigating the issue say New Zealand should follow the United Kingdom in moving toward a new body to oversee the use of such AI-based predictive tools.
The experts, working under the Law Foundation-funded Artificial Intelligence and Law in New Zealand Project, agreed there was a place for them but argued they needed to be watched, and that the public not be kept in the dark.
"At present, public information about predictive analytics in government comes mainly from media exposes," said Associate Professor Colin Gavaghan, of Otago's Faculty of Law.
"In the latest case with Immigration New Zealand, even the Immigration Minister himself was in the dark about the system."
Gavaghan said the public needed a general picture of how the systems operated.
"Are they systematically denying benefits to one segment of the population, or denying them parole, or whatever?
"That's one of the big concerns that people have about these systems. It may or may not be a justified concern, but we won't know unless somebody's checking the outcomes."
There were also concerns at an individual level, he said.
"You can't appeal a decision against you without some sort of detail about how that decision was arrived at. So again, we ordinary punters need to be able to get some sort of comprehensible explanation of how those decisions were made."
But he acknowledged that might not be a straightforward task.
"In some cases, the outcomes of predictive analytics are pretty incomprehensible, even to those who know something about them.
"In other cases, the manufacturers won't give up the detail of how they work, citing commercial sensitivity.
"One of the things we're proposing is that, when government departments buy these things, they give some thought to their explainability."
At the same time, he added, data security was critically important.
"Striking the right balance between understanding how these systems work, while at the same time keeping private data private, could be a delicate task, and again, it seems like some expert advice could be very useful in that regard."
Another member of the project, philosophy and law researcher Dr John Zerilli, said the issues were particularly pressing now, in light of the recent scandal surrounding Facebook's misuse of personal data.
"There are growing calls - which we fully support - to regulate the use of personal data gathered by social media sites," Zerilli said.
"But the process of regulating giants like Facebook will be a complex matter involving lengthy international negotiations.
"In the meantime, there is no reason why New Zealand should not put its own house in order over the use of these same tools in its own Government."
The project members proposed a new agency that would publish a complete list of the predictive tools used by Government departments.
With each of the systems on the list would be basic information about their design, variables and techniques used.
Broadcasting, Communications and Digital Media Minister Clare Curran hasn't responded to the panel's call.
But a Department of Internal Affairs spokesperson told the Herald discussions around the issue were underway across government departments.
Following February's D7 meeting of the world's leading digital nations, New Zealand was also leading a new working group on digital rights.
National deputy leader Paula Bennett, who was social development minister when the tools were being developed by MSD, argued there was too much oversight already.
"The protection of individual data is vitally important but what is also important is that government agencies have the information they require to help those in need, and we have always taken the view that the use of data by the public service is already over-regulated," Bennett said.
"The National Government invested heavily in data and data protection, and our reforms around the Privacy Act were about protecting individual data as well as making it easier for agencies to share information."
"The social investment model was designed to break down silos to ensure vulnerable New Zealanders got access to services targeted to their specific need."
- By Jamie Morton
