Immigration NZ's pilot data collection system, used to prioritise the deportation of overstayers, was described as "illegal" by the Green Party earlier this month and has been put on hold.
The system made decisions based on factors such as age, gender, criminal history and any history of illegal employment, and was originally thought to include country of origin.
However, Immigration NZ has since said it had never considered nationality in deportation decisions.
University of Otago Faculty of Law associate professor and Centre for Law and Policy in Emerging Technologies director Colin Gavaghan said there were "fairly regular shock stories" about data analysis and more transparency was needed.
"Even the Immigration Minister himself was in the dark about the system."
Along with academics from the departments of computer science and philosophy, Associate Prof Gavaghan last week published an article on the Pundit website calling for an "oversight body".
A watchdog with members who had good knowledge of emerging technology and the law could compile a list of predictive analytics used by publicly-funded bodies and provide information to the Government on the systems, Prof Gavaghan said.
"It's not about slamming the Government."
A"black box" scenario, with data going into the box and a result coming out without an explanation, was not ideal, especially if individuals wanted to challenge decisions.
"You can't put the black box in the witness stand and ask it questions," he said.
It was possible accidental discrimination could take place in data collection programmes due to a system glitch, but he did not know whether that had ever happened.
The proposal had had good feedback from members of Government so far, Prof Gavaghan said.
"The UK is doing something similar, putting a watch on the development of this kind of technology."
Computer science Associate Prof Alistair Knott said when it came to data collection systems the public did know about, people only had partial information about how they worked.
"We're told, for instance, that the Immigration NZ tool is `just an Excel spreadsheet'," he said.
"But many algorithms can be run in Excel: what algorithm is being run in this case? On what data? With what results, and what margin of error?"
Prof Gavaghan said if a committee was created it would be good to have someone from either Statistics NZ or the DIA on it, as well as potentially someone from the Human Rights Commission.
Philosophy and law academic Dr John Zerilli said regulating "giants like Facebook" would involved "lengthy international negotiations".
"In the meantime, there is no reason why New Zealand should not put its own house in order over the use of these same tools in its own government."