His views have been likened by one academic as the boldest political move in criminal justice since the abolition of the death penalty in 1961.
Mr Little said ``so-called law-and-order'' policies have been a 30-year failure and locking up more people with longer sentences has not made New Zealand safer.
``New Zealand needs to completely change the way criminal justice works,'' he said.
``One of the major challenges is to turn around public attitudes - to say that what we have been doing for the last 30 years in criminal justice reform actually isn't working. Our violent criminal offending is going up.''
The comments follow an open letter from 32 academics in criminal justice calling on the Government to reject the building of a mega prison in Waikato designed to hold up to 3000 inmates.
Those academics yesterday welcomed Mr Little's views.
The proposed upgrade of Waikeria Prison is due to be decided next month by the Cabinet and poses a huge challenge for Labour since it came into government promising to reduce prison numbers by 30% in 15 years.
The promised reduction comes at a time when the Department of Corrections has 10,695 prisoners and room for only another 300.
``We just had this rapid increase in the last few years that cannot be explained by anything other than penal policy that, frankly, has got out of control,'' Mr Little said.
He said he wanted a ``national conversation'' which sought out the best ideas but also led to a better informed nation that understood ``tough-on-crime'' policies were leaving a legacy of failure.
He planned to hold a criminal justice summit that would seek a range of views and inform the public.
He laid out a vision of a therapeutic approach to issues which drove criminal offending.
``We know the majority of those in prisons have issues other than they are nasty people.
``They have health issues and other problems and if we actually spent a bit of time on those things, we can stop their offending.''
He said other possible changes being considered were to the Parole Act 2002 and the Bail Amendment Act 2013 - considered two of the main drivers behind the prison population boom.
Mr Little said the Government faced three large issues - the question of Waikeria, criminal justice reform and the promised new 1800 police officers.
He said there was a risk with more police they would ``arrest more people which puts more pressure on prisons''. The three issues needed to be consistent with reducing the prison population.
Dropping the numbers of prisoners would symbolise a criminal justice system that was ``more humane and more effective'' because it targeted the causes of criminal offending ``for those for whom those causes can actually be fixed''.
Mr Little also believed prisoners serving three years or less should again have the right to vote.
It was time to stop treating prisoners as less than human, to give them a vote and a role in society.
He said the imbalance of Maori in prison - 52% of the 10,695 prison population - revealed systemic problems in the criminal justice system,
``There is a built-in systemic bias or prejudice and we've got to understand that. We've got to something about it.''
University of Canterbury criminologist Jarrod Gilbert likened Mr Little's rejection of ``knee-jerk'' policy-making to that of then minister of justice Ralph Hanan's success in ridding New Zealand of the death penalty and embarking on reforms.
University of Auckland professor of indigenous studies Tracey McIntosh was ``heartened'' by Mr Little's comments.
``I have always thought New Zealand could be a global leader in decarceration.''
Victoria University criminologist Elizabeth Stanley welcomed the opportunity for a criminal justice summit.
She said there had never been an opportunity for a proper conversation about penal reform.
