Little first entered Parliament in 2011, and led the party in the lead-up to the 2017 election but stood down to make way for Jacinda Ardern's leadership in the face of poor personal polling.
Little said he wouldn't take up his list spot with the Labour party in Opposition after the party's loss at the weekend's election.
He said it was important to give the next Labour government every opportunity "to hone their skills and cement a strong team".
"The Labour movement is bigger than any caucus or parliamentary term, and bigger than any individual," he said.
"I'm grateful for the opportunities my country and party have given me to serve the community."
"Labour stands for the fundamental values of millions of New Zealanders: a fair day’s pay for a fair day’s work, opportunities for the many not just the few, and respect for all people and communities.
“There are no words to adequately thank my wife and son, and the many others who have given unwavering support throughout my time in politics."
Little has held high-profile portfolios including Health, Justice, Immigration, Defence, Courts, Workplace Relations, Treaty Negotiations, and Public Service.
He has also held more specialised roles as Minister for Pike River Re-entry, and the Government's Response to the Royal Commission's Report into the Terrorist Attack on the Christchurch Mosques.
Little's decision not to take up the list slot will mean Shanan Halbert can return - as he is next on Labour's list.
Halbert lost his Northcote seat at the election.
Labour leader Chris Hipkins said he’d had a conversation over the weekend with Little, so his resignation wasn’t a surprise.
He acknowledged Little’s humility for putting the interests of the Labour team first.
He said it was “admirable” that he had decided to move on and create space for someone else.
Hipkins described the mood as “bittersweet” and said there was a “certain law of gravity” with election results.
“Good people’s lives get affected by that and we’ve fare-welled some incredible colleagues this morning.”
He accepted emotions were running high.
“Everyone will accept that it’s a difficult time, election defeats are never easy.”
- RNZ/NZ Herald