Mr Jones said he had told Labour leader David Cunliffe of his decision as well as the Party President Moira Coatsworth.
He has been in talks with Foreign Minister Murray McCully for some time about a possible role in international fisheries management and expected that to go through.
He said he would leave at the end of next month after reflecting over Easter about his future.
"I was true to myself after I lost the leadership. I'm not able to give Labour the 100 per cent that I ought to be giving. They need a team that will stay for the long term and I'm not in that space.''
TV3 reported Mr Jones would be taking on a role of Pacific Economic Ambassador, created by the National Government especially for him.
Foreign Minister Murray McCully confirmed he had approached Mr Jones early in the year about a role focusing on economic development across the Pacific.
He denied the role was created for Jones, saying he would have appointed somebody else if Jones declined.
He said the final discussions were yet to be held, but he had made it clear to Mr Jones that he was Mr McCully's first choice because of his background on Te Ohu Kaimoana (Maori Fisheries Commission) and his commercial skills.
He said the role would involve developing the aid New Zealand provides to Pacific countries for fisheries as well as other economic development areas.
Senior members of Labour were not aware of his decision to leave.
Mr Jones entered Parliament in 2005 and was promoted to Cabinet in November 2007 as Minister for Building and Construction. He was re-elected to Parliament in 2008 and has served as spokesman for Maori Affairs and Economic Development, and associate spokesperson for Finance.
Last year, he contested the leadership of the Labour Party in a leadership election, but lost to David Cunliffe.
In recent months, he has been an outspoken critic of supermarket chains using their buying power to influence suppliers.
