Sir Paul, New Zealand's first Maori Governor-General, was 78-years-old when he lost his battle to cancer.
Late last month he was diagnosed with cancer and announced he was stepping back from most aspects of his public life.
Sir Paul was Governor-General from 1985 to 1990.
Mr Key today said Sir Paul modelled his Governorship on the life of a bishop saying, "a bishop travels, a bishop stands alongside his people and searches for common ground".
"Sir Paul's contribution to New Zealand did not end when he left Government House. He spent another two decades serving at the highest level. We are indebted."
Sir Paul's was "a life spent giving", Mr Key said.
"His contribution was enormous and New Zealand is a poorer place for his passing."
A state funeral was being arranged before a burial at St John's Theological College.
Former Prime Minister Helen Clark also paid tribute to Sir Paul.
"Am deeply saddened by passing away today of Sir Paul Reeves, former New Zealand Governor General. Rest in peace, Paul," she tweeted.
Sir Paul's family will spend time with him until his tangi begins tomorrow at the of the Holy Sepulchre in Auckland, family spokesman Reverend Jayson Rhodes said.
"The whanau acknowledge and are very aware of the immense grief and loss felt by Maori, the church and the wider community, and there will be time and opportunity for people to pay their respects in the days to come."
Born in Wellington in 1932, Sir Paul was educated at Wellington College, Victoria University, St John's Theological College, Auckland, and St Peter's College, Oxford.
He became a deacon in the Anglican Church in 1958, a priest in 1960 and was made the Bishop of Waiapu in 1971, Bishop of Auckland in 1979 and Primate and Archbishop of New Zealand in 1980.
He was chairman of the Environmental Council from 1974 to 1976 and president of the National Council of Churches 1984-85.
Sir Paul belongs to the Puketapu hapu of the Te Atiawa of Taranaki.
After completing his term as Governor-General, Sir Paul took up a number of national and international positions, including that of Anglican Observer at the United Nations in New York, a position he held for three years.
Sir Paul was also deputy leader of the Commonwealth Observer group to South Africa and Chair of the Nelson Mandela Trust. He went on to chair the Fiji Constitution Review Commission from 1995 to 1997 and has also done similar work for the Commonwealth in Guyana, on constitutional reform.
He also observed elections in Ghana and South Africa on behalf of the Commonwealth.
He was admitted to the Order of New Zealand in 2007, the country's highest honour.
This followed being made a Knight Bachelor and Knight Grand Cross of the Order of St Michael and St George in 1985 and a Knight Grand Cross of the Royal Victoria Order in 1986.
He was also awarded the Queens' Service Order for public services in 1990.
In May 2002, Sir Paul was appointed inaugural Chair of Toi Te Taiao: The Bioethics Council.
He was elected Chancellor of AUT University in February 2005.
Green Party co-leader Metiria Turei said Sir Paul broke through many barriers to achieve what he did in his life, "particularly because he lived through a period of overt racism in New Zealand".
"Perhaps as a result, he was always a staunch advocate for those living in poverty and for peace and justice.
"In recent years he did great work to advance the Treaty Settlement process for his people.
"We will always be grateful for his advocacy and compassion," Mrs Turei said.
Anglican Church spokesman Archbishop David Moxon said Sir Paul "aimed to bring people together, to listen to people at the edges and to speak to issues".
"In his role he was always present amongst the people. He continued to follow the gospel and remained a bishop contributing a great deal to our church and community. Bishop Paul was and remains a taonga to us and future generations."
Sir Paul is survived by his wife Beverley, Lady Reeves, his three daughters, Sarah, Bridget and Jane, and six grandchildren.
