Foreign Affairs Minister Winston Peters removed Goff from his post as High Commissioner to the United Kingdom, after comments he made about US President Donald Trump at a Chatham House event in London.
Peters said Goff's conduct was not the way a diplomat should behave, and left his position "untenable".
Chris Seed, who left MFAT just over a year ago, will serve as Acting High Commissioner while the search continues for a permanent option.
He will travel to London late next week.
He previously served as New Zealand's High Commissioner to Australia and Papua New Guinea.
Peters said Seed was one of New Zealand's "most senior and accomplished diplomats."
Peters acknowledged the difficulty of the decision and wished Goff well, saying he would be held in "high esteem".
"Mr Goff has contributed significantly to the interests of our country over a long period of time.
"As a senior minister with a range of domestic and internationally focused portfolios, mayor of Auckland, and then as a diplomat, Mr Goff has dedicated his professional life to serving the New Zealand people. We continue to hold him in high esteem, and we wish him well."
An announcement will be made in due course about a permanent replacement, once approval has been received from the UK government.
Prime Minister Christopher Luxon said he was happy with the appointment.
"I'm very comfortable with Chris Seed, he's a really accomplished and seasoned diplomat, and that's what we need in that role. We need our diplomats to be diplomatic, and I fully support that decision."
Labour leader Chris Hipkins was similarly complimentary about Seed's credentials.
"I've got a lot of time for Chris Seed. He's got a very well-respected public service history. I am sad for Phil Goff, as I said last week I think Phil's comments were a step too far for an Ambassador, but I am concerned that bringing him home is a bit of a kneejerk response."
