Police were alerted to the presence of the mortar at a property on Guildford St, Burnside, about 9.40am on Thursday.
"An NZDF Explosive Ordnance Disposal team was advised and has now safely removed the object," said a police spokeswoman.
"A cordon was initially in place as a precaution but has now been removed."
A mortar fires explosive projectiles called mortar bombs at low speeds for short ranges and at high-arcing trajectories.
Mortars have been used for hundreds of years, originally in siege warfare. By World War 2, mortars could fire as many as 30 bombs per minute, and had a range of more than 2300m with different shell types.