A council spokesman said the tree — which would normally require notification for removal as it was on the significant tree register — needed to be cut down as investigations into safely replacing the retaining wall without removal of the tree had been unsuccessful.
The work was being done under urgency as it was probable the collapse of the retaining wall would damage the neighbouring house, he said.
Full removal was expected by the end of the week and Olveston House was looking for ways to repurpose the wood from the felled tree.