Council property services group manager Dave Bainbridge said asbestos had been found in the building, which has an asbestos roof, last year and since then access to the site had been restricted.
Asbestos at levels which could be harmful to people's health had been identified by a specialist, Mr Bainbridge said.
It would take about five weeks for the demolition to be completed and the area would then be resurfaced and additional car parks would be made available.