The warning came as Mr Orders confirmed the council was set to remain beyond a self-imposed debt ratio limit for at least the next three-year council term.
The council's 2013-14 pre-draft budget - to be considered by councillors later this week - showed the council would begin repaying more debt than it was borrowing for the first time in 10 years.
Dunedin Mayor Dave Cull said that showed the council had ''turned a corner'', although the organisation's finances would remain ''tight'' for years to come.
''I think for the next few years it's important for council to discipline its spending,'' he said.
However, the size of the council's debt meant it would still be operating beyond its self-imposed limit, which sought to restrict interest as a percentage of total revenue to no more than 8%, until 2016-17, Mr Orders confirmed.
''The position then becomes more comfortable, but at the moment it's still quite finely balanced,'' he said.
Mr Cull said the council's 8% debt ratio was ''quite low'' compared with other councils, although ''it would be better to get it down''.
Core council debt, excluding companies, was due to peak at $271.7 million in 2012-13, and would then begin a gradual decline towards a goal of $200 million by 2021-22.
Consolidated debt - across the council and its companies, including stadium debt - was also due to decline, after hitting $616 million in 2011-12.
The drop in core council debt reflected cost-cutting that had halved capital spending from $105 million to $49 million and reduced new debt from $55.6 million to $9.3 million, from 2012-13 to 2013-14.
''That's a major change in terms of the structure of budgets in the council,'' Mr Orders said.
It also reflected the council's new financial strategy, included in last year's long-term plan, and a warning by Standard and Poor's, which said the council's ''high debt burden'' weighed on the council's credit rating.
Despite that, Mr Orders said the council would still have ''little or no'' headroom for new spending until 2022.
Councillors would also have to decide between a smaller 2.8% rates rise or using $1.4 million in additional savings, either to accelerate debt repayments orpursue another priority project.
Spending the $1.4 million would push the rates rise up to 4%, but Mr Orders' report to councillors said opportunities to increase debt repayments should be taken ''wherever practicable''.
Stadium debt was scheduled to be repaid over 23.5 years, after councillors approved an extra $1 million a year in debt repayments last year, but the cost of servicing the loans remained ''significant'', Mr Orders said.
''Levels of debt are still high ... you cannot say we are in a comfortable position - far from it.''
However, if the council stuck to its long-term plan, the amount of debt being paid off would gradually rise and the situation would improve, he said.
''You start to get to a position, provided council sticks to its long-term plan ... where, year on year, you are increasing your repayment of debt, as opposed to how much you are borrowing.
''That's a crucial issue in terms of managing council finances.''
