Mr English was commenting on the Government's financial statements for the year ended June, which showed a remarkable turnaround from the previous corresponding period.
The operating deficit more than halved to $4.5 billion from $10.5 billion in 2009.
The forecast deficit was $5.7 billion.
Lately, Mr English has preferred to focus on the operating balance before gains and losses (obegal), which increased $2.4 billion from last year to reach $6.3 billion on June 30.
Recovering investment markets contributed to the Crown making a net gain for the year of $1.8 billion, recovering some of the $6.6 billion loss incurred in the previous year.
Mr English said the recovery was not being led by traditional "sugar fixes" of borrowing, consumption and retail spending.
"New Zealanders understand our need to rebalance the economy away from debt and spending towards savings and investment.
"The Government's economic programme - including the tax package this month - aims to achieve that over the next few years so we can create faster growth and sustainable jobs."
Signs of progress were clear from estimates of households' equity in their homes, he said.
Treasury analysis showed households withdrew several billion dollars of equity from their homes between 2003 and 2008, effectively borrowing to boost their disposable incomes.
Households were now reducing borrowing and injecting equity into their homes, effectively saving, Mr English said.
In the year to March 31, that equity injection amounted to about $5 billion.
"That's a significant change, equivalent to about a 10% reduction in household incomes available for spending.
It remains to be seen whether this trend continues but the early signs are promising."
At the same time, household consumption had fallen from a peak of more than 64% of GDP in 2007 to 62% this year.
The Reserve Bank was forecasting that would fall further to about 60% of GDP by 2013.
That was why the current recovery would remain "quite different" to past ones.
It would not be fuelled by debt and consumption, Mr English said.
"While this will make it challenging for retailers and other domestic industries in the short term, it is what the economy needs over the long term as we build our future on savings, productive investment and exports," he said.
Annual nominal GDP growth remained positive, as it was supported by the strength of New Zealand's trading partners, particularly Australia and China.
Strong demand from China led to a rapid recovery in New Zealand commodity prices and terms of trade, boosting nominal GDP in the March and June quarters of 2010.
While core Crown revenue fell for the second consecutive year, expenses remained flat year-on-year.
Treasury said that increasing divergency had led to a continuation of operating deficits.
Core Crown tax revenue fell as both tax cuts and recessionary impacts continued to reduce the Crown's income.
Operating deficits had led to a decline in net worth for the Crown of $4.5 billion to just below $95 billion at balance date.
Health and education expenses increased by $1.1 billion in the year and benefit expenses increased by $1.5 billion across several benefit types.
The largest increases were to New Zealand Superannuation and unemployment benefits, which rose by $500 million and $300 million respectively.
Council of Trade Unions economist Bill Rosenberg said there was still room for fiscal stimulus for an economy that was stalling.
"The minister of finance says the very slow recovery is due to people reducing borrowing on their homes instead of spending on goods and services.
"However, we cannot be confident this is a trend rather than a sign of people's concern about the future and the difficulty in borrowing."
A stimulus could be applied in ways that improved our infrastructure and housing, he said.
