In Budget 2013 today, the Minister for Tertiary Education, Skills and Employment further reined in measures on student loans, but said new sanctions allowing the arrest at the border of student loan borrowers who have defaulted on their repayments are just for extreme cases with high levels of default.
Mr Joyce said nearly half a billion dollars was owed by overseas-based student loan borrowers.
He was not being harsh, he said, as the sanctions would apply only to the worst of the worst.
"It would only be for the people who are deliberately refusing to take any notice.
"We are talking about people that have been contacted by IRD and are refusing to act and then effectively deliberately defaulting," Mr Joyce said.
It is unclear if that assurance will help quell student protests this year.
Last year protest erupted just days after Budget 2012 when changes were announced increasing the student loan repayment rate from 10 cents to 12 cents in the dollar, removing the voluntary repayment bonus and removing eligibility for student allowances for postgraduate study.
Students in Auckland, Wellington and smaller centres took to the streets, tipping over rubbish bins, and lighting small fires to protest the changes.
Around 400 University of Auckland students blockaded Symonds Street.
Included in Budget changes today, repayment thresholds would be increased for overseas-based student loan holders, and fixed repayment obligations would be introduced.
A borrower with a $20,000 loan balance would pay it off in 15 years, rather than in 35 years under the old rules.
The interest they would pay would halve from $19,940 over 35-years, to $8425 over 15 years.
Mr Joyce has been organising for a private debt collection agency in Australia to help find defaulting borrowers.
"This tracing activity involves a group of 10,000 borrowers, believed to be in Australia, who have made little or no effort to repay their debt, or even make contact with Inland Revenue," Mr Joyce said.
While the Government had already reined in $60 million from overseas student loan borrowers, according to Revenue Minister Peter Dunne, Mr Joyce said they needed to "up the ante".
The cost to the taxpayer had already dropped from 48 cents in the dollar in 2009, to 39 cents in the dollar by March his year.
Mr Dunne said 14 per cent of overseas-based borrowers did not meet the interest charged on their loans.
Labour's tertiary education spokeswoman Megan Woods feared the move was a continuation of the moves in the last Budget.
"Particularly the reduction of the repayment holiday from three years to one year - it doesn't allow New Zealanders that freedom to go away in that important part of career development overseas and then come back with all these skills.
"This is a really punitive approach."