The change takes effect next year and saves the government $877 million over the next four years.
The Budget also increases tertiary institutions' government subsidies 2.5 percent at a cost of $266 million over four years.
Institutions will also be allowed to increase the fees they charge students by up to six percent.
The tertiary education budget includes 65 million over four years to continue monthly payments to employers who employ apprentices.
Eligibility for the scheme will start from January 2025 and be based on similar settings to the previous scheme. It will include provider-based and work-based learners studying at Level 3 and above on the NZQA framework.
The government is also increasing the interest rate charged to overseas student loan borrowers by one percent for five years, from 3.8 percent to 4.8 percent.
Late payment interest for overseas and New Zealand based borrowers will also increase by one percent.