Gao, 33, stole $6.7 million after discovering $10m in his bank account due to a major bank error.
Instead of returning the money, Gao embarked on an overseas spending spree, splashing out at glitzy hotels and casinos.
Gao's then-partner, Kara Hurring, joined him in Macau.
Eventually, the pair were caught, and Gao - who had spent two and a half years on the run - was sentenced to four years and seven months imprisonment in August last year.
The Parole Board confirmed he was released from prison on parole yesterday.
Gao had spent time in a Hong Kong jail, in Auckland prison and on electronically-monitored bail prior to the commencement of his sentence.
The Parole Board's decision, released earlier this month, said despite the sizeable nature of his fraud, it was satisfied his risk of offending could be managed by imposing strict release conditions.
These included restrictions on interactions with Ms Hurring and access to bank accounts.
While in prison, Gao was considered a low-risk offender, the decision said.
Gao had also admitted his actions had caused great shame, had broken the law and contradicted his family's values.
The Parole Board found he did not pose an undue risk to the community, and refusing him parole once his eligibility date was reached would therefore be wrong.
Ms Hurring, who has a son with Gao, was sentenced to nine months' home detention over her role in the theft.
Almost $3.8m is still missing from the $10m transferred into Gao's bank account in April 2009.
