Malinga claimed his third ODI hat-trick, a record at this level, as Australia's innings crumbled with the final five wickets falling for just one run to post 211 after being put in by Sri Lanka captain Tillakaratne Dilshan.
In reply, in spite of early scalps from Mitchell Johnson and youngster James Pattinson, a 111-run stand between Chamara Silva (63) and Mahela Jayawardene (71) for the fourth wicket took the match away from Australia.
Despite the defeat, Australia still claim a 3-2 series victory going into the three-test campaign beginning in Galle on August 31.
Australia seemed set for a total around 250 before Ajantha Mendis (3-49) broke a 43-run sixth wicket stand between David Hussey (46) and Brad Haddin (8) to begin a remarkable collapse.
The next over Malinga (3-35) dismissed Johnson (1), John Hastings (0) and Xavier Doherty (0) to send the home crowd at the Premadasa Stadium into a frenzy.
Mendis then returned and with the first ball of the following over removed Haddin to complete Australia's demise.
Things started promisingly with ball for the tourists with Johnson (1-45) removing Upul Tharanga (0) before Pattinson (2-41), in his first game on tour, dismissed Kumar Sangakkara (7) and Dilshan (17) to have Sri Lanka 3-33.
Silva and Jayawardene settled things, and made the most of some errant Australian bowling which had them concede 15 wides inside 20 overs.
Despite the fall of Silva to Doherty (2-54) and a brief rain delay, the hosts coasted to a comfortable win which was only slightly blighted by the late dismissal of Jayawardene.