Ferguson took three wickets and conceded no runs from his four overs as New Zealand bowled PNG out for 78, after putting them into bat in Trinidad and Tobago. New Zealand chased down their target in the 13th over for the loss of three wickets.
The start of the match was delayed because of rain, but no overs were lost.
Finn Allen and Rachin Ravindra both went early, but 35 from Devon Conway, 19 from Daryl Mitchell and 18 from Kane Williamson guided the side to victory.
Ferguson's bowling figures meant he became just the second bowler to concede no runs from four overs in a T20 international, and the first at a World Cup. Saad Bin Zafar did it for Canada against Panama in 2021.
Trent Boult, Tim Southee and Ish Sodhi also took two wickets each.
It was an emotional day for Boult, playing his last T20 international for New Zealand. The 34-year-old took two wickets for 14 runs.
"Feels a little bit weird, a few emotions over the last couple of days.Looking at the tournament, obviously gutted to not go any further. It's an unusual feeling, but very proud of what I've done in the black cap.
"It's sad that it's my last stage here to be playing for New Zealand in T20 cricket."
Boult is unsure if he'll play any other form of cricket for New Zealand.
New Zealand were eliminated from contention for the Super Eight stage of the tournament after losing their first two group games to Afghanistan and West Indies.
Their next outing is a home test series against England in November and December.
