For the past six years, the confectionery company has been voted the most trusted brand in the country, but its support has crumbled to 36th out of 133 brands in the latest Reader's Digest trust survey.
Cadbury spokesman Daniel Ellis said the company, which has a factory in Dunedin, was not surprised to lose the top spot as the year had been a "challenging one".
Last year, the company briefly replaced cocoa butter with palm oil in a cost-saving measure and moved the production of some of its Kiwi favourites - such as Moro bars - overseas.
"We've undergone many changes in the past two years, some of which we know have not been popular but were necessary to ensure we keep our Dunedin factory going strong and providing local jobs."
Asked if the company could retain the top spot again, Mr Ellis told the Otago Daily Times, "we believe we will".
"We know many people have been disappointed by the changes but we're confident that we are making significant steps in the right direction.
The love and trust Kiwis have shown us is extremely important to us and respecting this love is our top priority."
Despite the dip in consumer confidence, sales had remained strong, he said.
However, in another bitter blow for Cadbury, New Zealand chocolate rival Whittaker's debuted on the list at number five.
Food producer Wattie's won the top spot, followed by car maker Toyota, electronics company Sony and ice cream maker Tip-Top.
Top ranking brand
1.Wattie's (9)*
2.Toyota (4)
3.Sony (3)
4.Tip Top (2)
5.Whittaker's (n/a)
6.Panadol (5)
7.Fisher and Paykel (7)
8.Panasonic (9)
9.Colgate (8)
10.New Zealand (13)
*(Last year's position given in brackets)
Category winners
Bank: Kiwibank
Beauty and Cosmetics: Gillette
Cars: Toyota
Computers: Sony
Credit Cards: Visa
Electronics: Sony
Fast food: Subway
Food: Wattie's
Health and wellbeing: Panadol
Home improvement: Mitre 10
Mobile phones: Nokia
Retail: New Zealand Post
Telecommunications: Vodafone
Travel: Air New Zealand
Whitegoods: Fisher and Paykel
