A net 30% of small businesses were pessimistic about general business conditions, the most since March 2009.
Confidence slipped in both micro firms, those employing up to five people, and intermediate-sized businesses — those employing between six and 20 people.
The gains from March disappeared in June, ANZ retail and business banking managing director Antonia Watson said.
Regulation remained the biggest concern facing small firms, particularly those in agriculture.
Difficulty in finding skilled employees was an ongoing challenge for many small businesses, particularly in construction, manufacturing, services and retail, she said.
External factors were also affecting confidence levels, including a cooling housing market and intense online and offshore competition for retailers.
The ANZ composite growth measure for small firms — a key proxy for GDP growth — fell across all sectors. Agriculture and construction reached negative territory, Ms Watson said.
The composite growth measure fell in all regions. Wellington remained the highest. Canterbury and other North Island areas fell into negative territory.
Sentiment remained pessimistic, suggesting small firms were still experiencing sustained demand for their goods and services but potentially at a decelerated pace.
Hiring intentions fell to 4%, slipping away from a short-lived recovery seen in March and affected the construction and agricultural sectors.
Profit expectations fell deeper, affected by the construction sector dropping 54 points to 14.
Rising construction costs and labour shortages were probably the reason for the negative feeling in the construction sector, she said.