Many people have taken to social media expressing their concerns about the alert, whether it be not receiving it or being shocked by the piercing sound.
The alert was sent out between 6pm and 7pm on Sunday to all mobile phones in New Zealand but was only picked up by around 2 million cell phones.
Sarah Stuart-Black, director of the Ministry of Civil Defence and Emergency Management, said the test met expectations, going through to around a third of New Zealanders.
"The initial signs are that the test was successful. People received the alert from the top of the North Island to the bottom of the South Island so the system itself worked seamlessly," she said.
She said there was some variability in how different handsets behaved. More people would be included in the system over time as they upgrade their phones or download the latest software updates.
The technology could not be picked up by all models of phones. Stuart-Black said they were looking at this through a survey people were encouraged to complete online following the test.
"One of the things that will come out of the survey is how the technology works with different phones," he said.
"Every handset is different, the alert overrides silent mode on some of them but not others."
Stuart-Black said the survey would help Civil Defence figure out which handsets worked with the system and which didn't. She also encouraged people to look at the setting their phone was on when the alert came through and note whether it made a sound.
Around 15,000 people had filled in the survey by late on Sunday night.
While there were still some people who complained they hadn't been warned the alert was coming, Stuart-Black said Civil Defence had done its absolute best to inform the public.
"We always understood it wouldn't have connected with everyone but we had a consolidated effort in the leadup to make sure people knew it would happen."
Andrew Leys called the Herald to complain that he and his wife had no warning of the impending emergency alarm.
"None of us had any idea it was going to happen. We were aware that they could access the phones and they were talking about deploying the technology, that was all."
He said Civil Defence has a responsibility to send out a text warning prior to the emergency alert so as not to frighten people.
"Why do they expect that everybody is going to read the media on the day they're doing a test - some of us have lives. They could have simply texted us."
But Stuart-Black said Civil Defence actually doesn't have access to anyone's phone number - which is why it was hard to tell how many people have received the alert.
Why you didn't receive the alert:
Your phone was in flight mode
If your phone was on aeroplane mode you will not have received the alert.
Civil Defence spokesman Anthony Firth said the setting effectively did the same as turning your phone off.
Your iPhone doesn't have the latest update
If the software on your iPhone isn't up to date you likely will not have received the alert.
If your phone is on Civil Defence's list of mobile alert capable phones then you should check that your software is updated.
Your model of phone isn't supported
Civil defence has a list online of the phones that are compatible with the alert system.
If you have an older or unusual model of phone it might not be able to receive Emergency Mobile Alerts.
You have unselected Emergency Alerts on your phone
The setting to opt in or out of Emergency Alerts can be found under 'settings' on most smartphones.
Make sure you have this turned on.
