Researchers say the ancient penguin, called Kumimanu biceae, weighed about 100kg, and was much bigger than the largest of these flightless seabirds alive today, the emperor penguin, which grows to about 4-1/4 feet (1.2 meters) and about 40kg.
The bones were found on Hampden Beach in North Otago.
The only ancient penguin yet discovered that might have been larger than Kumimanu is known only from a leg bone, said ornithologist Gerald Mayr of the Senckenberg Research Institute and Natural History Museum Frankfurt.
"Gigantism in penguins evolved more than once," Mayr said.
Kumimanu, named after a creature from Maori folklore and the Maori word for bird, is the second-oldest known penguin. The older one, also from New Zealand, was 61 million years old.
Kumimanu's partial skeleton lacks the skull. Mayr said other fossils indicate that the earliest penguins possessed much longer beaks than their modern relatives, useful for spearing fish.
"It would have been very impressive: as tall as many people, and a very solid, muscly animal built to withstand frequent deep dives to catch its prey," said Alan Tennyson, vertebrate curator at the Museum of New Zealand Te Papa Tongarewa, another of the researchers in the study published in the journal Nature Communications.
"It would not have been the kind of bird that someone could catch alive. It would have been considerably more powerful than a person."
Kumimanu and other early penguins had already developed typical penguin features including flipper-like wings and an upright stance. Studies suggest early penguins were brownish, not the trademark black and white of today's penguins, Mayr said.
Penguins are thought to have evolved from a flying ancestor perhaps resembling a cormorant, Mayr said. The asteroid that doomed the dinosaurs 66 million years ago also eliminated the large marine reptiles that dominated the seas, clearing the way for fish-eating divers like penguins.
Kumimanu lived long before Antarctica's glaciation. At the time, New Zealand and Antarctica were subtropical.
"It's a common myth that penguins only live in very cold environments such as the Antarctic region," Tennyson said. "Today, Galapagos penguins live at the equator, and many fossils show that early forms of penguins lived in warm seas."