The economist has just written a book looking at the issue, called Appetite for Destruction.
He told TVNZ's Q+A programme fast food was only one factor of the problem, but the main focus needed to be on processed foods sold in supermarkets.
"You go through a supermarket and it's virtually all now processed food, which is not a crime in it's own right, obviously, but when you do the decomposition of processed food and you see how energy intensive it is and how nutrient light it is, our bodies simply can't handle it."
The food manufacturing industry had made far more food available globally, so fewer people go hungry, Dr Morgan said.
"But it's the quality of food I'm talking about."
Health professionals should be involved in creating a system, such as traffic light colours, where a red light labels on foods attracted the highest tax, Dr Morgan said.
He dismissed the Labour Party idea of removing GST from fresh fruits and vegetables.
"I wouldn't interfere with GST, it's the wrong thing to do - the whole idea of GST is to have complete coverage with no exceptions."
He said an excise tax, similar to the one for tobacco was a better option.
