Do we rely on rankings research or do we stick to the facts?
During 2023 University of Otago-led research claims to have established that alcohol causes more harm to New Zealanders than meth and other illegal drugs.
A recent opinion in the ODT by Dr Rose Crossin and Prof Joe Boden from the University of Otago, who were co-authors of the rankings research, supports new legislation that regulates all psychoactive drugs including those that are legal or prohibited. They reference the rankings research from 2023 in their opinion.
But the facts behind the research — though not as attention-grabbing — call into question the soundness of the claims.
The research in question refers to two "facilitated workshops" attended by experts from across medicine, psychology, justice, community services and harm reduction. They got together and ranked 23 different drugs for their perceived harm using 17 "harm criteria", some centred on harm to the user and others relating to harm caused to others.
When considering harm to users, the experts were asked to think about "a single person who typically uses that drug in what was considered to be a typical way".
It’s not difficult to see the enormous amount of subjectivity potentially involved in this approach. It’s hard to imagine what a typical user of any drug might be, let alone a typical consumer of alcohol, which was — by far — the most widely used substance under consideration and therefore would have the greatest possible variety of consumers.
Given that there is actual data that proves that 84% of adult New Zealanders consume alcohol in a responsible manner — i.e, at a level in keeping with recommendations based on decades of research — one wonders what each expert’s typical user looked like when they were assessing harm ratings to arrive at the above results.
Further, the research — by its own admission — makes no allowance for prevalence of use, even though prevalence of use clearly impacted the results. Alcohol is legal and more widely consumed than any other substance with which it was compared, so it’s perhaps unsurprising it was deemed to be associated with the greatest overall harms.
For argument’s sake, if heroin were as widely consumed, one would expect its overall harm rating would be exponentially higher than the harm rating it was given.
Perhaps another way to look at this: if sugar were included in the list of substances considered by the group, isn’t it possible that its harm would outweigh any other?
Such examples demonstrate that in a comparative numerical rating process, prevalence of use invariably affects the outcome.
Further, there are myriad other contextual factors that a reductive process like that described above can not possibly take into account.
The journal article also tells us that the results of the rating exercise were reviewed by the panels before they were finalised to "ensure that the final rankings demonstrated face validity". In other words, the panelists were given an opportunity to tweak the ratings before they were finalised if the group didn’t like what they saw. This is hardly an objective process by any measure!
It’s disconcerting that such "science" might be used to inform the regulatory and policy framework around alcohol.
Techniques such as those described above — dubbed multi-criteria decision analysis — might be useful in some settings, but can they be regarded as authoritative science?
Prof Jonathan Caulkins, at Carnegie Mellon University and, his colleagues from America’s Society for the Study of Addiction do not think so, describing such efforts as "fundamentally flawed, both conceptually and methodologically".
The British government didn’t seem to think so either. Similar research to the University of Otago’s was conducted in the UK about 15 years ago under the leadership of Prof David Nutt who — perhaps unsurprisingly — is one of the authors of the Otago study.
Nutt was fired from his role as chairman of the government’s Advisory Council on the Misuse of Drugs by Britain’s home secretary in 2009 after producing a paper which claimed the Class A drug ecstasy was safer than equestrian sports.
Nutt is now co-founder and currently chief scientific officer of a company that develops alcohol alternative beverages — a conflict-of-interest that the University of Otago research paper fails to mention.
Perhaps that’s an unwelcome distraction from the point, which is that policy and regulations around alcohol should be guided by facts and science, not subjective, composite rankings.
We all know that alcohol abuse causes both individual and social harm, which is why New Zealand’s guidelines for alcohol consumption provide guidelines for responsible alcohol consumption and why alcohol sale and supply in New Zealand is regulated.
— Virginia Nicholls is the executive director of the New Zealand Alcohol Beverages Council.
About the NZ Alcohol Beverages Council: The council is a pan-industry group that comments publicly on matters relating to the beer, wine, spirits and beverage industry. It focuses on supporting responsible alcohol consumption and advocates for a fair and balanced debate on alcohol regulation in New Zealand.
