Trey Ratcliff was excited to show his friend Sam Wave what he had been working on during lockdown.
"I had just had this rather intense psychedelic experience," Ratcliff, world-renowned travel photographer and immersive mindfulness exponent, says.
"I thought, wow, that’s so beautiful. How can I try to recreate that using [computer] code?"
It was the fruit of that labour, birthed at his Queenstown home during the first lockdown of the Covid-19 global pandemic, that Ratcliff shared with Arrowtown-based indie musician, Wave (AKA Sam Mehrtens).
"It was great, but it needed music. I thought, custom music from a true craftsman like Sam would be perfect," Trey recalls.
Wave was equally enthusiastic. Machine Elf was born — a collaboration producing visual and audio VR creations that have been viewed by more than 300,000 people seeking calmness or escapism.
It was also during the March, 2020 lockdown that Dunedin graphic artist Max Mollison used the enforced break to continue honing his AR skills. Now, amid growing global recognition, Mollison has brought those skills to bear on a new exhibition at Otago Museum, for which he was given a wide open brief.
"The initial brief was just ‘Make something sea monster-related, but abstract, kind of chrome-effect’," Craig Scott, head of the museum’s exhibitions and creative services, says.
"Go nuts and make something cool," adds museum marketing manager Kate Oktay.
"And he has."
The result is a cutting-edge exhibition, accessed through smart phones, that populates the gallery space with flying sea monsters and other aquatic augmentations.
VR is an artificial, 3-D, digital environment that people can immerse themselves in, often via VR goggles.
AR overlays the real world with digital objects or information, most often viewed on smart phones (although AR glasses are increasingly available).
Highlighting this technological sea change, the 2021 Technology Investment Network (Tin) report, released last month, revealed New Zealand’s top 200 revenue-earning, high-tech companies recorded $10.4 billion in export revenue despite the disruption of the pandemic. This was a 14.4% increase on the previous year. It means New Zealand technology companies are now the country’s second highest offshore revenue-earning sector.
In fact, that growth appears to be largely because of, not despite, Covid.
“It is difficult for most Kiwis to think about what the world was like before the Covid pandemic, but the nation’s digital future is no longer a conversation about what the future may hold, because it is happening now, since Covid arrived,” Pham said.
The VR and AR slice of this burgeoning tech pie is growing much more quickly than the whole.
A new report by global tech market research firm IDC, released in October, predicts the augmented and virtual reality market will grow eight-fold during the next five years.
This year the global VR and AR market is worth $6.56 billion. By the start of 2026, that is expected to be $53.48 billion.
It means AR and VR is the fastest growing category among emerging smart devices, including high-tech wearables and smart home devices.
Much of the growth in virtual and augmented reality is coming from the ever-more immersive online gaming world, of which Dunedin wants to become the hub for a high-flying New Zealand game development sector. The city is already home to the government-backed New Zealand Centre of Digital Excellence (Code). Last week, Code gave nine Dunedin game development studios grants totalling $1.19 million.
But AR and VR is not only about gamers using their phones to capture digital monsters hidden in the real world or wearing goggles to do battle in a zombie-infested wild west.
Sam Wave has been living in and around Queenstown for more than a decade, since completing studies in philosophy and marketing at the University of Otago. He turns his hand to film editing and other creative income-earners while pursuing his music-making passion. It is not easy in a Covid-altered world that can suddenly cancel shows and tours. But it has allowed him to produce his second album Oasis Ballerina and create accompanying digital content in partnership with Trey Ratcliff.
Ratcliff had already pioneered High Dynamic Range (HDR) photography and established the world’s top travel photography blog StuckInCustoms by the time he moved his family from Austin, Texas to Queenstown a couple of years before Wave moved there. His photographic artwork is owned by collectors including US actor Leonardo DiCaprio and New Zealand businessman Michael Hill. A recent foray into non-fungible tokens (NFTs) — a digital collision of art and cryptocurrency — has earned him an astonishing $6 million in just a few months.
Fractals are infinitely complex patterns that appear the same at different scales, such as snow flakes and trees branching. Abstract fractals can be created by instructing a computer to repeatedly calculate an equation, for which it produces a visual representation.
"It’s a fun and somewhat esoteric practice," Ratcliff says.
The video fractals Ratcliff produces are then matched to music by Wave or other musicians; the music then interacting with the images, further changing colours and shapes.
"They are beautifully intertwined."
Dozens of Machine Elf’s visual and audio art compositions have been uploaded to YouTube, where they have quickly gained a large and enthusiastic following. They have also been picked up by online wellness company Tripp.
The aim of the pieces is to offer escapism and visual mindfulness, Wave and Ratcliff say.
Feedback from viewers suggests the combination of visual and audio elements in an immersive VR environment helps people take a break from their over-active or anxious thoughts.
"We’ve found that if you put people in these visual meditations, these visual fractals, especially with a Virtual Reality headset on, that it really relaxes the mind, almost like a forced meditation," Ratcliff says.
Max Mollison is back in Dunedin, adding an augmented layer of fantasy and facilitation to the city that gave him birth.
Raised in Dunedin, he studied fashion in Wellington after finishing his art-heavy secondary education at Kavanagh College. He then went to Orlando, Florida, to work as an intern at Walt Disney World Resort before more interning, this time for a fashion label in Sydney, Australia.
Mollison came in contact with Otago Museum in 2017, when he made a piece of clothing for a museum exhibition. He then worked for the museum for two years, mostly as a traditional media graphic artist.
He took time off last year, during which he honed AR software skills he had been developing for a couple of years.
"I really enjoyed using it as an extension of fashion design; an adornment, decoration, that people could try on and which a lot more people could access ... They could try this augmented reality look on themselves."
Mollison has an active presence on social media, showcasing the fashion, fonts and AR filters he creates. His more-than-130 posts on video-based social media platform TikTok have garnered him 13,700 followers and 166,700 likes.
Recently, pop star and trans icon Kim Petras, who attended this year’s MTV Video Music Awards with Paris Hilton, posted a TikTok to promote her new single using an AR filter made by Mollison.
Now back at Otago Museum, he has created Plunge — a free attraction to complement the museum’s latest headline exhibition, Sea Monsters: Prehistoric Ocean Predators, which opened this month.
Visitors, when they enter the Plunge gallery space, scan QR codes enabling them to see a watery world populated with digital 3-D sea monsters, ancient and modern, with scales and flesh of chrome, glass and water, swimming around the transformed room or circling the heads of other patrons. Some QR codes reveal water orbs or grant the power to generate and throw AR bubbles.
A solely-AR exhibition created by a museum’s in-house design team is a New Zealand first, placing the province and its museum on the crest of the technology wave.
Mollison built the exhibition almost single-handedly using open source, 3-D, computer graphics software Blender and the Spark AR Studio software that works on Facebook and other Meta apps such as Instagram.
A technical difficulty was keeping the file size for each of the exhibitions’ 10 interactive elements within 4MB.
"Trying to get everything down into one small package is quite hard considering some of these things are quite elaborate," Mollison explains.
The museum is excited about the AR exhibition and its potential, Oktay says.
Museums worldwide are seeing declining numbers of youth and 20-somethings coming through their doors.
"What Max is doing really speaks to that generation and brings a whole fresh ... reason for them to come and engage with institutions and collections like this."
