Other realities: The artists who enlarge our world

A first augmented reality (AR) sea monster exhibition in New Zealand, Dunedin, and psychedelic virtual reality (VR) experiences created in Queenstown are surprising and fascinating examples of how AR and VR lead a global technological transformation accelerated by Covid. Bruce Munro takes on a real, augmented and virtual look.

Trey Ratcliff was excited to show his friend Sam Wave what he was working on during the lockdown.

“I had just had this rather intense psychedelic experience,” said Ratcliff, world-renowned travel photographer and immersive mindfulness exhibitor.

“I thought, wow, that’s so beautiful. How can I try to recreate this using [computer] coded?”

It’s the fruit of that labor, born in his Queenstown home during the first lockdown of the global Covid-19 pandemic, that Ratcliff shared with Arrowtown-based independent musician Wave (AKA Sam Mehrtens).

“It was great, but it needed music. I thought personalized music from a real craftsman like Sam would be perfect,” Trey recalls.

Wave was equally enthusiastic. Machine Elf was born – a collaboration producing visual and audio VR creations that have been seen by over 300,000 people seeking calm or escape.

It was also during the March 2020 lockdown that Dunedin graphic designer Max Mollison used the forced break to continue honing his AR skills. Today, amid growing global recognition, Mollison put his skills to good use in a new exhibit at the Otago Museum, for which he received an open briefing.

“The initial brief was just ‘Do something sea monster related, but abstract, kind of chrome effect,'” said Craig Scott, head of exhibitions and creative services at the museum.

“Go crazy and do something cool,” adds Kate Oktay, the museum’s marketing director.

“And he did.”

The result is a cutting-edge exhibit, accessible via smartphones, that populates the gallery space with flying sea monsters and other aquatic augmentations.

These diverse creative expressions, both in Otago, are signs that the world is in the midst of a Covid-accelerated technological transformation, with virtual reality and augmented reality leading the way.

Virtual reality is an artificial 3D digital environment that people can immerse themselves in, often through virtual reality glasses.

Augmented reality covers the real world of digital objects or information, most often viewed on smartphones (although augmented reality glasses are increasingly available).

Highlighting this radical technological change, the 2021 Technology Investment Network (Tin) report, released last month, found that New Zealand’s 200 top-grossing high-tech companies recorded $ 10.4 billion in revenue from export despite the disruption of the pandemic. This is an increase of 14.4% over the previous year. This means that New Zealand tech companies are now the country’s second largest offshore revenue generating sector.

In fact, much of this growth appears to be due, and not despite, to Covid.

In his 2021 annual report to the New Zealand Tech Sector Association, NZTech President Mitchell Pham spoke of the mass digital transformation, which he said began during the first lockdown of 2020.

“It’s hard for most Kiwis to think about what the world was like before the Covid pandemic, but the nation’s digital future is no longer a conversation about what the future may hold, as it happens. produced now, since the arrival of Covid, “said Pham. .

The VR and AR part of this booming tech pie is growing much faster than the whole.

A new report from global technology market research firm IDC, released in October, predicts that the augmented and virtual reality market will grow eightfold over the next five years.

This year, the global virtual reality and augmented reality market is worth $ 6.56 billion. At the start of 2026, it is expected to rise to $ 53.48 billion.

This means that augmented reality and virtual reality are the fastest growing category among emerging smart devices, including high-tech wearable devices and smart home devices.

Much of the growth in virtual and augmented reality has come from the increasingly immersive online gaming world, which Dunedin wants to become the hub of a high-flying New Zealand game development industry. The city is already home to the government-backed New Zealand Center of Digital Excellence (Code). Last week, Code awarded nine game development studios Dunedin with grants totaling $ 1.19 million.

But augmented reality and virtual reality aren’t just about gamers using their phones to capture digital monsters hidden in the real world or wearing goggles to fight in a zombie-infested Wild West.

Otago practitioners show that the application of technology can range from potentially empowering therapeutic experiences to helping a new generation engage in equally empowering educational experiences.

Sam Wave has lived in and around Queenstown for over a decade, since completing studies in Philosophy and Marketing at the University of Otago. He turns to film editing and other sources of creative income while pursuing his passion for music. It’s not easy in a world altered by Covid that can suddenly cancel shows and tours. But it allowed him to produce his second album Oasis Ballerina and create accompanying digital content in partnership with Trey Ratcliff.

Ratcliff had previously pioneered high dynamic range (HDR) photography and started the world’s best travel photography blog, StuckInCustoms, by the time he moved his family from Austin, TX to Queenstown a few years ago. before Wave settles there. His photographic works are owned by collectors, including American actor Leonardo DiCaprio and New Zealand businessman Michael Hill. A recent foray into non-fungible tokens (NFTs) – a digital collision between art and cryptocurrency – earned it a staggering $ 6 million in just a few months.

Over the past year and a half, Ratcliff’s computer science background and personal mission to “help spread awareness and mindfulness to the world” have merged into immersive VR fractal videos and the partnership. Elf Machine with Wave.

Fractals are infinitely complex patterns that appear the same on different scales, such as snowflakes and tree branches. Abstract fractals can be created by having a computer repeatedly calculate an equation, for which it produces a visual representation.

“It’s a fun and somewhat esoteric practice,” says Ratcliff.

The video fractals produced by Ratcliff are then matched with the music of Wave or other musicians; the music then interacts with the images, again changing colors and shapes.

“They are beautifully intertwined.”

Dozens of Machine Elf’s visual and audio art compositions were uploaded to YouTube, where they quickly gained a large and enthusiastic following. They were also picked up by online wellness company Tripp.

The purpose of the pieces is to offer escape and visual mindfulness, say Wave and Ratcliff.

Comments from viewers suggest that the combination of visual and audio elements in an immersive VR environment helps people take a break from their overactive or anxious thoughts.

“We’ve found that if you put people into these visual meditations, these visual fractals, especially with a virtual reality headset, it really relaxes the mind, almost like forced meditation,” Ratcliff says.

Max Mollison is back in Dunedin, adding an augmented layer of whimsy and facilitation to the city that gave him birth.

Raised in Dunedin, he studied fashion in Wellington after graduating from high school with an art focus at Kavanagh College. He then went to Orlando, Florida to work as an intern at Walt Disney World Resort before doing another internship, this time for a fashion brand in Sydney, Australia.

Mollison first came into contact with the Otago Museum in 2017, when he made an item of clothing for a museum exhibit. He then worked for the museum for two years, mainly as a traditional media designer.

He took a leave of absence last year, during which he honed his AR software skills that he had been developing for a few years.

“I really enjoyed using it as an extension of fashion design; an adornment, a decoration, that people could try on and that a lot more people could access … They could try this augmented reality look on them- same. “

Mollison has an active social media presence, showcasing the fashion, fonts, and AR filters he creates. His more than 130 posts on the video social media platform TikTok have earned him 13,700 subscribers and 166,700 likes.

Recently, pop star and trans icon Kim Petras, who attended this year’s MTV Video Music Awards with Paris Hilton, released a TikTok to promote her new single using an AR filter made by Mollison.

Returning to the Otago Museum, he created Plunge, a free attraction to complement the museum’s latest flagship exhibit, Sea Monsters: Prehistoric Ocean Predators, which opened this month.

Visitors, when entering the Plunge Gallery space, scan QR codes allowing them to see an aquatic world populated by digital 3D sea monsters, ancient and modern, with scales and flesh of chrome, glass and water, swimming around the transformed room or circling the heads of other guests. Some QR codes reveal orbs of water or grant the power to generate and launch AR bubbles.

An AR-only exhibit created by a museum’s in-house design team is a first in New Zealand, putting the province and its museum on the crest of the tech wave.

Mollison built the exhibit almost on his own using the open source 3D computer graphics software Blender and the Spark AR Studio software that runs on Facebook and other Meta apps like Instagram.

A technical difficulty was to keep the file size for each of the 10 interactive elements of the exhibits to less than 4 MB.

“Trying to put everything together in a small package is quite difficult given that some of these things are quite elaborate,” says Mollison.

The museum is excited about the AR exhibit and its potential, Oktay says.

Museums around the world are seeing a decreasing number of young people and young people in their twenties coming through their doors.

“What Max is doing really speaks to this generation and brings a whole new reason for them to come and engage with institutions and collections like this.”