Music in the metaverse: How digital avatars could be the future of touring

18 Apr 2022

The avatars of ABBA’s Agnetha, Björn, Benny, and Anni-Frid seen on the side of the Sydney Harbour Bridge. Image: Will Hartl

From a Tupac hologram to ABBA avatars, Theo Tzanidis and Dr Stephen Langston look at how technology could provide new experiences for music fans and disrupt an industry.

Click here to visit The Conversation.

A version of this article was originally published by The Conversation (CC BY-ND 4.0)

It was a technological feat that made history, wowed audiences and brought a dead rapper back to life. In April 2012 at the Coachella festival in California, Tupac Shakur took to the stage with Snoop Dogg and Dr Dre. He’d been dead for 16 years, killed in a drive-by shooting in Las Vegas.

But this was Tupac the hologram, foul-mouthed and lifelike, performing before a “shocked, then amazed” crowd.

Since humans first delighted in the sound of music, advancements in technology have managed to make musical expression immortal. Throughout history, innovators have strived to create original, accessible and eternal performances.

As engineering knowledge developed, musical instrument design advanced. Many classical composers introduced pioneering instrumentations into their scores, adding depth and colour that broadened the listening experience.

Accurate systems for notation matured, offering music an essence of immortality through printed manuscript. In 1853, Édouard-Léon Scott de Martinville’s phonautograph pioneered an audio recording technique.

In 1912, WC Handy composed Memphis Blues, a song that took the US by storm and influenced the development of popular music. Published on paper, it was wildly popular in the dance halls and soon every band in America was asked to play it. This public demand was recognised by a fledgling recording industry, which soon flourished.

Technology and creativity

A breakthrough in the quality of music capture came with the advent of hi-fi and stereo. Those who embraced the technology artistically could transport a lifelike performance experience into the homes of the masses.

One of the biggest bands of the 1970s and 1980s, Swedish supergroup ABBA, embraced pioneering recording techniques. Behind this technology was the creative genius that produced millions of record sales and performances dominating the 1970s and beyond.

After the apparent demise of the group, Benny Andersson and Björn Ulvaeus expanded into the theatrical genre, composing musicals. Along with their interest in emerging technology, this sowed the seeds to recapture and reinvent the ABBA machine 40 years later.

May 2022 sees the latest technological advances in musical immortality when ABBA return to the live stage after a 40-year absence. But this time they return as humanoids – the digital hologram ‘twins’ of the original global phenomenon.

George Lucas’s Industrial Light and Magic has created holographic lookalikes that interact with a live band in a specially designed purpose-built theatre in east London. Benny, Björn, Anni-Frid Lyngstad and Agnetha Fältskog have provided the pre-recorded vocals and motion-captured movement, which will then be reproduced by the digital avatars.

The doppelgängers are more youthful in their appearance – around their 30s, when the band were at the peak of their fame – raising an interesting conundrum concerning ABBA’s human mortality against their new immortality in the metaverse.

ABBA’s music is undoubtedly timeless; the simple tunes with incredibly complicated structures appeal to millions. The ‘Abbatars’ are a reinvention for a new audience, but will they continue beyond the lives of their originals with new creators pulling the strings?

Besides ABBA and Tupac, there are other instances where digital twinning has been identified as a key money-making strategy. The digital band Gorillaz’ 2006 Grammy performance blended flawlessly with Madonna’s. And Richard Burton’s hologram performed on a global tour of War of the Worlds in another 2006 performance.

Music in the metaverse

Customising 3D avatars has become a unique way for artists to create virtual brands across several digital platforms. They can connect virtually with fans and increase loyalty and engagement, while fans can interact, express themselves and experience new things.

This is now achievable using AI software to make holograms, as researchers at MIT demonstrated in an experiment that created holograms fairly instantaneously.

Ziva Dynamics, a pioneer in simulation and real-time character creation, employs synthetic AI-powered avatars to create autonomous and complex movement simulations based on real muscle, fat, soft tissue and skin contact.

In 2021, in a project called Lost Tapes Of The 27 Club, Google’s Magenta AI was even used to compose songs in the styles of musicians who died at the age of 27, including Jimi Hendrix, Jim Morrison and Amy Winehouse.

These technologies have the potential to create realistic synthetic and AI holographic representations of departed artists, allowing them to continue creating, influencing and performing for future audiences.

Epic Games, creator of the phenomenally successful Fortnite, predicts that digital twins will combine with the metaverse, an emerging network of fully immersive digital worlds.

Disrupting the music business

Whereas live tours are time intensive and costly for new artists, a low-cost metaverse ‘tour’ might be a new way for music lovers to see live performances. Virtual performances by Justin Bieber, DeadMau5 and The Weeknd have already become popular recently.

In this emerging branch of the music industry, record labels and marketing firms could be replaced by decentralised autonomous organisations. DAOs are online organisations that operate like cooperatives, making all decisions jointly.

DAOs are already disrupting the music business – along with NFTs, which are a way of transferring property between people online. In October 2021, PleasrDAO – a collective of decentralised finance leaders, early NFT collectors and digital artists – paid $4m for Once Upon a Time in Shaolin, a one-of-a-kind album by New York hip-hop legends Wu-Tang Clan.

While the release of the album predates the rise of NFTs, PleasrDAO now owns the rights and has imposed strict restrictions on duplication, distribution or public exhibition.

A music-focused DAO like Pleasr may acquire bulk concert tickets, finance and organise events and manage fan-owned record labels and marketing agencies to secure investable commodities like first-edition LPs, artwork and instruments. This has the potential to benefit fans, new music genres and artists alike.

This creates a new, decentralised route to the market for artists free of corporate interests or interests of individual producers, developing a fairer landscape for the future. With digital avatars likely to be at the centre of this new vanguard, it will be fascinating to see how it develops in the months and years to come – and whether it will be enough for music audiences.

The Conversation
By Theo Tzanidis and Dr Stephen Langston

Theo Tzanidis is a senior lecturer in digital marketing at the University of the West of Scotland. Dr Stephen Langston is programme leader for performance at the university.

10 things you need to know direct to your inbox every weekday. Sign up for the Daily Brief, Silicon Republic’s digest of essential sci-tech news.