AI opera Chasing Waterfalls poses existential questions about human identity and the influence of the digital world
- Hong Kong composer Angus Lee collaborated with artificial intelligence for the production, which is part of the 2022 New Vision Arts Festival
- The AI was not only taught to create musical and written contributions for the opera, it was also trained to sing its own original music in a featured scene
How many “digital twins” do you have?
The term refers to one’s virtual existence in the digital realm, particularly on social media platforms for work, romance, friendship and other aspects of everyday life. You may have multiple versions of digital twins, with each reflecting a different aspect of your personality.
This is the case in the world of Chasing Waterfalls, an opera that was developed with artificial intelligence (AI) involved as a co-creator. In both its form and content, this technology-driven musical production seeks to explore what the future holds for humanity as our lives and identities become progressively digitalised.
The upcoming New Vision Arts Festival will present the Asian premiere of Chasing Waterfalls, developed by German multimedia artist collective Phase7 Performing.arts and featuring the work of Hong Kong composer Angus Lee and German writer Christiane Neudecker. Lee’s music and Neudecker’s libretto are both complemented by material created through AI technology, including the performance of an AI singing voice developed by Berlin-based sound studio and Chasing Waterfalls co-composer Kling Klang Klong.
The opera is about how the identities of the main character, Ich – German for “I” – and her five digital twins develop in the online world within the flow of AI algorithms. When confronted by her virtual alter egos, Ich begins to understand and also question who she truly is.
The production explores how our identities may evolve when the boundaries between the real and virtual worlds become increasingly blurred, and how digital operations are influencing the decisions we make.
AI as collaborator and performer
Lee, who also serves as the conductor on Chasing Waterfalls, describes the opera’s musical style as “eclectic”, noting that “tonal, atonal, hip-hop, electronic, vaporwave and traditional opera-sounding music can all be heard”.
And while AI-generated material – both music and text – has been woven throughout the seven-scene opera by its artists, the fifth scene is entirely the work of the AI, which writes and performs its own piece that can differ with each performance.
Lee recalls hearing that the most difficult part of the process was training the AI to sing. Norwegian soprano Eir Inderhaug, who plays Ich, spent a week in a Berlin recording studio singing vowels, syllables and consonants in varying lengths, pitches and volumes, all of which were loaded into the AI’s neural network to help it develop a singing voice. That training, led by Kling Klang Klong with technical support from German IT firm T Systems MMS, took months.
The creative team also had to teach the AI how to compose music so that it would be able to create its solo scene. To ensure that the AI’s performance would fit into the overall architecture of the opera, certain parameters such as the musical key and entry points were set in advance.
Lee explains: “AI composition is all about setting rules. For me, when I compose, that’s the most important composition process, too.”
He notes that without rules, one cannot make good choices for developing a musical piece logically. “This happens to be exactly the same as what we did to the AI. I think this is a very nice parallel between human creativity and AI creativity.”
But when it came to setting the libretto – which Neudecker wrote in German with contributions from the AI – to music, Lee saw first-hand how digital boundaries are being blurred. “I speak German, and if I wasn’t told [the difference], I couldn’t tell which parts were co-written by Christiane and AI, and which part was purely AI,” he says.
“Perhaps this is where the real and virtual worlds merge; the divide is certainly not clear,” he adds. “One of my biggest concerns is whether there will be a point when AI-generated art, which is entirely derived from human artistry, completely takes over what is created by humans. This is why I rendered the [opera’s] ending with a dystopian feel.”
Value of arts in the virtual world
Lee is particularly apprehensive about how closely AI-generated music could sound like music composed by humans. He believes that most audience members at Chasing Waterfalls will not notice on their own that the fifth scene is entirely created by AI.
“AI-generated music is still rudimentary … but if pushed hard enough, the technology will reach the level of humans. This is worrying,” Lee says.
The composer is also concerned about how AI’s ability to create new pieces of music in very little time will fuel what he calls a “fast-food culture” in which constant consumption is put ahead of artistic expression. He has been left wondering how music and other art forms will survive in the future.
According to Yat Siu, chairman and co-founder of Hong Kong-based digital entertainment company Animoca Brands, the virtual world is not going to spell doom for the arts. While he agrees with a key point made in Chasing Waterfalls – that our real-life existence is becoming increasingly intertwined with our online selves – he says aspects of the digital landscape like the metaverse and Web3 technologies could actually prove to be a game-changer for artists.
“I think Web3 is the answer for all industries that are considered more quasi-niche – I would consider classical music as one of them,” says Siu, who is a classically trained musician himself.
On the blockchain, for example, music can be accessed directly from its creators, rather than through an intermediary like a streaming service or social media platform.
Siu explains that relationships can be forged when individuals purchase non-fungible tokens (NFTs) of musicians’ compositions or performances, which helps connect them with other fans as well as the artists. This results in an interactive online community forming around a shared musical interest.
“You can’t build this on some of the largest streaming platforms – they’re not designed that way,” Siu says. “So this will open up more possibilities. My general view is that Web3 will reintroduce diversity in content and in culture more broadly.”
Lee remains hopeful that however advanced AI-based creators may become in music and the arts, people will always recognise the value and worth of human creations.
“AI art might be a genre in itself, but human-generated art still stands a chance against it,” he says. “Art created by humans will be able to find its own place in the whole wide spectrum of art.”