Art of the future – the art of together

Ancient art forms rest on a foundation of tradition, and as such they can easily become trapped by their history. Innovative departures are all very well, but it is a challenge to get them to drill down to the foundations and to ensure that they attain lasting value instead of just scratching the surface. The Finnish National Opera and Ballet (FNOB) has taken this issue to task by launching a comprehensive project of exploration called Opera Beyond.

The purpose of this project is to explore the artistic applications of cutting-edge technology across multiple platforms and thus introduce new means for producing meaningful experiences at various levels of an opera production. The potential of technology can act as a catalyst and challenge conventional thinking right down to the core of the artform, from the conception of what an artwork is to the relationship with the audience and to the creative process.

“We’re going on an expedition. We’ve never done anything like this,” says Lilli Paasikivi, Artistic Director of the Finnish National Opera. “We need to be open to the opportunities that this project will bring up. There are a lot of unknowns here, and it’s exciting that we can’t know at the outset what we’re going to find!”

“Now that people have secluded themselves because of the pandemic, experiencing art in digital form has become hugely more important. In this context, the aims of the Opera Beyond project are very timely. Communality and encounters with art have found new forms in this time of isolation,” Paasikivi adds.

“The lack of real social interaction will increase the need for art experiences. After this suspension, when it is safe to seek out collective artistic events once again, the reunion of audiences and performers will be joyous.”

The company of course always keeps a sharp lookout for new trends in performance technologies, but this project entails a deliberate step away from the comfort zone of opera to seek new and concrete approaches. Once performances are reinstated, the potential of artificial intelligence and virtual reality will be tested in an immersive installation named Laila, a collaborative effort by Esa-Pekka Salonen, singer-songwriter and actress Paula Vesala and the Ekho Collective. Salonen is providing the music, Vesala is creating the dramaturgy, and the work is being executed by the Ekho Collective, which won the ideas competition for an interactive and immersive artwork held by the FNOB. Nearly 200 applications from 32 countries were received.

Sharing ideas

Opera Beyond is the result of a snowball effect. In 2016, Esa-Pekka Salonen became Artist in Association of the FNOB. “When we started talking about cooperation with Esa-Pekka, he expressed his interest in new technologies. I took him up on this word, because it was a good idea for a development project for us,” says Lilli Paasikivi.

The contemplating of new technology evolved into a broad-based project, described at the FNOB as an ‘ecosystem’. It includes experimental productions, innovative concrete production tools and collaboration development. 

A conference was planned for May 2020 to bring together performing arts people to discuss new technology. This conference has now unfortunately been cancelled due to COVID-19. The premiere of Laila has also been cancelled, and the FNOB is investigating whether these events could be rescheduled for later. Paasikivi would be pleased for the Opera Beyond project to be able to offer a forum for such a meeting of professionals.

“We want to give space to dialogue, to interaction and to ideas – and I hope it will translate into concrete benefits for operators in this field!” 

The Jane and Aatos Erkko Foundation was on board with Opera Beyond from the start. It was a condition of the Foundation’s funding that the know-how generated in the project would be shared with the entire sector. The approach was international from the first. “All opera houses explore new technologies, but the FNOB project is one of the most extensive. We should be able to learn from each other more,” says Paasikivi and goes on to say that the visual element in opera has been constantly changing over the decades and that production teams have always been interested in new possibilities. Projections and video art have become commonplace in the performing arts in the past 20 years or so, and they have improved hugely in the process. Technological advancements progress rapidly, and this poses a challenge for arts institutions.

“We have an obligation to explore the potential of new technologies, because that ensures that opera as an artform will not become stagnant,” says Paasikivi. “It’s not about replacing something valuable with technological tricks; this can enrich opera and bring it to the present day, making it relevant in a new way.”

Laila is a pilot work for testing new technology, an installation within a dome-like structure that can be exported from the FNOB and function as a calling card for the Opera Beyond project. Paasikivi notes that the next step is to introduce new technologies to productions on the main stage.

Paula Vesala, who is responsible for the dramaturgy of Laila, points out that the marriage of technology and art goes back a long way. “Filmmakers have developed camera equipment, musicians have built new instruments such as synthesisers from scratch, people interested in acoustics and sound reproduction have created new systems, and so on. For me, technology is a tool for exploring communication and how to create meanings and experiences in new ways and in new environments,” she says.

Cooperation creates a virtuous circle

Visual designers, programmers, service designers. Artists from various branches of the arts. Experts in the moving image, audio and robotics. The Ekho Collective is a multidisciplinary cluster of people who share an interest in technology, experientiality and immersion. The working group came together as a result of being inspired by the Opera Beyond ideas competition; its seven members (now eight, Tuomas Norvio having joined later) were excited by the idea of testing their skills in the context of art.

“We were really excited that the National Opera wanted to do something like this,” says Saara Mäkinen, and Minja Axelsson and Iina Taijonlahti agree. Mäkinen and Axelsson are experiential designers; Mäkinen is also a visual designer, and Axelsson is a programmer. Taijonlahti is a dance artist and choreographer.

“We’d all been waiting for a chance to combine art and technology in an unanticipated way, opening up new artistic potential,” Minja Axelsson explains.

The working group describes Laila as “an interactive, immersive work that invites the audience to build a world of its own. The musical and visual world of the work is shaped in interaction between audience members and the artificial intelligence.”

The work is a laboratory that explores not only the potential of technology but also the potential of multidisciplinary cooperation. The Ekho Collective brings together multiple perspectives that joined forces with the visions of Esa-Pekka Salonen and Paula Vesala.

“It’s the best when everyone is prepared to look at ideas and opportunities from a different point of view. That can create a virtuous circle of development,” says Saara Mäkinen with enthusiasm.

Minja Axelsson adds that the discussions went really deep. “It’s so interesting when everyone has a slightly different take on technology and on art and on the relationship between the two.”

Prototypes and testing

The trio emphasises that Laila was built around the experience, not around the technology. In creating the world of the work, they first decided what kind of an experience they wanted to create and then investigated what technological solutions would be needed to achieve it.

“What’s important is that you don’t just have hypotheses but also the tools to test your scenarios in practice. As with any artform!” says Iina Taijonlahti.

At the time of this interview, in February, the Ekho Collective was about to begin building the actual hemispherical installation in a physical space.

“When technology plays such a huge role, the process is rather special: it’s iterative, testing,” says Saara Mäkinen. “When you divide things up into smaller components and share them in various forms, everyone knows where we are and we can validate which designs work. It’s prototyping.”

The prototypes for this work began with Lego figures and mixing bowls and ended up as virtual simulations designed for exploring various kinds of experiences. At the last stage, the completed work will be tested on actual people.

“What is interesting is that in this work technology is the object of study and even of criticism,” says Minja Axelsson. “Our hope is that the immersive nature of the work will make the technology more tangible and help to deconstruct the mythology that the media have spun around artificial intelligence.”

An essential part of this is that in Laila – as in immersive and interactive art in general – the objective is to make audience members active participants. Taijonlahti explains how traditional art works are put on display for visitors to look at, while immersion means that visitors can, indeed, immerse themselves in the work and be active participants. In Laila, the presence and movements of the visitor have an impact on the world of the work.

“Immersion makes the work more porous. It is less ambiguous then, and it is different for everyone,” says Taijonlahti.

She says that the key artistic question in Laila has to do with agency – both that of a human and that of technology. “What kind of agency does technology have, how can we use agency as material for an artwork, what is our relationship to technology?” Taijonlahti says. “Technology and human reflect off each other. Technology can be used to take humanity forward, but technology cannot exist without human agency.”

She notes that the word ‘technology’ comes from the Greek word tekhne, meaning ‘skill’. It can also be interpreted as potential. “Not just as a means to an end, but as a tool for humans to learn something new about their environment and realise their own potential.”

Positive questioning

Lilli Paasikivi is happy that Opera Beyond has opened up channels for a new kind of dialogue between branches of culture and to the corporate world. The partnership of Opera Beyond with the Experiments in Arts and Technology department at Nokia Bell Labs was announced in February.

From the perspective of the FNOB, this involves questioning its own operations in a positive way: could we do things in other ways than those that we are used to? “We have to ask ourselves what opera will be like in a few decades. We can stand with one foot in tradition and the other in the present day, looking towards the future and keeping our ears open to what’s happening in the world,” says Paasikivi. The expectation is that Opera Beyond will yield new ideas for both artistic content and operations more generally. Paasikivi also hopes for courage in the world of opera to seize the potential of technology. “It’s entirely possible that not everything will work, but we have to be able to try things out. I don’t want us to consider opera to be overly sacred and untouchable.

Opera Beyond examines and carries forward the marriage of art and technology in the history of human kind. As Paula Vesala says:

“Technology is sometimes seen as inhuman and even as frightening. Actually, it has helped us communicate in many new ways, to extend our life spans and to operate globally. In the face of climate change, pandemics and all other great challenges facing humanity and the Earth, the role of art is clear: it explores humanity and interprets the meaning and character of life – something that no code or clinical analysis can ever fathom.”

Translation: Jaakko Mäntyjärvi
Featured photo c) Ekho Collective 

::

The second edition of the Opera Beyond Jam Sessions takes place on Tuesday, May 18th, 2021 at 15:00-17:00 CET on Zoom. Opera Beyond Jam Sessions are a series of interactive online events, where the participants can exchange thoughts and share best practices at the crossroads of performing arts and new digital technology. The curated program includes inspirational master classes from visiting speakers, concrete case studies from various organizations and open discussion between all participants. Participation is free of charge. Read more and sign up here.

Virtual technology is changing the performing arts

Saara Mäkinen, Minja Axelsson and Iina Taijonlahti of the Ekho Collective consider it important that the creators of an immersive artwork dare to yield power to the audience and to technology.

“That’s what immersion is also about – the visitor having autonomy in relation to the world of the artwork,” explains Iina Taijonlahti.

Laila, a work by the Ekho Collective that is to be premiered at the Finnish National Opera and Ballet in June 2020, is meant to be open to multiple ways of experiencing. Some may even feel it to be slightly intimidating. The aim is for every visitor to encounter the work on their own terms, whether actively or passively.

So what technology does Laila use? At its core is artificial intelligence, with which visitors interact in various ways. Motion tracking technology is used to facilitate this: every visitor has an impact on the sound and visual environment through their presence. This means that every visitor will have a unique experience. The motion tracking is implemented using LiDAR technology, similar to radar. No sensors need to be installed on visitors.

The working group considered it important that visitors should not have to use devices such as VR goggles, which would isolate them from each other. Instead, visitors are given wearable speakers through which they hear their individual soundscape. “We want the immersive space to be a shared experience, so that the reactions of individuals become part of the artwork,” says Iina Taijonlahti.

In addition to motion tracking, Laila makes use of a multi-channel 3D sound system and projection mapping. This creates a world with texture and dimension, with the sound surrounding the visitor and projections seamlessly filling the hemisphere all around, creating walls and a ceiling.

Simulation helps with design

An immersive performance is one that engages multiple senses and enables persons experiencing the artwork to immerse themselves in it. Timo Tuovila, Production and Technical Director at the FNOB, says that people typically think that immersion is the same as virtual technology. This, however, is too restrictive. “Immersion can be implemented in various ways, and virtual technology is only one of these,” he says. Virtual technologies include Extended Reality (XR) and Mixed Reality (MR); these are further sub-divided into Virtual Reality (VR), Augmented Reality (AR) and Augmented Virtuality (AV).

“You could say that opera has always been an immersive experience. The visual dimension has been important ever since opera was born, and opera productions have always made use of the latest technology available,” says Tuovila. Projections were introduced to opera performances as far back as the 1950s, using film projectors. In recent years, projection technology has progressed with immense strides. “Video projectors and 3D animation now allow practically anything to be visualised,” says Tuovila.

Projection mapping technology, which creates a seamless image across a surface, was first tried out at the FNOB in Duke Bluebeard’s Castle in 2013. In Rheingold, premiered last autumn, featured impressive projections on LED screens. Tuovila also notes that pioneering tracking technology was used in Rheingold to have spotlights track singers automatically using sensors affixed to their costumes.

“We’ve used new audio technology for amplification in musicals so that the sound system follows the location of the performer, adjusting the panning so that the sound seems to be coming from where the singer is.”

The challenge for VR technology at the moment is that VR goggles rob live performances of much of their community feeling. However, even here there have been interesting new departures in the cultural sphere. Then She fell by Third Rail Projects, running in New York, combines VR, performing arts and escape-room experiences.

Virtual reality and XR technology have interesting potential for production processes too. Testing things virtually would reduce the need for travel and construction, and the design process would thus be more ecological. Developing a simulation tool for this purpose forms part of the FNOB’s Opera Beyond project.

“At the moment, we do not see the visual end result until about two weeks before the premiere, even though the production was modelled on a computer one year earlier. Now we want to see whether we could achieve this in virtual reality with a feasible amount of work,” says Tuovila.