The New Real
Drew Hemment is Professor of Data Arts and Society, Director of The New Real, and Director of Festival Futures at Edinburgh Futures Institute and Edinburgh College of Art within the University of Edinburgh. At The Alan Turing Institute, Drew is Theme Lead for Arts, Humanities and Social Sciences. Over 30 years, Drew has been one of the key figures who has shaped the field of digital art and culture in Europe. He has worked with cities and nations representing culture and research at the highest level, and worked for the Singapore Government on Smart Nation and Singapore’s 50th anniversary. In 1995, Drew founded FutureEverything, named by The Guardian one of the top 10 ideas festivals in the world. He is presently working to develop a transformative research agenda and build a national capability on AI and the Arts in partnership with The Alan Turing Institute and Edinburgh’s Festivals. His work has been recognised by 14 international awards including Soil Award 2019 (Winner), STARTS Prize 2018 (Honorary Mention), Lever Prize 2010 (Winner), and Prix Ars Electronica 2008 (Honorary Mention). He is a Turing Fellow and a Fellow of the Royal Society of Arts.
I was drawn to work in Edinburgh because Edinburgh, as we know, is celebrated as the world’s foremost festival city. But not everyone in the festivals community will know that it’s also the home of Artificial Intelligence (AI) in Europe. It’s been a pioneer in AI since the 1960s, when the first lab in Europe was founded by Donald Michie. He had been inspired to ask could a machine think during all night chess sessions with Alan Turing in the code-breaking group at Bletchley Park in the 1940s.
In simple terms, AI is technology that completes tasks that we normally associate with humans – so decisions, predictions and, to a degree, creativity. AI has been around since the 1960s, but over the last ten years we’ve had really major advances in an area called machine learning which uses advanced algorithms to detect patterns in vast troves of data. By finding patterns it is able to predict new instances, and this enables AI to generate new outcomes based on training from previous data. This has given rise to the powerful ‘generative AI’ tools we see today, like ChatGPT and Midjourney, which are having a huge impact in the arts and in society at large.
I founded The New Real as a joint initiative between Edinburgh’s festivals, the University of Edinburgh and The Alan Turing Institute, to explore the potential of AI for the arts. It started with the realisation that there’s a significant artistic community who has been working with AI and machine learning for more than a decade. I felt this was an important artistic movement that we wanted to celebrate and support. There are a couple of things that are significant about that artistic movement. Firstly, that it predates the current AI boom. A lot of people talk as if AI appeared from nowhere and has only been with us for the last year, but actually there’s a long tradition of artists working with AI that extends way back to its origins in the 1960s. Then curatorially what interests me is the way artists are working critically with AI. They are not just using AI as a new shiny tool. AI algorithms and data are the material with which the artists produce their work. And they are also asking questions of AI and its social consequences. They work with AI as both tool and topic – that’s the key thing. It is that dual aspect of using AI as a creative tool and a creative medium but also using artistic practice to explore, to question and to challenge AI.
AI is impacting the visual arts, performance, cinema, games, and it is being used in many different disciplines. Many of the people I work with are visual or new media artists. There is a deep tradition within music too of working with AI and algorithmic processes, and right now the games industry is really being shaken up by AI. If we think about AI in society more broadly, AI is often in the background. It’s AI that generates the recommendations that shape what we listen to on Spotify or watch on Netflix. It’s not something we are necessarily aware of, but it shapes our experience.
There are serious challenges and problems around AI. AI can amplify existing harmful bias and injustice in society. It is also energy intensive and the business models surrounding AI mean that a lot of the value goes to a very small number of companies. The current generative AI is based on harvesting data that’s available on the public Internet without crediting a lot of the original creators and certainly without them getting paid for their original work. The kinds of jobs that will go and the ones that will come may entrench or increase existing inequalities between people who are quite privileged and have access to technology and those who don’t. A major problem is that AI is ‘black boxed, it doesn’t reveal information about its internal workings, and it’s very hard for a human to understand how one recommendation has been made over another. And the technology and industry are so diffuse it can be hard to hold people and companies to account.
Within The New Real, we’ve developed our own research that looks at how we can drive and fuel artistic experiences, but also make AI more accessible, transparent, and easy to comprehend. I’m myself hugely excited by the astonishing art we see today being produced with AI, and most of all I’m excited by the way artists make these data systems, the algorithms, and the data itself tangible so that we can have direct experiences of it. Artists enable us to explore them in new ways and help us to build understanding of those systems, which is essential because then we can hold those systems to account – and if they’re accountable, then we’re more likely to have fair and responsible technologies.
At the same time, however, the current generation of tools offer limited creative agency or control. It’s true we are seeing a democratisation of these technologies. Some of the tools are free to use, some of them have a small charge, but they’re very accessible. But at the same time, we have less control, and less agency. We can’t see how the tool is working, we can’t easily take it apart, or modify it. Often the only input that we have is the text prompt. We can type some text in and generate an output. But we can’t see why or how our input led to one particular output and not another.
This is a challenge we are tackling head-on in The New Real, we are looking for ways to make AI more transparent, and to give artists greater creative control. Many of the artists that we work with have some software skills, but we are trying to give people more control without needing those software skills. We collaborate with artists and engineers to develop new tools and to create artworks that put those tools through their paces. Our idea is to give artists the means to directly train and probe a model so they can build understanding about how the AI sees the world. This can then help them to generate more profound artworks and tackle some of the social challenges surrounding these technologies. The current work we are doing is on multi-modality – looking at how we can work in text, images, touch and sound. We’re looking at how we can fine tune a model that works across modalities. We might look at a model with images to generate sound. The artists work and create interfaces to this data in any medium. It can be a screen, it can be sound, it can be text.
The first programme we did was in 2019 and 2020, which led to the Preternatural exhibition at the Fringe in 2019, and then the eponymously named show The New Real at Edinburgh International Festival in 2020. We commissioned artworks that have since gone on to tour internationally, and one work from 2020, The Zizi Show by Jake Elwes, is presently the subject of two major solo exhibitions by Elwes, at V&A and Gazelli Art House in London. This work looks at the intersection of machine learning and drag performance. It’s a visually stunning work that is incredibly powerful and instantly gives you a sense of the sublime. And it’s also highlighting one of the main problems in AI – the way it can reflect and amplify harmful bias. Jake wanted to address the lack of representation of marginalised communities in society and in data. So, Jake staged a project where he worked with a community of drag performers, giving those artists paid work during lockdown, and capturing data that he used to retrain the AI models, so non-binary identities, and different sexualities, could be represented. It’s a very strong artwork that also has this political dimension.
This work originating in Edinburgh has had influence across different artforms internationally. We have been able to build international partnerships through our curatorial and research programme, and we have worked collaboratively with European festivals and partners. One example has been our work with Ars Electronica, who have been long standing collaborators of mine, ever since I founded FutureEverything back in 1995. In 2020, they were launching their own programme called the AI Lab, which was a partnership with some major science institutions around Europe, such as Fraunhofer and CERN. Ars Electronica adopted our research theme as the theme of the inaugural AI Lab programme. Through that programme we launched an artist residency in partnership with them, and that led to two very significant artists, Anna Ridler and Caroline Sinders, coming to work in Edinburgh and developing work that was then presented both within the Edinburgh festivals and also at Ars Electronica.
Edinburgh is the ideal home for this work. One of the very powerful things about Edinburgh is there is an ecosystem of festivals: the international festival, the arts festival, the book festival, the science festival, the Fringe, etc. We have been able to look at AI as a scientific theme and bring artists to shed new light on it as a domain through the Science Festival. And we’ve also worked with the other festivals to explore how AI artists can bring change and challenge ideas.
The New Real programme ties partnerships between the artists who have been at the forefront in addressing these challenges and Edinburgh’s festivals – and this really cuts to something fundamental about festivals as places where society reinvents itself; places where we ask the profound questions and where we get to negotiate and navigate the challenges that we face. The work that we have started is now at the forefront and shaping the debate – the arts and the sciences really learning from each other.
Festival Life creates shared moments of audiences and artists, eye-to-eye
