The Entanglement – AI, Platform Musicking and the Future of Music

Jennifer Walshe speaking at the Musica Ex Machina Symposium in Lausanne, Switzerland (Photo: EPFL/Alain Herzog)

The Entanglement – AI, Platform Musicking and the Future of Music

Given the vast amount of user data that streaming platforms have now amassed, combined with advancing AI technology, it is only a matter of time until they offer the ability to generate music, writes Jennifer Walshe. But what does this mean for the future of the art form?

‘Today, with the cost of creating content being close to zero, people can share an incredible amount of content.’
Daniel Ek, CEO of Spotify

‘The future is there… looking back at us. Trying to make sense of the fiction we will have become.’
William Gibson, Pattern Recognition

To speculate on the future of music is to follow in the footsteps of the many musicians who have dreamed of many different musical futures – Sun Ra’s flight to the stars, Tony Conrad’s desire to ‘destroy’ western musical composition, Pauline Oliveros’ complete dissolution of sonic boundaries. It is an artistic project, a sometime duty, and often, an inevitable folly. Nonetheless, as someone interested in the many different forms the future takes, and, to paraphrase William Gibson, how evenly or unevenly those futures might be distributed, I’ve spent a lot of time over the last few years trying to think through how the wide-scale adoption of AI in music will affect the everyday listener. I’d like to trace a thread here across three positions – the platform, the user and the creator. These positions are not discrete – they blur into and affect one another deeply. And though I’ll briefly touch on each position, I want to emphasise that this blurriness between the roles of platform, user and creator is key to how we will listen to, make and think about music in the future.

As much as we may regard music as an expression of the soul’s ineffable longing that transcends language – and let me be clear, it is – it’s crucial that we acknowledge that it is an art form that functions within and is deeply shaped by the impact of the technologies used to create it. Contemporary headlines might scream that AI will change music forever, but music has always been and continually is in the process of being changed by the introduction of different technologies. Western classical music was overwhelmingly affected and defined by the introduction of equal temperament, jazz by the saxophone, and pop by the drum machine and DAW (Digital Audio Workstation), for example. 

All future music will be affected by the affordances of different AI systems, whether or not the musicians use AI to write it. This begins with the streaming platforms, which, for the majority of listeners, are the primary point of contact with recorded music. AI works across these platforms in a multitude of ways. Firstly, it facilitates the music information retrieval analysis necessary for the platforms to understand what they are hosting. Secondly, it is used to monitor streaming habits, root out bad faith streamers, and apportion royalties in the deeply idiosyncratic fashion these platforms have established. Most significantly, though, AI powers personalisation – the recommendation engines designed to keep the listeners listening for as long as possible, and is crucial in the analysis of the huge amount of data created by each user. 

The scale of data
This data is, for the most part, proprietary and inaccessible to the public, but publications by current and former employees of streaming platforms show both the mind-boggling scale of the data being gathered, as well as the fascinating insights the platforms can glean from it. Analyses by Spotify researchers, for example, detail how the scrubbing behaviour of users is the fastest way to determine where the drop in a song is located; that only roughly half of songs are listened to all the way through; how analysis of over 3 billion ‘skip events’ demonstrates that listeners tend to skip whenever there is a change in a song’s structure.

While these publications give us some idea of the data that has historically been gathered by Spotify (and, presumably, its competitors), the recent patents the company has lodged give us some idea of the data it may look to capture and analyse in the future. A Spotify patent granted in 2021 titled ‘Identification of Taste Attributes from an Audio Signal’ details how voice requests made by a user to Spotify’s AI would be handled. Separately from parsing the textual content of the request, an incredible amount of data is extracted from the user’s voice – emotional state, gender, age and accent, as well as information about the environment and social situation the request was made in. I would argue that a whole host of other, deeply personal data not mentioned here could also potentially be extracted – given current state-of-the-art AI voice recognition, inferences about health and intoxication levels could also be made. I find myself asking – is it really necessary for a platform to know this much about a user in order to recommend a song? Under the logic of what Nick Srnicek calls ‘platform capitalism’, the answer is yes. The collection and analysis of user data is the primary purpose of any platform. The streaming platforms do not own the music that they facilitate access to – they are intermediaries, and their survival depends on their ability to harvest data from their listeners, and sell the fruits of the analysis of that data back to their listeners in the form of recommendation engines. 

An image from Spotify’s 2021 patent, ‘Identification of Taste Attributes from an Audio Signal’.

Enmeshed with the platform
This is where the blurriness begins. Music hosted on streaming platforms, in the most salient way, ceases being a work of art, or even entertainment – it functions as the mechanism through which incredibly granular and personal information about the user is retrieved and analysed. In turn, the user’s experience of the platform is itself completely unique. As Gustav Söderström, Spotify’s Chief R&D Officer stated, ‘There isn’t just one Spotify experience. There are actually more like 345 million different Spotify experiences – one for each listener.’ Keep in mind he was speaking in 2021, when the user base was 345 million. It is currently 626 million.
 

Through the user’s listening behaviour and interactions with the recommendation engines, the user becomes enmeshed with the platform on a scale that is shockingly intimate. The user becomes part of the platform. They understand that their every action will be interpreted and analysed within the confines of platform logic. I think of one friend who very precisely tuned his YouTube algorithm for Italo-disco, and who delighted deeply in the obscure cuts he was served; I think of another friend whose daughter purchased Taylor Swift’s most recent album on vinyl, and whenever she played it, would simultaneously stream the same album on Spotify with the volume set to zero, so that the algorithm would have that data for her year-end Spotify Unwrapped. The platforms function because of the users who live alongside these recommendation engines, training and tuning them, giving up their data freely.  

I would argue that in the future, the vast amount of user data held by the platforms, coupled with AI, will result in the generation of huge amounts of music. The user won’t just be served playlists or pre-existing music. Platforms will generate highly individualised, unique tracks and genres for the user, bespoke musical filter bubbles beguiling enough to keep those earpods in place. And so the user is ultimately also a creator. Their listening data, curiosity (or lack thereof) about unexplored regions of genre-latent space, their desire to listen to music that makes them feel like a main character, or have their best friend’s name integrated into a club banger, will fuel the generation of music. 

The streaming platforms don’t offer the ability to generate music yet, but given that these platforms are saturated with AI, it seems only a matter of time. If we turn our attention to the current generative music platforms, it is clear that there is no longer any meaningful distinction between user and creator. It has been dissolved by AI. The company Suno’s website states that their mission is to ‘break barriers between you and the song you dream of making. No instrument needed, just imagination.’ The website Udio notes how AI ‘has the potential to… enable anyone to create extraordinary music… anyone with a tune, some lyrics, or a funny idea can now express themselves in music.’ Boomy notes that its service allows people to ‘create original songs in seconds, even if you’ve never made music before… and get paid when people listen.’ Being a musician is as easy as typing into a context window. 

Glimpses of this future can in fact already be seen on the generative AI platforms that supply bespoke functional music. Functional music is music that is intended to soundtrack and/or optimise activities such as work, meditation and exercise. White noise, whale song, lofi beats, hushed covers of pop songs, ambient piano, to each their own. While functional music can clearly be placed in a historical context linking it with muzak and relaxation cassettes, its current prevalence is inextricably linked to streaming platforms – Glenn McDonald, former ‘Data Alchemist’ at Spotify, notes how streaming ‘more or less created a consumer market for almost-generic music as background noise’ (emphasis McDonald’s). Over the last decade, a host of platforms have emerged that use AI, neuroscience and sleep psychology to create and serve their users functional music. Users of platforms such as Endel and Brain.fm hand over incredibly granular personal information in exchange for individualised functional music that promises to help them optimise their work, leisure and sleep. Endel, for example, uses ‘bedtime, natural light levels, weather, and time of day… head position… heart rate and motion data’ to serve the user personalised soundscapes. Brain.fm’s music ‘contains patterns that shift your brain state with entrainment’, claiming that ‘our music sounds different – and affects you differently – than any other music.’ On these platforms, the user enters willingly and enthusiastically into a deeply entangled state with the platform, in return for optimisation. Sound exists beyond the artist, album or playlist; sound is instead precisely designed, ‘scientifically proven to increase focus’, audited first and foremost for mood management, for assistance, for comfort. Given that functional music is already extremely popular on streaming platforms, it seems to me that this is where AI-generated music might most profitably first emerge for these platforms.  

Platform Musicking
The musicologist Christopher Small defined ‘musicking’ as the present participle of the verb ‘to music’. To music is ‘to take part, in any capacity, in a musical performance, whether by performing, by listening, by rehearsing or practicing, by providing material for performance (what is called composing), or by dancing.’ Small’s definition is inclusive at core, allowing us to think about music as an activity that involves a multitude of participants, all of whose contributions are valued, rather than the single exalted vision of the maestro-composer. I must admit that I’m surprised that he’s not being quoted by the CEOs of the companies I just mentioned.
 

I would argue that the entanglement between platforms, users and creators will be the most prevalent form of musicking in the age of AI. Building on Small’s definition, I call this ‘Platform Musicking’. Platform Musicking will, of course, occur in parallel to the huge range of AI-facilitated musical outputs and experiments that will happen in commercial and institutional spaces. But it will be this shift to Platform Musicking that will affect most listeners on an everyday basis. Music will cease to be an independent artistic entity that is simply hosted on a platform – it becomes an emergent phenomenon, produced through the user’s engagement with the platform. Music on demand, music as an artform which requires only the user’s data or imagination or funny idea, music as an endless, highly personalised flow, fleeting and non-linear. A phenomenon underwritten by AI at every level, and monetised all the way down, where the user owns neither the music they listen to nor the huge data traces they create. Music untethered from artists, from album structure, from traditional genres and instruments; music untethered from collective experience, fractured instead into millions or billions of unique user experiences, entangled inextricably with the platforms. 

I have no doubt that people will continue to make music, and that for many, the dedication and time spent acquiring skills will always be part of the challenge that makes being a musician exciting and worthwhile. I’m sure many musicians will integrate AI into their work, and while many will make AI the conceptual focus of what they’re doing initially, gradually the shine will wear off and using AI to isolate vocals, mix and master, and generate ideas for beats, will become no more notable than using Ableton Live or Dorico. It also seems clear that streaming platforms will continue to bring joy to many people, and be key to the success of many musicians. What I am curious about, though, is how music will be affected by Platform Musicking, by the restructuring of music into a bespoke digital flow; by the impact that ever-increasing amounts of data and ever-increasing personalisation will have; and how music and musicians will be forced to advocate for themselves as we move through a wholescale reconceptualisation of what music is, how it is made, and why.

This is an expanded version of a talk originally given at the EPFL Pavilions Musica Ex Machina Symposium in Lausanne, Switzerland, on 20 September 2024. Visit https://epfl-pavilions.ch/en

Jennifer Walshe’s latest recording, URSONATE​%​24an AI-generated performance of Kurt Schwitters’ Dadaist work, is available from Bandcamp. Visit https://jenniferwalshe.bandcamp.com.

Subscribe to our newsletter.

Published on 24 October 2024

Jennifer Walshe is a composer and performer and Professor of Composition at the University of Oxford.

comments powered by Disqus