The Semantic Machine
A New Listening Experience
THE SEMANTIC MACHINE APP
The Semantic Machine is a mobile app that offers a contextual listening experience based on semantic web technologies. The app adapts the music it plays to the listener’s environment, taking into account factors such as location, time of day, and weather conditions. The music is organised and controlled in a multi-hierarchical relationship of mappings, creating a unique listening experience for the user.
The Semantic Machine is the result of a collaboration between artist Tracy Redhead and post-doctoral researcher Florian Thalmann as part of the Fusing Audio and Semantic Technologies for Intelligent Music Production and Consumption (FAST) project. The project was funded by EPSRC and included partners such as Queen Mary University of London, The University of Nottingham, and the BBC.
The artwork also explores themes related to ethical considerations of AI, data profiling, and social privilege. It serves as a reminder to pay attention and be involved in conversations around the ethics of data and AI. The artwork was composed and built with the help of semantic audio technology.
The work serves as a metaphor for surveillance capitalism, a warning of the dangers of monetizing personal data and the potential loss of personal autonomy. The machine speaks to us in a dark tone, warning us that we are on the edge of hindsight, and urging us to take action to preserve our personal autonomy.
What is the Semantic Machine?
We wanted to create
a song that is fluid and adjusts
to its surroundings
The Semantic Machine a fictional system based on the datafication of human experience. The semantic machine is a metaphor of surveillance capitalism. This system is made of up of your personal data that you give away for free every time you use your computer, smart devices, wearables. The work is an ever-changing song that is influenced and controlled by the location, weather, and time of day of the user. Conceptually the semantic machine is the centre of this artwork, which is an argument with the human mind and AI algorithms. It serves as a warning of surveillance-driven business models undermining our personal autonomy. (Balsillie, 2021)
The semantic machine speaks to us in a dark tone that it knows us better than we know ourselves. It warns us that we are on the edge of hindsight, that monetising our personal data, our experience, our inner thoughts are a plague in our society. The machine tells us to ignore what is happening and carry on, too dance away all your fears of doomsday. The song changes just like we all do based on the weather, time of day and location. It’s like the work has a mind of its own.
Creators
Tracy Redhead and Florian Thalmann
Tracy Redhead
Tracy Redhead is a creative interdisciplinary artist working with data mapping, programming, music interaction, composition, production, and dynamic music. She is Lecturer of Electronic Music and Sound Design at the UWA Conservatorium of Music.
After releasing her last Album in 2008 as a singer/songwriter, she has been developing her skills and qualifications via a Research Masters (QUT) and PhD in Music (University of Newcastle). She recently won the EU-Commission funded ‘Music Bricks’ award and was invited to participate in the ‘Performance Lab’ at Music Tech Fest (Berlin and Stockholm).
Florian Thalmann
Florian Thalmann is a musician and postdoctoral researcher in semantic audio, computer music, and interdisciplinary arts at Kyoto University. He has degrees in music theory, computer science and art history and has previously worked at the University of Bern, University of Minnesota, and Queen Mary University of London.