The project will combine the Innovate UK-funded Brighton Digital Catapult 5G testbed for high-speed content streaming and low latency, Google Resonance Audio tools for 3D sound, the Echoes Creator for geolocative sound and outdoor heritage locations in Brighton. This will work with ambisonics, user location, head-related transfer function (HRTF), environmental modelling to provide accurate convolution reverbs to provide immersive experiences, streaming multi-channel ambisonic audio, 5G technology and other emerging relevant audio and mobile technology. It will advance scientific solutions for achieving low latency spatial audio with 5G. A co-creation methodology will engage audiences and heritage partners throughout the design, production and evaluation process.
The project will analyse the user experience of heritage content when walking around urban outdoor heritage sites (eg Brunswick Square in Brighton) with specific headphones linked to a smartphone. They would listen a 3D sonic experience designed to convey information about the specific heritage context (including sounds, music and spoken words), something that could be imagined as an experience that includes features of 3D computer game sound, GPS sound walks, guided heritage tours and immersive AR/VR experiences.
This builds on existing research undertaken at Brighton on 3D immersive audio technology, GPS audio for heritage sector, 5G immersive experiences, augmented reality, and cultural and zheritage audience development though mobile/digital technologies (eg NetPark).
The project aims to address the following research questions.
What kind of heritage-specific content emerges through experimentation with 3D immersive audio in a 5G testbed?
How could this help heritage institutions to engage new audiences and to understand in detail how they behave near cultural heritage (user movement, location, interaction data)?
How do audiences experience heritage engagement when delivered through immersive 3D audio in public or urban spaces?
What does a prototype public lab for the creation, curation and dissemination of 3D immersive audio look like and what can the heritage sector learn from it?
Following a literature review, this project involves a two-step experimentation phase to create a working setup/prototype environment. In the first step, this involves engaging with the key emerging technologies individually, including Echoes, Google’s Resonance Audio and the 5G testbed, reviewing different HRTF techniques/models to find the best match for sonic AR, testing different IMUs (inertial measurement unit) to find the most accurate head-tracking technologies.
The second step links these technologies, for example using multi-channel ambisonic (360º) recordings in a locative context to provide soundscapes which react to the orientation of the listener, combining ambisonics and location to enable users to move between ambisonic recordings, working with environmental modelling to provide accurate convolution reverbs to provide immersive experiences, streaming high-resolution 3D sound (4–16 channels) via 4G and 5G technology. This makes a contribution to achieving low latency spatial audio with 5G.
Co-design (beyond user-centred/ux design) will be a key feature of the methodology, including co-creation workshops with heritage and industry partners as well as members of the public (potential audiences). The 3D heritage audio content will be developed through an iterative process that includes user testing at all stages. Analysis of user data (movement, location, interaction) will also be key and will be done in R. Qualitative audience responses will also be collected (questionnaires or focus groups, observations) and the analysis will include triangulation between quantitative user data and qualitative audience response data. Overall, this methodology creates a prototype public lab for the creation, curation and dissemination of 3D immersive audio for the heritage sector. The role of the University of Brighton is to lead and oversee the project, including academic and methodological rigour, and to provide access to its sound studios.
The role of the heritage partner, The Regency Town House, is to provide information and data about the heritage context of the prototype location (Brunswick Square) and to support co-design workshops and content creation. The role of the industry partner is to provide technical insights and support into the audio technology and processes. The role of the Digital Catapult Centre Brighton is to provide access to the 5G testbed.