New research at De Montfort University Leicester (DMU) is exploring a radical new way of experiencing music by allowing listeners to step inside orchestral compositions, to move through sound in a digital realm, and even reshape elements of a performance around them.

Using commercially available virtual reality headsets, researchers transformed classical works by Gustav Holst and Béla Bartók into immersive digital worlds where users can explore music as a physical space rather than simply listening passively.

The project was led by Dr Jethro Shell, Senior Lecturer at DMU's School of Computer Science and Informatics, and Professor Sophy Smith from the University for the Creative Arts, with support from the Philharmonia Orchestra.

The Instrument groups visualised as cubes within the Neptune experience virtual environment, where each colour represents a separate instrument collection.

Their research, published in the journal Electronics, introduces a new framework called Multi-Stem Audio Visualisation in Immersive Spaces - MAVIS for short - which enables orchestral recordings to be transformed into interactive virtual reality experiences.

Participants in the study were able to move through musical environments, approach different instrument groups, isolate particular sounds and, in some cases, physically reposition audio elements within the virtual space to create their own unique soundscape.

One immersive experience was built around Neptune from Holst’s The Planets. In this version, the orchestra was represented by floating coloured cubes arranged within a circular virtual environment. Different colours represented different instrument families, including strings, brass, percussion, vocals and woodwind. Users could move around the space, interact with the cubes and alter the positioning of musical elements around them. 

A second experience, based on a section from Bartók’s Bluebeard’s Castle, used a virtual forest environment with dynamic lighting to represent different musical layers and sounds. Researchers found that this more naturalistic setting slightly improved users’ understanding of the relationship between the visuals and the music.

 A section of the forest representing Bartok's Bluebeards Castle as visualised within a VR headset.

The study identified three important principles for successful immersive music experiences: creating clear visual relationships with musical sounds, allowing audiences freedom to explore and shape their own experience, and ensuring visual complexity enhances the music rather than distracting from it.

Researchers believe the framework could help musicians, orchestras, museums, galleries and immersive technology companies develop richer and more engaging experiences for audiences as virtual and mixed reality technologies become increasingly accessible.

Potential future uses could include interactive concerts, immersive museum installations, music education tools and new forms of digital artistic performance.

Dr Shell said: “Most people experience orchestral music as something fixed and distant. This research explores what happens when listeners can move through the music, interact with it and shape their own experience of a composition in real time.

“We wanted to explore whether music could become a place people inhabit rather than simply something they listen to. By using frameworks like MAVIS in virtual and mixed reality environments, composers and designers can create entirely new forms of multi-sensory musical experience.”

The research was part-funded by the Arts and Humanities Research Council’s Audience of the Future programme.

Published in the journal, Electronics, the paper can be found here: MAVIS: Multi-Stem Audio Visualisation in Immersive Spaces Framework.

Posted on Friday 12 June 2026