Spatial music systems / Audio-reactive interaction
Sound Escape VR
Four-channel spatial sequencer
The primary artifact shows the in-headset music surface: four sample channels arranged across a 16-step grid with transport and tempo controls kept inside the composition space.
Compose a loop and watch the world respond: a spatial music prototype connecting a sequencer to neon geometry, light, and audio-reactive movement.
- My role
- Independent designer and developer — spatial interaction, audio-reactive visual direction, Unity implementation, and Quest adaptation
- Engagement
- Independent Prototype
- Period
- 2023 - 2025
- Resources
- 4 reviewed artifacts
01 / Context
Creative challenge
•Music tools and VR spectacle rarely shared one loop
Traditional sequencers make composition precise but screen-bound, while many music experiences in VR remain passive visualizers. Sound Escape explored whether arranging a loop, hearing it, and watching the world respond could become one embodied interaction.
•Audio response needed to reflect the signal
The environment could not rely on decorative animation alone. Frequency ranges, amplitude, timing, and user-selected samples needed to drive visible changes that remained connected to the music being composed.
•The Oculus-era prototype needed a hardware update
The project began around an earlier Oculus interaction stack. Moving it to Unity 2022.3 and Meta Quest 3 required explicit work across device detection, floor tracking, controller input, grabbing, collision behavior, haptics, and Android XR configuration.
Original Sound Escape identity
The original identity establishes the synthwave world that framed the prototype before the interaction stack was reworked for Unity 2022 LTS and Meta Quest 3.
02 / Mandate
Direction and intent
•Make loop composition graspable in space
Create a direct sequencer surface where a player can place notes across four channels and 16 time slots, adjust tempo, preview the loop, and retain the pattern between sessions.
•Connect sound structure to the environment
Translate live stereo spectrum data into stable frequency bands that could control Koch lines, trails, light, color, scale, and other spatial feedback.
•Preserve player-created state
Keep grid patterns, tempo, sample origins, and short microphone recordings available locally so composition behaved like a tool rather than a one-pass visual scene.
•Adapt the interaction layer for Quest 3
Update the XR camera, input bridge, controller profile, floor origin, grab behavior, hand collision, and haptic settings without presenting planned passthrough or voice work as finished functionality.
03 / Build
Creative decisions and execution
1.Built the stateful 16-step sequencer
Implemented a four-channel by 16-slot grid, scheduled sample playback against Unity DSP time, tempo control from 60 to 240 BPM, play and rewind state, and PlayerPrefs persistence for patterns and tempo.
2.Added sample capture and local recovery
Created four AudioSource channels with 80s synthwave defaults, six-second microphone recording, WAV persistence, sample-origin tracking, and local restoration of recorded or default clips.
3.Mapped eight FFT bands into generative geometry
Used stereo GetSpectrumData calls with a Blackman FFT window, normalized eight frequency bands and buffered amplitude, then drove Koch-line emission, trail color and width, lighting, and object scale from those values.
4.Reworked input and tracking for Quest 3
Added Quest-aware device detection, XRInput or OVRInput selection, floor-origin setup, controller deadzones, grip-or-trigger grabbing, raycast remote grab, hand-collision safeguards, haptic settings, and scene diagnostics for particle and light load.
Eight-band audio-reactive geometry
Project documentation captures Koch lines, neon towers, and the sequencer environment driven by normalized FFT bands rather than a disconnected animation timeline.
04 / Outcomes
Outcomes
•A working composition-to-visualization loop
The repository contains the functional core sequencer, local pattern and tempo recovery, sample playback and recording paths, FFT analysis, and audio-reactive Koch geometry shown in the project documentation.
•The world responds to authored audio state
Player-created patterns and samples feed the same eight-band signal system used by the neon geometry, connecting composition choices to emission, color, trail behavior, light, and scale.
•Quest 3 compatibility became explicit code
A dedicated compatibility update added Meta Quest 3 detection and configuration across camera tracking, input, grab ergonomics, collisions, haptics, Android XR settings, and performance diagnostics.
•A private-build prototype with a bounded roadmap
The public repository and demo document an implemented spatial music system, not a released consumer product. Exploration puzzles, a larger narrative journey, voice controls, broader accessibility work, multiplayer jams, and expanded fractal libraries remain planned or in refinement.
Spatial performance environment
The environment board shows the sequencer, held sample objects, luminous architecture, and reflective space that turn loop building into a spatial performance rather than a flat control panel.
Tools
Tools and implementation
Continue the evidence trail
From proof to role fit
Compare Sound Escape VR with adjacent systems, or carry its reviewed capabilities into an Adaptive Focus brief.