02 / AUGMENTED REALITY + MUSIC LEARNING / CASE STUDY

SoundQuest

Sound provides functional feedback.

An AR music-learning experience exploring how playful tasks, audiovisual guidance and embodied interaction can make musical learning more approachable.

ROLE
Full project ownership / all aspects
TOOLS
Unreal Engine 5 · Figma · Blender · Maya · ZBrush · Marmoset Toolbag · Substance 3D Painter · FL Studio · Ableton Live 12 · REAPER
PLATFORM
AR · Mobile interface
DURATION
Jul 2024–Mar 2025
PROJECT TYPE
Individual project
PROJECT OVERVIEW

Learning through doing.

SoundQuest is a UI/UX concept for making music learning more playful through AR exploration, guided practice and audiovisual feedback. The wider app journey, AR tasks and learning companion are design proposals. Virtual Music Hall is an implemented Android first-person music scene built around original 3D assets and an original composition. A separate Android AR prototype tests spatial object interaction.

Concept world / virtual Music Hall
LEARNING CHALLENGE

Make an abstract subject tangible.

Music theory can feel detached from the act of making music. SoundQuest explores a more embodied approach: learners discover tasks, interact with instruments and receive feedback in context.

Its visual world and interface turn learning into an exploration, connecting AR instruments, a virtual music hall and guided piano practice.

USER JOURNEY

A clear loop from action to response.

Progression supports the musical activity instead of replacing it.

  1. 01

    Discover

    Find a musical task.

  2. 02

    Explore

    Encounter an instrument in AR.

  3. 03

    Play

    Try a musical interaction.

  4. 04

    Listen

    Receive audiovisual feedback.

  5. 05

    Continue

    Move to the next task.

INTERACTION DESIGN

Different ways into music.

The concept connects mobile UI, AR exploration and practice support. Virtual Music Hall is the implemented first-person Android module: players can move through the music environment and start an original song. Separately, an Android AR prototype tests placing and transforming the project’s 3D music assets in physical space.

AR instrument exploration

A designed experience in which learners discover music-related tasks through their surroundings.

Piano guidance

A concept for connecting visual cues with physical keys and learner actions.

Visual Music Hall

An implemented Android first-person scene where players move through the environment and start original music.

Piano learning / project demonstration
Projected guidance / close-up
UI/UX DESIGN

One service, several ways to engage.

These screens document the SoundQuest product design: an experience that connects location-based discovery, AR instrument encounters, sound collection, personal progress, community and reflective listening. They are UI/UX design outputs that define the proposed service; they are not presented as a complete released application.

Mission hub

A clear entry point for sound collection, location exploration and meditation.

Discovery task

A GPS-led task turns finding an instrument into a legible, step-by-step goal.

Personal and social progress

Profile, vault, leaderboard and community views make contribution and progress visible.

Reflective modes

Sound collection and meditation extend the experience beyond a single task.

IMPLEMENTATION STATUS

Separate evidence from future direction.

Designed: the SoundQuest app journey, AR learning tasks, guided piano practice, feedback system and community features remain concept work.

Prototyped / implemented: Virtual Music Hall is an Android first-person scene with original music playback; a separate Android AR prototype supports object placement and transformation.

Future direction: haptic or vibration feedback remains a designed direction unless implementation evidence is supplied.

Virtual Music Hall — implemented Android scene

Virtual Music Hall is a completed first-person Android music scene within SoundQuest. I created its 3D music assets and environment. The player can move through the scene and click a play control to start my original song. This is a focused implementation of spatial exploration and music playback, rather than a full build of every proposed SoundQuest learning feature.

Watch the Android AR prototype walkthrough

Android AR prototype / Unreal development workspace

Separate Android AR interaction prototype

The AR Pawn handles plane candidates and object placement. Its Blueprint also separates one-finger selection from two-finger gesture actions used to translate, rotate and scale the placed object.

Android AR prototype / interaction logic overview

Implemented music playback

Within the first-person Android scene, clicking the play control starts an original track composed for Visual Music Hall. This is the implemented audio interaction evidenced in the prototype walkthrough.

Concept / future direction

Haptic / vibration feedback: Designed / Future Direction.

TESTING & EVALUATION

Prepare the questions before the evidence arrives.

The evaluation should test whether feedback makes the next action clearer, rather than assume that it succeeds.

Evaluation questions

  • Can users understand the AR cue?
  • Can users identify the next action?
  • Does sound feedback help learning?
  • Is projected guidance readable?

Prototype review observations

  • The location task gives the learner a recognisable purpose before entering the AR view.
  • Full-screen AR imagery makes the instrument the focus, while a single action keeps the next step legible.
  • The home screen needs to distinguish exploration, recording and reflective listening at a glance.
  • Progress views are most useful when they show both collected activity and a reason to return.

These are design-review observations from the prototype and interface walkthrough, not findings from a formal user study.

Design changes

  • Added task copy and a visual instrument clue to make the GPS objective clearer.
  • Reduced the AR screen to one primary action: collect the instrument bonus.
  • Grouped mission, community, vault and profile under a persistent home navigation.
  • Separated sound collection and meditation into dedicated modes so exploration does not compete with reflective use.
REFLECTION

Feedback needs to explain what happened.

Audio and visual cues should connect an action with a musical response. The remaining design question is whether that response helps a learner understand what to do next.

Original music / production process
DEMO

Show the working prototype.

This walkthrough documents the implemented Virtual Music Hall scene, where the player moves through the Android environment and starts an original song, as well as the separate Android AR object-interaction prototype.

Watch the Android prototype demonstration

NEXT PROJECT

The Sacred Sin