Case Study · XR/VR
XR/VR Rehabilitation System
Cross-platform XR rehabilitation with hand tracking, upper-limb exercises and multi-user therapy sessions.
Multi-user XR rehabilitation system for upper-limb therapy with hand tracking, synchronized avatars and controller-free interaction.
- ROLE
- Software engineering — XR interaction architecture, hand tracking, avatar systems and multiplayer synchronization.
- YEAR
- 2025
TECH_STACK
Technologies
VIDEO DOCUMENTATION
Video documentation
EXTERNAL_MEDIA_BLOCKED
This project video is hosted on YouTube. Enable external media in cookie settings to load the embed here.
VRArm Dual View
CONTEXT
Technical context
Upper-limb rehabilitation in XR requires more than rendering a virtual environment. Users need responsive controller-free interaction, accurate hand and finger representation, synchronized avatars and reliable shared-session behavior between therapists and patients.
MEDIA
Project media
PROBLEM
Problem
The existing rehabilitation environment relied primarily on headset controllers and lacked complete hand-based interaction and synchronized virtual hands for connected users. The system needed natural controller-free manipulation, correct avatar behavior when switching input modes and reliable synchronization of detailed hand and finger movement across shared sessions.
FILES
Project files
- View PDF
XR Rehabilitation System
PDF
OVERVIEW
Project overview
This project extends a shared XR rehabilitation environment for guided upper-limb therapy between therapists and patients.
It introduces two hand-tracking-compatible interaction rigs built around OpenXR and Meta XR workflows.
The system supports controller-free hand interaction, synchronized hand and finger movement and full-body avatar integration through inverse kinematics.
Hybrid input handling allows users to switch between tracked hands and physical controllers while preserving the correct interaction and avatar state.
Configurable scaling enables adjustment of the avatar, arms, hands and palms to better match individual users.
The complete solution operates within a multi-user session architecture designed for shared rehabilitation exercises.
APPROACH
Technical approach
- Built two hand-tracking-compatible XR interaction architectures using OpenXR and Meta XR workflows
- Implemented virtual hand visualization and mapped tracked hand joints to avatar hands and fingers using inverse kinematics
- Added near, far, poke, grab, ray, teleport and object-use interactions for controller-free rehabilitation tasks
- Implemented hybrid input handling for switching between hand tracking and physical controllers
- Extended Mirror Networking to synchronize hand transforms, finger movement and interaction state across participants
- Developed configurable scaling for the full avatar, arms, hands and palms
- Validated functionality and runtime performance on Meta Quest 2 and Meta Quest 3
FOCUS
Technical focus
- XR interaction architecture
- Real-time hand and finger tracking
- Avatar inverse kinematics
- Controller-free interaction systems
- Hybrid hand and controller input
- Multiplayer state synchronization
- Cross-platform XR workflows
- Avatar scaling and calibration
- Real-time 3D interaction
- Runtime performance validation
ARCHITECTURE
Architecture notes
- Local hand-tracking input is separated from avatar animation and networked representation
- Tracked hand-joint data drives virtual hand rigs and avatar finger articulation before synchronization
- OpenXR and Meta XR rigs share the same rehabilitation environment through separate interaction abstractions
- Hybrid input management controls transitions between tracked hands and physical controllers
- Mirror Networking synchronizes hand transforms, finger movement and interaction state across participants
- Inverse kinematics maps tracked hand and controller targets onto full-body avatars
- Avatar scaling is handled independently for the body, arms, hands and palms
- Rig recalculation is applied after proportion changes to preserve correct avatar alignment
- Device-specific tuning is isolated from the shared interaction and session architecture
HIGHLIGHTS
Engineering highlights
- Two hand-tracking-compatible XR rigs using OpenXR and Meta XR
- Real-time virtual hand and finger articulation from tracked joint data
- Controller-free near, far, poke, grab, ray and teleport interactions
- Hybrid switching between hand tracking and physical controllers
- Inverse-kinematics integration for full-body avatar synchronization
- Mirror-based synchronization of hand transforms and finger movement
- Shared therapist–patient rehabilitation sessions
- Configurable avatar, arm, hand and palm scaling
- Functional and performance testing on Meta Quest 2 and Meta Quest 3
OUTCOME
Outcome
A functional multi-user XR rehabilitation system supporting controller-free upper-limb therapy through tracked hands, interactive rehabilitation objects and synchronized avatars. The resulting architecture supports OpenXR and Meta XR workflows, hybrid input modes and configurable avatar proportions, providing a strong foundation for further rehabilitation exercises, device support and clinical validation.