KuMa Digital
A complex interactive installation for cultural heritage institutions, combining cutting-edge web technologies to create immersive digital museum experiences.
The Challenge
Two partner institutions needed a digital platform to increase visitor engagement while maintaining zero-downtime content publishing for ongoing exhibitions.
The Solution
Built a multi-layered interactive platform featuring Three.js for 3D room exploration, egjs-view360 for 360-degree virtual tours, MediaPipe computer vision for hand gesture recognition, Fabric.js for canvas-based drawing tools, touch interactions via Hammer.js and Interact.js, GSAP animations, and spatial audio with procedural effects.
The Results
Increased visitor engagement by 45% across two partner institutions with zero-downtime content publishing.
What I built
MODULAR EPISODES
Multi-experience platform built around standalone episodes sharing a core engine I authored. Each ships with its own content, build, and version - the curators update one without redeploying the rest.
360° IMMERSION
Wired @egjs/view360 with custom hotspot logic and procedural spatial audio that pans with head motion. Visitors walk curated room tours rather than skim a single skybox.
CREATION TOOLS
Built a Fabric.js canvas workspace where visitors sketch, assemble forms, and save their creations to the museum's gallery - touch-first inputs so it works on kiosk monitors and tablets without a mouse.
3D & ANIMATION
Hand-tuned Three.js room scenes paced by GSAP timelines, with motion timed to each artwork rather than imposing a generic house feel. Frame budgets watched closely so installation hardware never stutters.
GESTURE & TOUCH
Gesture + touch-first input layer combining MediaPipe hand-pose tracking with Hammer.js and Interact.js. Visitors can pinch, swipe, or wave at the camera; the platform routes whichever signal lands first.
ZOOM & DETAIL
Optimized for high-res art assets: progressive image pyramids feed ngx-image-zoom, with predictive prefetch so visitors can drill into brushstrokes without a single frame drop.
MULTILINGUAL
Transloco-driven i18n covering all episodes. Keys are split per-episode so curators can edit copy in one room without redeploying the platform.
STATE & SYNC
Centralised RxJS + NgRx state coordinating the episodes - current room, audio mix, gesture mode, and visitor session all live in one observable graph so the platform stays consistent across screens and inputs.