A reason to build.
Media controls usually assume a tap on a screen. I wanted to explore a different interaction: recognize a hand gesture, map it to an action, and make the result clear without taking over the app already on screen.
How the pieces fit.
- 01
Built the Android interface with Kotlin and Jetpack Compose, with a setup flow for the permissions needed by camera tracking, media access, and the overlay.
- 02
Connected CameraX frames to MediaPipe hand recognition and mapped gestures to configurable playback and volume actions.
- 03
Kept recognition in a foreground service. Added proximity-based waving for night mode, persistent presets, and automation entry points for routines.
- 04
Added a movable status overlay that briefly expands to acknowledge an action, so feedback can sit alongside another active app.
What came out of it.
Implemented the gesture recognition, media-action mapping, day/night modes, background service, settings, and floating feedback interface in a public Android project.
Public Kotlin source includes the recognition service, settings and state models, model tests, and debug/release build instructions.
What I’d carry forward.
Recognition is only one part of the interaction. Permissions, service lifecycle, repeated detections, and visible feedback all shape whether a gesture feels like a reliable control.
Working on something with similar edges?
Let’s compare notes ↗