Skip to content
BackHCRL · BIONIC HAND

Bionic Hand

A browser analog of my HCRL research — your hand drives a 16-DOF robotic hand, live.

Hand tracking available on desktop — idle demo shown

Human Centered Robotics Lab · UT Austin

From a sensor glove to your webcam

At HCRL I wrote C++ firmware on a Raspberry Pi Pico that synchronizes a 16-DOF bionic hand to a remote sensor glove at ~1ms sensor-to-actuation latency. This is the public, in-browser analog: the webcam replaces the glove, but the pipeline — capture telemetry, solve joint angles, actuate the hand — is the same shape. Nothing is uploaded; the model and the math run entirely on your device.

01 · Track
Track

MediaPipe Hands runs in your browser (WASM) and returns 21 3D landmarks per frame — the webcam stands in for the lab's sensor glove.

02 · Solve
Solve

Bone vectors between landmarks resolve into per-joint flexion angles, thumb opposition, and wrist orientation — ~16 DOF of articulation.

03 · Actuate
Actuate

Those angles critically-damp the procedural hand's joints in the render loop — the same telemetry→angle→actuation path as the real firmware.

MediaPipe HandsReact Three FiberJoint kinematicsC++ / RP2040 (real rig)