A webcam toy built with OpenCV and MediaPipe. An illustrated hookah stands in the corner of the frame with its hose hanging on a hook. Close your fist on the handle to pick it up, bring it to your lips and pucker to draw, then move it away and blow: smoke leaves your mouth. Make an “O” for smoke rings, puff your cheeks for a big burst. Open your hand and the hose floats back to its hook.
/opt/homebrew/bin/python3.11 -m venv .venv
.venv/bin/pip install -r requirements.txt
The MediaPipe models are already in models/. To re-download them:
curl -L -o models/hand_landmarker.task https://storage.googleapis.com/mediapipe-models/hand_landmarker/hand_landmarker/float16/1/hand_landmarker.task
curl -L -o models/face_landmarker.task https://storage.googleapis.com/mediapipe-models/face_landmarker/face_landmarker/float16/1/face_landmarker.task
.venv/bin/python main.py # camera 0, 1280x720, mirrored
.venv/bin/python main.py --debug # with landmark and blendshape overlay
.venv/bin/python main.py --source clip.mp4
Keys: q quit, d toggle debug overlay, r reset the session.
macOS asks for camera permission the first time. If the window never opens, grant your terminal app access under System Settings > Privacy & Security > Camera.
camera -> mirror -> HandTracker + FaceTracker (hookah/trackers.py)
-> grip score, mouth shape, head yaw (hookah/gestures.py)
-> state machine + lung reservoir (hookah/session.py)
-> hookah, hose, bubbles, coal, smoke, HUD (hookah/render/)
| State | Trigger | What you see |
|---|---|---|
| idle | hose on its hook, a pulsing ring marks the handle | close either fist inside the ring to pick it up, or hold a fist nearby for half a second and the handle comes to you |
| holding | fist on the handle (grip score above 0.6) | mouthpiece follows the fist, hose sags from the port |
| at mouth | tip within half a face-width of the lips, or fist right by the mouth | tip snaps onto the lips |
| drawing | at mouth and mouthPucker above 0.35 (keeps going above 0.22) |
coal glows, water bubbles, lung meter fills |
| exhaling | hose away from the mouth and funnel / pucker / open jaw | smoke leaves the mouth in the direction you face |
Blendshapes mouthFunnel + slightly open jaw give rings; cheekPuff gives a
burst. Smoke only appears while the hose is away from the mouth. An open or
lost hand for a third of a second drops the hose, which floats back to the hook.
Accuracy notes: hands are tracked by position, never by MediaPipe’s left/right label, so both hands behave the same; the grip score averages eight finger measures; the mouth centre is the mean of twelve lip landmarks; blendshapes and the mouth position are smoothed over a few frames; “at mouth” and “drawing” use hysteresis so they do not flicker at the threshold.
Smoothness: capture, tracking and rendering run on three threads. The window is drawn at the camera’s frame rate with the newest landmarks, so a slow MediaPipe frame never stalls the animation. The hose is a verlet rope with gravity and a floor, so it swings, sags and piles up like a real one.
.venv/bin/python tools/render_demo.py out
Writes one PNG per state to out/ using scripted hand and face observations.
Every threshold and size lives in hookah/config.py; every colour in
hookah/palette.py.