Gesture & Hand Tracking Tutorial
Learn how to use hand gestures to control the 3D structure viewer.
Overview
CatGo uses MediaPipe Hands for real-time hand tracking through your webcam, enabling gesture-based interaction with 3D structures.
Step 1: Enable Gesture Mode
Open the Gesture Settings pane and enable hand tracking. Grant camera permission when prompted.
Step 2: Calibrate
Position your hand in the camera view. The tracking overlay shows detected hand landmarks.
Step 3: Available Gestures
Rotation
- Open palm drag: Rotate the structure by moving your open hand
Zoom
- Pinch: Bring thumb and index finger together/apart to zoom in/out
Pan
- Two-finger drag: Pan the view
Selection
- Point: Extend index finger to hover over atoms
- Pinch tap: Quick pinch to select an atom
Step 4: Customize
Adjust gesture sensitivity, tracking smoothing, and gesture mappings in the settings pane.
Troubleshooting
- Ensure good lighting for reliable hand detection
- Keep your hand within the camera frame
- The MediaPipe model is loaded locally for privacy
Related
- Gesture Tracking Module — Architecture reference
- Voice Control — Combine with voice commands