2026-02-20 | 4 min read
From Webcam Gestures to Cursor Control: Air Cursor Notes
Lessons from real-time hand landmark detection, gesture mapping, smoothing, and usability tradeoffs in computer vision UX.
Key Takeaways
- Implemented real-time hand landmark tracking with MediaPipe and OpenCV.
- Mapped gesture states to cursor movement, drawing, and erase actions.
- Improved stability with smoothing logic to reduce jitter in pointer control.
Air Cursor was built to test whether natural hand motion can replace traditional pointer input for short interaction tasks. The project uses webcam input, MediaPipe landmarks, and a gesture mapping layer that converts hand states into cursor events.
The hardest part was not detection accuracy. It was interaction stability. Raw landmark data is noisy, so the cursor can jitter even when gestures are correct. I added smoothing behavior and threshold checks to keep pointer movement usable for drawing and navigation.
I also learned that gesture systems need explicit state transitions. Without clear enter and exit rules, commands like click, draw, and erase can conflict. Defining those states made the overall experience more predictable.
The next milestone is recording a concise walkthrough video that demonstrates tracking accuracy, mode switching, and real cursor interaction in one clip.