Apple has filed a patent that shows how future Mac computers could use cameras and LiDAR to spot hand gestures while also offering Face ID login. The research outlines a system that combines fast “event cameras” with normal frame cameras to track hand motions in real time.
How Gesture Detection Works
The system would use two kinds of sensors. A frame camera would capture full images at a standard rate. An event camera would record pixel changes instantly as they happen. Event cameras can track quick motions more accurately than frame cameras alone. The frame camera would first find where the hands appear. Then the event camera would focus on that area to log gestures precisely.
Role of LiDAR and Bounding Boxes
A Mac’s LiDAR scanner could map the distance between the user and the screen. By knowing that distance, the system can draw a “bounding box” around the hands. This box tells the event camera where to look, so it does not record unnecessary background data. The patent describes using depth data and image masks to separate hands from the rest of the scene.
Link to Face ID on Mac
Although the patent never says “Face ID” by name, it implies use of the same TrueDepth sensor array found on iPhones and iPads. That array combines infrared flood illuminators, a dot projector, and cameras to read facial features. A similar setup on a Mac would support secure login and unlock, and could run gesture detection at the same time.
Challenges and Design Notes
MacBook lids have long been too thin to house the full TrueDepth system. Apple may need to alter the screen housing or use a small pod above the display. Illustrations in the patent show cameras at the base of the screen, but still angled to face the user. The company is exploring how to pack these sensors without making the Mac look bulky.
The patent demonstrates that Apple is scheming how to make Mac interaction to be more natural. Tapping the trackpad was unnecessary to scroll or pinch to zoom because users could nod to accept an incoming telephone call, and pinch to zoom and end up. When Apple extends Face ID and gesture detection to the Mac, it may fundamentally alter the way people work and play using a computer.