ST’s fifth-generation Human Presence Detection (HPD) is there to protect privacy and battery life when no one else is. Literally! Indeed, the technology, based on our VL53L8CP Time-of-Flight sensor and a machine learning algorithm capable of running on low-power microcontrollers, determines when the user is no longer in front of a display or is looking away, triggering a sleep mode or a system lock-up. It can also offer zero-touch login, recognize head orientation, or gestures. There are already more than 280 PC models equipped with the previous generations of our Human Presence Detection.
Table of Contents
What are the benefits of our 5th-generation HPD?
Greater efficiency
The fifth generation of our HPD reduces power consumption by over 20% compared to the previous generation, thanks in part to a more efficient VL53L8CP time-of-flight sensor and optimizations to our neural networks. The multi-zonal architecture of the device, which supports 8 × 8 zones, also enables new features compared to using a single-zone sensor.
For instance, a multi-zonal model can detect more complex movements, such as gestures. And now, thanks to our machine learning algorithm, ST’s new HPD solution can detect finer movements, like six different head orientations, to dim the screen when users are looking away. It also recognizes body postures, seven hand gestures to mimic live reactions on Teams, such as a thumbs up or hand waving, or whether another person is looking over the user’s shoulder.
Still the same privacy
Additionally, due to the nature of a Time-of-Flight sensor, which measures the absolute distance to the target based on the time it takes photons to travel, the solution protects the user’s privacy. There are no recorded images or videos, and, therefore, no way for a hacker to extract any information that would compromise someone’s identity. The sensor only captures distance information, which can’t be used to extract details about the user’s physiognomy, for instance. And since ST’s HPD only relies on our ToF sensor, our technology still works even if a tape or shutter covers the webcam, a common practice nowadays.
Why wait for 2025 to get all these features in a PC?
A well-known problem
The problem of displays and computers left on while no one is using them is not a new one. According to a 2004 report from the Lawrence Berkeley National Laboratory, “nighttime audits of office buildings [in the United States] found that 64 percent of computers, […] [and] 82 percent of LCD monitors were left on at night.” The study does not say how many were locked and required user authentication, but we can reasonably assume that a number of them were usable without entering a password. Not only is that a tremendous waste of energy, but it means those systems are vulnerable to an evil maid attack.
A well-known challenge
However, manufacturers have been wrestling with this problem because there were no straightforward solutions until ST’s HPD. And now, thanks to our fifth generation, manufacturers have a whole new toolset at their disposal. Indeed, for the longest time, processing a camera feed was too computationally demanding and came with harrowing privacy implications. And while the latter issue is now mostly resolved, as computer vision becomes ubiquitous, using a camera remains costly and a privacy nightmare. It also burdens battery life significantly. Hence, engineers experimented with alternative solutions, such as passive IR sensors capable of detecting body heat. Unfortunately, they proved highly inaccurate, as the seat in front of the computer remains warm long after the user has left.
Similarly, sensors that detect movements are problematic when users are in front of the computer but not moving, such as when they are reading. Some manufacturers have even attempted to use radars, which are very sensitive and can detect when a person is breathing. The problem is that they are so sensitive that they can pick up someone in another room, despite the presence of walls, and thus fail to detect that the user is no longer in front of the screen. Moreover, radars are also power-hungry and expensive, which is incompatible with the constraints governing most laptops. These challenges, therefore, explain why ST’s Human Presence Detection technology has been so popular.
What is at the heart of ST’s HPD?
Precision
At its core, ST’s HPD uses a Time-of-Flight sensor, which includes a laser emitter that sends an eye-safe and non-visible 940-nm wavelength IR light, waits for it to bounce off a surface, and uses the time it takes for a round trip from and to the sensor to determine proximity. Over the generations, our HPD evolved from using sensors that could cover a single zone to devices with multiple zones, such as the VL53L8CP, which includes 8×8 zones.
With the increase in zones and the improved signal-to-noise ratio comes an increase in precision. For instance, our device can detect the most minute chest movements without triggering a false positive when someone is in another room. Additionally, ToFs are significantly more cost-effective, and makers can hide the ToF sensor behind a black cover window without affecting its performance.
Efficiency
Using a device like the VL53L8CP also helps with power consumption. In standby mode, meaning when no one is in front of the computer and the system is in a sleep state but must still be ready to detect presence if someone shows up, ST’s HPD consumes about 0.9 mW, whereas competing solutions with an image sensor require ten times that. Similarly, once in use, our technology requires 28 mW, including the use of our machine-learning algorithm, while traditional solutions often exceed 100 mW. Given that HPD systems operate constantly to respond immediately to changes, these savings can have a noticeable impact on battery life.
Artificial Intelligence
Another critical innovation heralded by the fifth generation of our HPD technology is the use of four neural networks, one for human/object recognition, another for head orientation, one for hand posture, and a last one for body posture. The choice may seem unusual, considering the application must run on a resource-constrained sensor hub. However, it’s precisely why we are using AI. Traditional algorithms would be too big and demanding for a small MCU. The only way ST can offer wake-on-attention, adaptive dimming, walk-away lock, and hand gestures is by using optimized neural networks. The ST development kit, designed for manufacturers, runs on an STM32F4, but we ensure our algorithm fits on whatever PC makers use.
What did we do to make our HPD fit in any PC?
Optimized neural networks
Succinctly, for each AI network, ST acquired training data, labeled it, and designed each AI network topology to adapt to the targeted feature. The resulting algorithms take several inputs from the ToF sensor, typically a map of the distance determined by the sensor’s 64 zones and the signal’s amplitude. The inputs first pass through a few CNN layers, then several perceptron layers. Ultimately, the system can classify whether the object in front of the chair is human or not, recognize head orientation among six different positions, or distinguish between seven hand postures, which can replace live reactions in applications like Teams.
An optimized binary
Working in such a resource-constrained environment does demand a very high level of optimization. That’s why we work closely with every manufacturer to tailor our solution to their bill of materials. ST uses their toolchain, we optimize our code for their architecture, and compile our application for their systems. It enables us to run our machine learning application on their sensor hub, but it also means that we ship a binary rather than source code. PC manufacturers appreciate the fact that ST handles the complexity of creating such an application. It also explains why we don’t provide a publicly available code base.
Can any developer use something similar from ST?
Those wishing to experiment with our technology can grab an evaluation kit compatible with our Smart Presence Detection solution. They features a close cousin of the VL53L8CP, the VL53L8CX. We also provide a demo application that is a simpler version of our HPD technology. For the obvious optimization reasons listed above, it doesn’t feature a machine learning algorithm. However, engineers can still run an interesting smart presence detection system that could transform appliances or security products. Put simply, it’s a way for ST to make presence detection more accessible to all engineers.
