Discover STM32 solutions for artificial neural networks
To improve the accessibility of our content, please find the audio version of this blog post.
The FP-AI-FACEREC1 Function Pack is now available on-demand, thus enabling ST’s community to run new applications leveraging facial recognition on an STM32H7, thanks to its use of STM32Cube.AI. The package offers a binary for the STM32H747I-DISCO board and ST’s B-CAMS-OMV camera adapter board. The latter provides an extension connector for OpenMV and Waveshare camera modules. The software handles on-device enrollment, camera control, interfaces, joysticks on the board, image capture, pre-processing, and the machine learning library. Its database can store up to 100 users, and the process runs at 3.6 frames per second on the embedded RAM and flash. As a result, it’s possible to conceive an application that would not require external memory. Moreover, the solution only needs a low-resolution RGB camera, regular ambient lighting, and subjects at up to 1.5 meters (5 feet).
The New Price of Admission
During a roundtable with The ST Blog, a design house shared how customers increasingly want to benefit from AI. However, the barrier to entry is still high. Developing AI models for resource-constrained microprocessors may increase overall costs, and the necessary reliance on data scientists means smaller teams are at a disadvantage. FP-AI-FACEREC1 is, therefore, critical because it shows that it is possible to run a complex neural network algorithm on a microcontroller. Additionally, ST software tools help alleviate some of the inherent complexities to lower the barrier to entry. Put simply, the price of admission to AI just became an STM32 Discovery Kit since all development software works with free ST tools such as STM32CubeIDE and STM32CubeMonitor.
FP-AI-FACEREC1, a New Chapter in the Market Penetration of Machine Learning
Machine Learning Is Becoming a Necessity
The new ST Software package opened the door to applications that can benefit from artificial intelligence but can’t justify massive investments. When smartphones started authenticating users by scanning faces, manufacturers had to inject a lot of cash and manpower. The need for extreme accuracy and the stringent security certifications that govern such use cases demand nothing less. However, given the critical success of facial recognition among consumers, companies saw a rapid return on their investment. Problems arise when other industries need AI but can’t expect the same returns as smartphone vendors.
Another compounding factor is the rise of user-customized applications that rely on facial recognition. Indeed, more and more customers demand the ability to customize settings, alerts, or behaviors for each user. Traditionally, users choose their account by clicking on a button, selecting an avatar, or entering a login and password. With facial recognition software, the interface can automatically recognize users and launches their services or settings. A few years ago, video game console makers brought this idea to the general public. With FP-AI-FACEREC1, it is possible to offer a similar feature without requiring an expensive gaming system.
Facial recognition is increasingly at the center of new applications that focus on the user’s well-being. For instance, a maker of consumer ovens could use machine learning to detect a child nearby while the appliance is hot, and trigger an automatic lock to prevent accidents. Similarly, using facial recognition in an elevator can ensure children only go to their floor to prevent them from getting lost.
Machine Learning Is Becoming More Accessible
FP-AI-FACEREC1 is essential because it enables new industries to benefit from machine learning, thanks to its tradeoffs. Running the code on an STM32H7 means applications use fewer image layers and a lower resolution than systems focusing on secure authentication. An oven or elevator doesn’t need to meet the same standards of accuracy as a computer authenticating a user. FP-AI-FACEREC1 thus shows that it is possible to use less RAM and computational throughput while retaining an accuracy that fits mass-market applications. Moreover, the ST software pack can accommodate a global shutter to reduce motion blur or an infrared sensor to improve low-light performance. Combined with a Time-of-Flight sensor, the Function Pack could tackle proximity detection. Ultimately, the application example serves as a foundation for engineers looking to innovate.
The same solution will also work on an STM32MP1 to satisfy engineers that need more power. Some teams require an embedded Linux distribution to more easily run a web server. Others may need the power of a Cortex-A7 core for a GUI. In all cases, it is possible to use the same TensorFlow Lite model as FP-AI-FACEREC1 and run it on Linux. Developers can, therefore, enjoy more frames per second while benefitting from the same memory footprint. As a result, having a system that can run on a high-level RTOS makes facial recognition far more accessible.
FP-AI-FACEREC1, a New Story About Machine Learning on Embedded System
Facial Recognition Possible Thanks to Innovation
Machine Learning is a complex subject matter, and it can be intimidating. ST’s software package is a solution that tries to demystify the processes at work and that shows how far the industry has gone. The first step is to take a picture to determine if there’s a face to detect. Afterward, a person’s facial features are converted into an array of floating-point numbers (Float32Vector). The system is, therefore, entirely confidential since the picture itself is never stored in the database, and the whole process is entirely local. The application never sends data to the cloud. Finally, FP-AI-FACEREC1 distinguishes itself from the rest of the industry because it uses STM32Cube.AI to optimize its neural network. As a result, it uses less memory and offers greater performance on STM32 MCUs.
Facial Recognition Possible Thanks to Optimization
Another exciting aspect of the ST solution is that it helps understand memory usage and optimizations. For instance, each image captured by the camera takes 150 KB of RAM while the image buffer takes 225 KB. Most of the libraries can fit in the Flash, including the neural network libraries optimized by STM32Cube.AI. As a result, the entire application can fit inside an STM32H7. Developers can then tweak our implementation to fit their needs. However, FP-AI-FACEREC1 ensures that they start from a powerful implementation rather than a blank page.
Facial Recognition Possible Thanks to Collaboration
ST’s Function Packs serve as a stepping stone toward a final product. Teams can leverage demo applications during the prototyping phase before working on a production-ready implementation. To further speed up deployments, engineers can rely on Nalbi, a member of the ST Partner Program. Indeed, the company offers deep learning models for computer vision that it optimized for STM32. Hence, teams can either use FP-AI-FACEREC1 or Nalbi’s services to have a production-ready implementation that takes advantage of STM32Cube.AI and optimizes performance for the most efficient bill of materials possible.
Engineers can experiment with FP-AI-FACEREC1’s pre-trained neural network to achieve a fast proof-of-concept. However, when it comes to production, developers will have to train the machine learning model with their database of faces. Indeed, this step directly impacts the facial recognition against genders, facial traits, skin tones, image angles, and more. As a result, it is imperative to use a training database that reflects the use-case. Alternatively, it is possible to leverage Nalbi’s production-ready software to build a final application.
