MEMS Studio now supports ST’s intelligent sensor processing units (ISPUs) like the ISM330ISN and ISM330IS. The tool is ST’s most extensive visualization, evaluation, profiling, processing, and optimization software for machine learning applications running on our sensors. It even includes an automatic filter selection utility for the LSM6SDV16X and others.
As the successor of Unico-GUI, Unicleo-GUI, and AlgoBuilder, MEMS Studio ensures that even engineers with little to no background in sensing and data science can design and configure the device. Moreover, it helps developers design decision trees and neural networks on their sensors to create smarter and more energy-efficient applications. Moreover, it brings code generation, sensor configuration, and data analysis into one application. Indeed, while we will continue to allow users to download Unico-GUI, Unicleo-GUI, and AlgoBuilder to support developers currently relying on them, MEMS Studio is now our flagship solution over time.
3, 2, 1…
Finite state machines and machine learning cores
Faithful readers of the ST Blog will remember the introduction of the LSM6DSO in 2018 and the subsequent LSM6DSOX. It was the first MEMS with a finite-state machine, which meant that it could classify motion and gestures without waking the MCU, thus offering unparalleled energy savings. And when the LSM6DSOX came with its machine learning core, it won awards because it transformed how the industry perceived AI at the edge. For the first time, it was possible to run a decision tree from within an inertial sensor, thus opening the door to smarter applications on the tiniest systems that required 20 to 100 times less current than when running the same algorithm on a microcontroller.
Unico-GUI
To make all this more accessible, ST took advantage of Unico-GUI. The solution is popular amongst developers working with ST MEMS, which made our devices more accessible by making them, in part, more easily configurable. For instance, engineers could use a familiar utility to setup their machine learning core or finite state machines. For instance, the ST Authorized Partner Synapse shared how their engineers found Unico-GUI so intuitive that it alleviated the need to outsource some work to dedicated software developers. Even university students found Unico-GUI so approachable that it enabled them to design medical innovations that leveraged the machine learning core of the LSM6DSOX to make sit-stand tests vastly easier to administer.
Unicleo-GUI and AlgoBuilder
As the capabilities of the ST MEMS grew, so did the features we offered. For instance, Unicleo-GUI showed how a sensor could process information and display the outcome in a GUI. The software even performed Fast Fourier Transform analysis of accelerometer data to provide frequency domain analysis for vibration sensing applications. Unicleo-GUI also worked alongside AlgoBuilder, a tool released a few years ago to help build and use algorithms while writing little to no code. However, we recognized that using three different pieces of software could get cumbersome. Moreover, as the features available on our MEMS grew, we knew we had to find a simpler way to make them more accessible. Thus came MEMS Studio.
… Let’s Go!
Sensor configuration
In essence, MEMS Studio handles sensor configuration, sensor data analysis, in-sensor AI development, and application creation. As the aim is to tailor MEMS Studio to developers with varied experience in embedded systems, we tried to make it accessible while offering advanced features. For instance, when it comes to sensor configuration, the new software supports development boards and offers UIs to configure devices quickly and start proofs-of-concept in minutes. For a novice, it’s as simple as connecting a board, finding it in the list of supported devices, and clicking Connect. Nonetheless, we also offer the ability to access the entire register map and monitor the status of interrupts directly from MEMS Studio to enable advanced operations.
Sensor data analysis
MEMS Studio offers a myriad of ways to see the information captured. Under the Sensor Evaluation tab, users can see results in a table, bar or line charts, scattered plots, and more. There are even graphical representations like a compass or an inclinometer and the ability to run an FFT analysis. Additionally, a Data Analysis tab can show information in the frequency or time domain and on a spectrogram. There’s even a way to export the data to ensure more complex data analysis. Indeed, MEMS Studio can alleviate the burden that falls on teams trying to gather, label, and analyze data. However, we also know that some will still want to use export data for more extensive manipulations.
In-sensor AI development
For ISPU-equipped sensors, MEMS Studio allows users to import pre-trained machine-learning models from popular frameworks. AI developers can even analyze the underlying neural networks in detail, optimize them, and then compile them for the ISPU. Developers can even validate the model’s performance from within the application. Moreover, MEMS Studio offers an AutoML tool known as AFS. In a nutshell, it’s an automatic filter, feature selection, and decision tree generation tool, described in research that ST published in IEEE Sensors Letters in 2023. The tool automates the generation of a decision tree and greatly simplifies the configuration of the machine learning core.
If we take the example of a labeled dataset captured with a sensor, the framework automatically selects the best set of features, filters, and window size to apply to the dataset from a search space of supported parameters. The framework then trains a decision tree within the resource bounds of the machine learning core and generates the register configuration file to deploy on the sensor for the trained ML model. In the case of a human activity recognition system, the framework achieves 93% test accuracy under 1 mW of power, which is 41 times lower than an ARM Cortex-M4 implementation. The framework also enables on-sensor fall detection with 95% test accuracy and under 0.3 mW of power.
Application development
Finally, MEMS Studio incorporates AlgoBuilder, which means that thanks to its block interface, it is possible to design entire algorithms without writing a single line of code. The software even includes example algorithms to get users started faster. MEMS Studio can also generate code before flashing it onto a board. The ST software can even work with external compilers like STM32CubeIDE, Keil µVision, or Embedded Workbench, to fit existing workflows. Additionally, developers can use the software to test behaviors to see if their finite state machine and their decision tree will correctly detect gestures or classify information.
