How should the industry treat AI at the edge? The Partners team at MathWorks, an ST Authorized Partner, sat down with The ST Blog to discuss their perspective on machine learning at the edge and share their experience with the STM32 community.
The member of the ST Partner Program is famous for software like MATLAB®, for scientists to analyze data, develop algorithms, and create models, and Simulink®, for Model-Based Design and simulation of dynamic systems. The latter supports STM32 MCUs, allowing the creation, deployment, and optimization of applications running on ST’s devices. MathWorks offers tools to develop AI for deployment to MCUs. It also covers many areas, including control design, signal processing, and embedded system design, with numerous add-on application-specific products. Consequently, although well-known in academia, the company also has a significant presence in the aerospace, automotive, and industrial markets.
It’s because MathWorks has such a capable and commonly used set of tools that we decided to talk to them about machine learning at the edge. While this is a topic we’ve covered for years, we knew that a take from a partner like MathWorks would help our community see things from a different angle to solve some of their challenges. Hence, we met with John Kluza, Embedded Partner Manager at MathWorks, and Reed Axman, Strategic Partner Specialist.
Table of Contents
Thinking about machine learning at the edge…
…Today
ST Blog: Have we reached a mass adoption of machine learning at the edge?
MathWorks Partners Team: No, there is still work to do before the broad adoption and deployment of machine learning at the edge. The industry is still catching up to the capabilities of edge devices like STM32 microcontrollers, but it is also wrestling with some fundamental challenges like efficiency and safety standards.
Have we reached critical efficiency yet?
No. While the industry has significantly improved the efficiency of neural network algorithms on microcontrollers over the past five years, we still need to make systems more efficient and easier to deploy reliably before machine learning at the edge reaches mass adoption. The technology has reached many niche markets, but it still has some ways to go before it becomes mainstream.
…Tomorrow
What needs to happen to make AI at the edge efficient?
One critical milestone is the ability to run the system code along with multiple networks on one embedded system. Right now, microcontrollers run one neural network very well. Running more than one, along with the rest of the system code while staying within the product’s power budget, is still challenging to achieve. At the same time, it’s also essential to guarantee that the right safety mechanisms are in place.
What do you mean by safety in AI? (Laughing) Will robots attack humans?
(Laughing) No, nothing like that, and science-fiction scenarios are doing more harm than good to this vital conversation. Safety in AI is about operating reliably. It means avoiding unexpected behaviors, ensuring fault tolerance and fail-safe operation, as well as protecting data.
How would developers create a safer AI application?
It’s not an easy question to answer, and it’s the reason why MathWorks offers neural network testing and validation tools. Simply put, MathWorks allows teams to verify the behavior of machine-learning algorithms and create redundancies before deployment in the field. Users can also simulate interruptions or changes in the sensor data to anticipate real-world conditions better.
Creating machine learning at the edge
The best approach to data science
What advice would you give a startup working on machine learning at the edge?
Clearly understand the specific requirements of your use case. Is real-time processing a primary requirement? Is data privacy a concern? What are the computing and power constraints? Once teams settle on a hardware specification, they can narrow in on AI models with favorable tradeoffs between inference speed, memory usage, and prediction accuracy. In that regard, leveraging existing tools and libraries can maximize efficiency and reduce the time to market. I’d also recommend creating a plan for securely handling data, anticipating connectivity issues if there’s an online component, and determining what scaling would look like. Once they have thought this through, they can better decide whether to hire data scientists to work on the machine learning algorithm.
So hiring a data scientist isn’t the first step or even always necessary?
It can be in certain situations. However, you don’t have to be a data scientist or an AI expert to create highly accurate models with MATLAB. Engineers already possess deep technical knowledge about their data and the problem they are trying to solve, often in a way that data scientists do not. For example, teams can start by designing a system with AI in mind and even try including a network from a model zoo before escalating to expensive and rare data science talent. It will help better define the problem to solve. Moreover, the current MathWorks tools enable systems engineers to make meaningful progress on AI problems themselves.
How should companies start working on a machine-learning algorithm?
With tools like MATLAB and Simulink, users can simplify data science and streamline deployment to embedded devices. We discuss workflows for deploying efficient AI networks to MCUs on our TinyML landing page, which also features how-to videos and webinars.
Does MATLAB replace a data scientist?
Obviously not. However, the functionalities provided by MATLAB, along with its interoperability with other deep learning platforms such as PyTorch and TensorFlow, facilitate collaboration and help expedite the implementation of machine learning at the edge.
The smart way to think about AI
Do you have examples of machine learning at the edge solving a problem?
Yes! With MathWorks and STM32 MCUs, AI at the edge can improve a product’s capabilities and often reduce system costs. We even offer a library of user stories that showcase leading companies using MATLAB and Simulink for machine learning and deep learning, including virtual temperature and pressure sensors, lidar classifiers, and ECG analysis.
For an example of computer vision and machine learning, check out MPCam and MPTherm, computer vision starts with an STM32MP1 in an all-in-one solution
When is it best to avoid machine learning at the edge?
There are cases where it’s possible to adequately control a system with traditional methods that require less computing resources than a neural network. For instance, Kalman Filters allow expert engineers to model a system as a big linear algebra problem, which can offer satisfactory performance in some situations.
What do you do if you don’t have access to such expertise or resources?
That’s when machine learning at the edge comes in. Sometimes, companies can’t get the expertise necessary to develop an accurate model. Similarly, if the system is very nonlinear or highly variable, it might not be possible or advantageous to model it with traditional methods. In these cases, machine learning at the edge can solve the same problems more cost-effectively. And if teams prune and compress their neural network algorithm to optimize it, they might even end up with a lower bill of materials!
What is more important, training or optimization?
They both play a critical role. However, if you have great training data and create a powerful model but have a poor implementation, it’s all for naught. Consequently, optimization is critical. A low-power system and a small memory footprint with an adequately performing network can get a team out the door. Improving a model’s performance over time is easier when teams have already optimized the underlying implementation.
The ST and MathWorks ecosystem
What does a collaboration between MathWorks and ST bring to the industry?
MathWorks provides the portability many developers require as they work with multiple MCU makers. Additionally, MathWorks offers a unified workflow that spans from data science and the creation of neural networks to the optimization, simulation, and deployment of those networks. On the other hand, ST software like STM32Cube.AI supports generating C code for STM32 MCUs, and STM32Cube.AI Developer Cloud adds network benchmarking and a Model Zoo.
How do STM32Cube.AI and MathWorks tools fit together?
Using STM32Cube.AI and MathWorks tools together offers a comprehensive workflow. For instance, engineers can start with a network from the ST Model Zoo, TensorFlow, PyTorch, or MATLAB and use STM32Cube.AI Developer Cloud for initial benchmarking. It thus helps developers choose the optimal target for cost, performance, and inference time. Teams can then integrate their models into Simulink to test them within a system. Afterward, the STM32 Hardware Support Package and Embedded Coder can perform Processor-In-The-Loop (PIL) tests and Rapid Prototyping, thus allowing engineers to evaluate the AI model, accompanying control logic and overall performance to see if it meets expectations.
Why should developers care?
Beyond the code generation that STM32Cube.AI offers, there are prototyping considerations to take into account. For instance, running the same benchmark on multiple Nucleo Boards thanks to the STM32Cube.AI Developer Cloud can help teams quickly choose the best device for each project.
What should STM32 developers do now?
We recommend they check out MathWorks tools for deploying AI to MCUs and learn more about our STM32 NUCLEO support. From there, they can watch videos, explore examples, or read documentation as they get started.
