What if a database for embedded systems, such as eXtremeDB, were the only thing standing between critical sensor data and a massive catastrophe? The scenario may seem far-fetched. And yet, how many engineers value the thousands, if not millions, of data points collected by their IoT systems? Many engineers carefully review the type of memory they use. Yet, they often gloss over how to structure it, which led McObject, a member of the ST Partner Program, to create the eXtremeDB embedded database management system (DBMS), and eXtremeDB/rt, the first hard real-time DBMS for microcontrollers. With a code size of 150 KB, and no reliance on heap memory, it requires a small client to take advantage of it.
The shortcomings of current systems
How expensive can a text file get?
Returning to the premise of this blog post — that losing sensor data could result in a catastrophe — let’s first consider the cost of losing that data. We already have real-world examples of software bugs corrupting sensor logs, leading to massive recalls and costing billions of dollars. Multiple car companies have faced this issue, but others have as well. Moreover, as the world increasingly relies on more sensor data for mission-critical applications, it’s easy to see that beyond the monetary costs, corruption of sensor logs could lead to physical harm. Which leads to the question very few ask: Is a text file the best place to store sensor data?
How tedious can a text file get?
The vast majority of IoT applications store sensor data in a CSV, TSV, or JSON file, but there are issues. If the IoT system experiences a critical crash during a write operation, it is impossible to ensure the file’s consistency. For instance, failing to write the footer of a JSON file renders the entire log unusable. Similarly, if two processes attempt to write to the same log simultaneously, the risk of corruption increases significantly. The lack of replication can also burden systems. For instance, syncing with a gateway that has been offline for a while can require a large bandwidth, which most IoT systems simply don’t have.
eXtremeDB as a solution to protect data, applications, customers, and companies
An age-old idea
Interestingly, those issues are far from new and explain why computer scientists invented databases in the first place. Some may be familiar with the acronym ACID, which stands for Atomicity, Consistency, Isolation, and Durability. It’s a concise way to explain the benefits of using a database. In a nutshell, databases circumvent the pitfalls of using a simple text file by defining a unit of work and implementing mechanisms to make the whole process vastly more efficient and resilient against errors and crashes. McObject explained that some IoT devices with very basic needs may still prefer to use a plain text file. However, many who underestimate the pitfalls of a JSON or CSV document could benefit from a database.
A new solution for embedded systems
McObject specifically built eXtremeDB for embedded systems, meaning that developers shouldn’t view it as a traditional relational database, like MySQL. Consequently, while eXtremeDB SQL can co-exist alongside eXtremeDB, it isn’t the primary setup. Most installations will use eXtremeDB on its own, which can utilize a combination of in-memory and on-disk databases, each optimized to minimize I/O operations and CPU bottlenecks by employing only a few instructions.
eXtremeDB stands out as the only actual real-time database in the industry, meaning it offers deterministic transactions compatible with a real-time operating system (RTOS). Competing solutions may focus on real-time analytics, but they don’t provide deterministic performance. Consequently, developers can utilize eXtremeDB for mission-critical applications, such as advanced driver assistance systems (ADAS).
To make eXtremeDB more accessible, McObject provides APIs in a wide range of programming languages, including C, C++, C#, Rust, Lua, and Python. Additionally, the company worked with ST to integrate its solution into the STM32Cube ecosystem. Concretely, it means that not only does eXtremeDB support both STM32 microcontrollers and microprocessors, but that McObject baked in heterogeneous processing to take advantage of all the cores available. The software company also leverages STM32 cryptocores to accelerate AES encryption, and even supports standard security libraries, like WolfSSL, the eponymous member of the ST Partner Program.
