Zephyr and embedded AI: where everything comes together
Once we have understood embedded AI as a way of bringing machine learning models to the IoT device itself, the next step is to analyse which platform can facilitate its integration into a real-world application. At this stage, Zephyr stands out as a particularly interesting option.
Zephyr can handle several key functions: data acquisition from sensors, task planning, energy management, communications and firmware updates. Furthermore, its ecosystem allows for the integration of technologies such as LiteRT (formerly TensorFlow Lite) and Edge Impulse. It also supports models generated using external tools, such as NXP eIQ Time Series Studio, and exported as libraries or source code for use within the application.
But how are these technologies integrated into a device via Zephyr?
How AI runs within Zephyr
In a Zephyr application, the inference cycle usually follows a clear sequence:
- The device retrieves data from the sensors selected for each use case.
- The data is prepared through processes such as filtering, normalisation, time-window calculations and feature extraction.
- The model is run using the chosen machine learning technology. This can be done by calling the APIs of the generated libraries, using an SDK or utilising the available internal resources. The result may be a class, a probability, an anomaly score or a regression.
- The app makes a decision. You can set an alarm, send a message, log an event, change the operating mode or combine on-device inference with external services.
Zephyr does not act as a ‘container’ for the model; it coordinates the device’s entire behaviour
This pattern is particularly powerful because Zephyr does not merely act as a ‘container’ for the model. It is the platform that coordinates all aspects of the device’s operation: sensors, connectivity, power, security and business logic.
In addition to the basic inference cycle, it is also important to consider the maintenance of the model once it has been deployed.
If the environmental conditions, the behaviour of the device or the quality of the data captured change, driftmay occur. In other words, the model may no longer accurately represent reality. In such cases, it will be necessary to consider whether they require further training or retraining.
The benefits of Zephyr for Edge AI
One of the reasons why Zephyr is well suited to Edge AI scenarios is that it allows artificial intelligence to be integrated as an integral part of the embedded application, rather than as a standalone component.
Its main advantages include the following:
- A common technological platform. Zephyr enables the integration of sensors, communications, energy management, real-time tasks, security, storage and software updates.
- A modular architecture. It is possible to incorporate external libraries, SDKs or code generated by model training and optimisation tools.
- Portability across different environments. An Edge AI project can start on a development board and subsequently be ported to other hardware.
In addition to these general features, Zephyr offers other specific capabilities for developing embedded AI solutions:
- The first is the use of inference runtimes tailored to microcontrollers, such as TensorFlow Lite within the existing LiteRT ecosystem for embedded devices. This makes it easier to run optimised models directly on the device. Zephyr also provides examples and modules to facilitate integration.
- Energy efficiency is another key aspect. Many IoT devices operate in environments where power consumption must be kept to a minimum. Zephyr offers energy management mechanisms that allow periods of low power consumption to be combined with specific instances when the models are running. The device therefore does not need to remain active at all times: it can wake up, analyse the data, make a decision and return to a state of lower power consumption.
- Finally, Zephyr includes features for maintaining and updating an embedded AI solution. It is possible to securely deploy new versions of the firmware or the model and to incorporate validation and fault-tolerance mechanisms. When the architecture requires greater modularity, LLEXT allows pre-compiled code extensions to be loaded at runtime. However, when using it, factors such as compatibility, available memory and security must be taken into account.
Limitations to bear in mind
Edge AI models must be adapted to the constraints of embedded devices.
Embedded AI should be seen as a complement to other levels of processing
Available memory, computing power and energy consumption are key factors when designing the application and selecting or training the machine learning model.
In these contexts, it is not enough for the model to be accurate. It must also be light and efficient enough to run within the device’s actual capabilities.
The quality of the data captured by the sensors is another critical factor. Signal noise, gaps in the readings, out-of-range values or changes in the environment can directly affect the outcome of the inference. For this reason, before running the model, it is usually necessary to validate, filter and prepare the input data correctly.
Edge AI does not, either, eliminate the need for all external processing. When the model runs on the device, each node typically operates with a local view: its sensors, its immediate surroundings and its operating context. This enables a rapid response and reduces reliance on the cloud, but it limits the detection of collective events and the correlation between different sources of information.
Embedded AI should therefore be seen as a complement to other levels of processing, not as a universal substitute for the cloud. Its value lies in bringing analytical and decision-making capabilities to the point where the data is generated, whilst always remaining within the device’s actual capabilities.
The combination of Zephyr and embedded AI represents, in short, a natural evolution of the IoT: the shift from devices that are merely connected to systems capable of interpreting their environment and acting intelligently.
This article is a follow-up to the piece titled 'Embedded AI: From the Connected IoT to the Smart IoT', written by José Antonio Sánchez Segovia too.