Getting up to speed with new AI tools like TinyML can seem challenging. The process of choosing an appropriate neural network model, then training and and adjusting it for use in an application is not trivial but assistance is available in the form of the increasing number of TinyML resources which are continuously being created. In this article, we consider the approach used by TensorFlow Lite, Edge Impulse, and Fraunhofer AIES to simplify the training and deployment of an AI model on an entry level microcontroller.

Deploying TinyML in Edge Applications

To monitor motor vibration in an industrial application, a developer might start by firstly gathering information about vibrations from accelerometers mounted on many different motors. To ensure that a broad cross section of data is included in the analysis, ideally each motor should be at a different stage in their respective maintenance cycle and the sample should also include a new model which has been subjected to little or no wear and tear. The next step is to select an algorithm to use with the sensor data to help uncover relationships between vibrations signatures and progressive stage of wear and tear. The sensor data is then used to train the algorithm which then becomes a system model that can be used to predict the condition of motors. While ML models are useful, inferencing involves making predictions  based only on available training data and is not an exact science.

Several factors are making edge-based inferencing increasingly popular. Firstly, it removes the requirement for data to be streamed to the cloud thereby reducing latency and bandwidth problems. For the same reason, privacy is also less of a concern. Edge devices also typically use low power microcontrollers (in the order of milliwatts) meaning they can run for long intervals on batteries operation.

Resources Simplify Development and Application of TinyML

As inferencing at the edge becomes more popular, many resources to support this activity have appeared.

TensorFlow Lite for Microcontrollers

TensorFlow, which was created by Google in 2015, is an established machine learning resource. It is an open-source platform for machine learning which offers a comprehensive suite of tools, libraries and shared resources. It enables developers to apply machine learning in scientific, medical, and commercial applications.

In 2017, Google launched TensorFlow Lite for Microcontrollers in response to the growing interest in performing inferencing using low power devices with only tens of kilobytes of available memory. TensorFlow Lite’s core runtime binary library uses only 18 kB of memory in an Arm Cortex M3 and has the ability to run basic neural network models. It does not require an operating system, C or C++ libraries, and has no need for direct memory allocation. The library, which is written in C++ 11, only requires a 32-bit microcontroller.

The models used by TensorFlow Lite for Microcontrollers are optimised versions of those in the full TensorFlow training environment. It is well supported, has been tested on the Arm Cortex-M series and later ported to other MCU architectures such as the Tensilica-based Espressif ESP32 series and the Synopsys ARC processor core series. A detailed guide to implementing TensorFlow Lite for Microcontrollers is available here.

Another set of  resources, TensorFlow Lite, is available for Android, iOS and embedded Linux-based single-board computers such as the Raspberry Pi and the Coral Edge TPU.

Figure 1 illustrates the workflow process using TensorFlow Lite for Microcontrollers.

Edge Impulse

Edge Impulse provides a comprehensive machine learning platform for embedded microcontrollers. The goal of Edge Impulse is to make it possible to bring an embedded ML project from inception to production in the shortest possible time frame and it has been applied in over 20,000 projects around the world, reducing development time to the order of weeks (instead of years). The emphasis is on using embedded devices equipped with various sensors, including audio and vision, and deploying them in large volumes. It is targeted at devices ranging from entry level microcontrollers to higher-end microprocessors and CPUs, including TensorFlow and Keras.

Edge Impulse TinyML can be used in industrial (predictive maintenance), heathcare (body sensing) and logistics (asset tracking) applications.

Edge Impulse have adopted the code of conduct of the Responsible AI Institute which requires that it not be used for criminal, surveillance, or defence purposes.

The model testing feature of Edge Impulse is shown in Figure 3. Here, a STMicroelectronics MCU connected to a digital microphone recognises two words – house and zero. Model testing shows the probability of distinguishing each word from background noise or other words.

Fraunhofer AIfES

The Fraunhofer research institute in Germany has developed a platform-independent library using the C programming language for embedded systems (AIfES). This open-source library is available under a GNU General Public License (GPL) arrangement. It uses the standard library GNU GCC which make it easy to implement and it runs on almost any hardware processing device, from basic 8-bit microcontrollers up to PC CPUs. A version of the library is also available for Arduino developers. AIfES is free for private use but requires a license agreement for commercial applications.

Figure 4 illustrates the operability and compatibility of AIfES with different platforms. It is compatible with and operates in a manner similar to common Python ML frameworks such as TensorFlow, Keras and PyTorch.

Deploying TinyML: Next Steps

In this article, we have presented three machine learning libraries and platforms that can help to accelerate the development of edge based TinyML applications. Learning how to deploy machine learning at the edge may appear challenging, but the growing number of available resources can improve understanding of complex neural networks and the principles of data science.