We've made Machine Learning on Arduino easy again

Anyone who has had the chance to experiment with the Arduino Nano 33 BLE board, following the classic path proposed in the official Arduino documentation, has very likely been intrigued by the possibility of using Machine Learning.

Machine Learning was indeed one of the distinctive features that marked the launch of this board, which coincided with Google’s release of the TensorFlow Lite Micro library.

Arduino Nano 33 BLE was the device chosen by Google’s team in 2018 to test the first version of the new library, and the Arduino version was the first example of porting to a specific microcontroller platform.

The project was carried out by a team that included software engineers from Google and engineers from Arduino.

That experience gave rise to an interesting workshop that was later used as the basis for the tutorial “Get Started With Machine Learning on Arduino”, still present in the official board documentation and visible at this link.

In 2019 we were among the first Italian training centres to include this tutorial in our courses on Tiny Machine Learning, since it nicely showcased the potential of this emerging technology.

Although the official tutorial’s update date is very recent (April 2024 as we write this post), anyone who has tried to follow the steps and use the published sketches over the past two years will certainly have noticed that compilation of the decisive sketch — the one that runs TensorFlow Lite on Arduino — fails with a series of errors.

What happened?

Searching the web doesn’t help much.

All the video tutorials are dated between 2019 and 2020, the period when everything worked perfectly.

From some threads on the Arduino forum and on GitHub issues, it becomes clear that the problem is well-known and that the cause lies in the misalignment between the library’s versions — no longer present in the official Arduino registry, but on the TensorFlow GitHub repository — and the sketch code shown in the tutorial.

Some answers in the Arduino forum threads are blunt and more or less state that, with the advent of ChatGPT-style AI, playing with Tiny Machine Learning on a limited platform like a microcontroller is by now a useless and outdated exercise.

We absolutely disagree, and that’s why, in preparing our new Arduino course, we decided to include Tiny Machine Learning among the topics, and we got our hands on TensorFlow Lite Micro to generate a new library — updated to June 2024 — capable of bringing direct experimentation with this technology back within everyone’s reach, especially of those who are not C++ experts on TensorFlow.

In this article we’ll explain why we believe Tiny Machine Learning is a topic of great current interest, especially in the era of ChatGPT, and what had happened to the original Arduino TensorFlow Lite library.

If you’re looking for a solution to the malfunction of the official Arduino tutorial on Machine Learning for the Nano 33 BLE board, you’re in the right place.

After our considerations on the importance of Tiny Machine Learning, we’ll explain how to write correct and working sketches that use TensorFlow Lite Micro on Arduino.

What is Tiny Machine Learning, and why does it matter?

Machine Learning is one of the research areas within the broader field of Artificial Intelligence.

From a practical, engineering standpoint, it’s a software development process alternative to the traditional one, based on defining explicit algorithms made of sequences of instructions, loops and if … then … else branches.

There are situations in which the traditional process is not applicable.

To grasp the concept, let’s look at some simple examples.

Imagine wanting to build a program that, given the lengths of the three sides of a triangle as input, outputs the category to which the triangle belongs: right triangle or non-right triangle.

It’s a trivial classification problem, easily solvable. The fastest path is to use the Pythagorean theorem and write a simple algorithm that checks whether the sum of the squares of the shorter sides equals the square of the longest side. If this property holds, the triangle is right-angled; if not, it isn’t.

In this case we were able to find an explicit algorithm that solved the classification problem.

Now consider a different problem. Imagine a robotic arm fitted with an IMU sensor (Inertial Measurement Unit) capable of detecting linear and angular acceleration values during the arm’s movement.

Depending on the load, operating conditions and other environmental factors, the sequences of values read by the sensor can differ greatly. Suppose we collect reading sequences in various operating conditions, both when the arm functions correctly and when it shows operational anomalies — e.g. due to motor overheating or actual faults.

The readings made in the time interval preceding the anomaly probably show patterns that can be used to predict the imminent fault.

By looking at the data sequences, can we identify the mathematical rules that allow us to detect the anomalous situation?

Most likely not.

When analysing situations of this kind, the graphical representation of the value sequences often appears clearly different even from a simple visual analysis, but moving from visual recognition of the “pattern” of the value curves to the formulation of an algorithmic rule is not easy and often impossible.

Just as, by analysing the binary sequence of a jpg file, it’s impossible to define a specific algorithm to identify the objects in the image, in the case of analysing value sequences from complex sensors like an IMU it’s usually impossible to define a specific algorithm to identify situations of potential anomaly.

In these cases Machine Learning offers an elegant and effective solution.

To proceed via this technique, the software development process is split into two phases.

In the first phase we develop software to collect and store data sequences and let it run for a sufficiently long period to capture both normal and anomalous situations.

The data thus collected is “labelled” — manually or automatically — to identify the samples that represent normal situations and those that represent alarm situations.

In the second phase we build software that implements a machine learning model, e.g. a convolutional neural network.

These “models” are generalised algorithms characterised by a high number of parameters.

For each input value, the result computed at the output depends on the values assigned to the parameters.

The “model” comes with a suitable training algorithm (also known as a model fitting algorithm) that, through an iterative procedure and the application of statistically derived functions and methods, processes the example data — labelled with the known classifications — to determine the values to assign to the parameters so that the “model” can correctly classify the input.

The final result of this development process is a software algorithm capable of solving our classification problem.

The software designer cannot determine a priori a precise algorithm for solving the problem, but through the learning mechanism, or model fitting, the system is able to automatically determine the correct parameters, based on a known dataset called the training dataset.

This is the essence of Machine Learning — a very powerful method that lets us solve data-analysis and classification problems that aren’t tractable by traditional means.

The example of predictive maintenance of the robotic arm indicates how Machine Learning can be a very valid tool in the field of digital control systems. How does all this get implemented in a context where data is collected and analysed by microcontroller devices?

As we’ve seen, in Machine Learning-based systems there are two distinct categories of algorithms: the learning or model fitting algorithms, and the prediction or classification algorithms, in which the input is processed through a mathematical model whose parameters have been pre-determined via learning.

The learning algorithms require long times and substantial computing resources. For this reason they’re usually run on dedicated computers equipped with multi-core CPUs and specialised GPU coprocessors (the same ones used for graphics) or TPUs (coprocessors specialised in the tensor computation typical of machine learning).

The classification algorithms, in cases similar to the examples we mentioned, usually require much more limited resources and computing power, and in most cases can also be run on board a modern 32-bit microcontroller.

The technology that allows the prediction algorithms of Machine Learning models and Neural Networks to be processed on limited devices like smartphones, single-board computers and microcontroller boards is called Tiny Machine Learning.

A Tiny Machine Learning-based solution usually involves a data-collection phase, a model-fitting phase to be carried out on dedicated computers or cloud computing platforms, and the installation and execution of the trained model on microcontroller or smart devices.

In 2018 Google released an open-source C++ library called TensorFlow Lite Micro, which made it easy and immediate to port a neural network model — built with the powerful and widely used TensorFlow platform — onto microcontrollers.

The initial enthusiasm from firmware developers and the blossoming of tutorials and pilot projects in open-source communities — like Arduino’s — were overshadowed by the recent advent of the new Generative AIs, which have partly modified the approach to general Machine Learning technologies.

The power of systems like those presented by OpenAI and its competitors is such that many data-analysis problems can be put directly to tools like ChatGPT, getting immediate, correct answers without having to build one’s own neural network model and train it appropriately.

Does it still make sense to implement on board a microcontroller a neural-network-based algorithm, when it would be easier and quicker to send the sensor data to a powerful AI system reachable in the cloud and invoked via API?

A simple web search for Artificial Intelligence combined with Arduino today leads to a number of pages describing — more or less amateur — projects in which the device uses an Internet connection to draw on an external AI service.

While this architecture is easy to build and much more immediate in an amateur or artisanal context, the solution of sending raw data collected from remote sensors to the cloud isn’t efficient nor economically convenient in an industrial context.

The real-time collection of continuous data flows from hundreds or thousands of remote sensors, and the corresponding forwarding to the Internet, requires adequate bandwidth and the consequent need for powerful transmission devices to overcome the interference and disturbances usually present in industrial plants.

It’s no coincidence that, within Industrial Internet of Things, Edge Computing architectures are becoming more and more widespread — architectures in which data processing happens on board the sensor devices, or on intermediate systems near them, forwarding only summary data or alarms to central systems.

Tiny Machine Learning is a solution compliant with the Edge Computing architecture, and for this reason its importance will grow in the immediate future.

At an industrial level, R&D in this area has never stopped, and the TensorFlow Lite Micro library itself is currently localised for all the main microcontroller architectures on the market, from ARM Cortex M4 through to ESP32 S3.

The Arduino world, however, is heavily affected by media trends and hype, which obviously influence the large community of enthusiasts — which also includes many students and researchers.

Tiny Machine Learning on Arduino seemed to be a topic past its prime, and that’s a pity, because historically this community has produced creative and innovative ideas that have managed to influence industrial research.

In reality, in recent months, interest seems to be picking up again, but the obvious obsolescence of the resources made available by official documentation and by the community quickly disillusions the less technical profiles who try to experiment with this technology.

For this reason we’ve decided to pick up the TensorFlow Lite Micro on Arduino porting project again and make resources and educational material available again that allow creatives and enthusiasts to bring Tiny Machine Learning into their projects in an intuitive way and in the typical Arduino style.

The TensorFlow Lite library for Arduino

The official C++ TensorFlow Lite Micro library is developed and maintained by Google as a general collection of C++ source files. The files are available under an open-source licence in a dedicated GitHub repository, separate from the general TensorFlow source code repository. The files were initially contained within the Lite branch of the TensorFlow repository, but in 2022 the code was moved into a dedicated repository.

Anyone used to contributing to fast-evolving open-source projects knows the apparently chaotic and often redundant structure of these GitHub archives. The essential, most significant files are mixed into an intricate tree of directories and sub-directories, alongside example files, experimental code and files of dubious utility. External documentation is practically non-existent and to find your way around you have to read the source code and study its structure and comments.

Google’s idea is that the actual porting of the library to a specific platform — e.g. mbed_nano Arduino or Raspberry Pi — is to be carried out by the team interested in doing it. To provide guidelines for this porting process, Google’s team chose to use the Arduino Nano 33 BLE board as an example. A version of the library adapted for and working on this Arduino board was therefore created and stored in this repository under the name TensorFlow Lite Micro Arduino Example. The word “example” in the repository name refers to the fact that this project was meant to serve as an example for anyone who wanted to adapt the library to their own platform.

Engineers from Arduino took part in the project, and the library was registered in the official Arduino library register with the name Arduino_TensorFlowLite, making it automatically installable through the Arduino IDE. The library’s GitHub repository was kept in sync — through GitHub’s automatic features — with the generic, non-specialised C++ source library, so that any changes made to the original source files would be replicated to the files in the derived repository.

However, as always happens in this kind of project, since porting operations on complex platforms like microcontrollers require a certain amount of manual work, this automatic alignment started to fail and was halted in 2022. After a few months the maintainers of the Arduino registry were asked to remove the library. This means that, since then, the library is no longer aligned with the latest changes in the TensorFlow Lite Micro system, and is no longer available from the Arduino IDE for automatic installation.

Making things worse for Arduino devotees, the automatic alignments performed between 2019 and 2022 never updated the version number, which is still 2.0.4-ALPHA. The “Get Started With Machine Learning on Arduino” tutorials on the Arduino site, in the official documentation of the Arduino Nano 33 BLE Sense board, had been kept aligned up to version 2.0.4-ALPHA. After the library was removed from the registry, the tutorial text was updated specifying that the library must be downloaded from the “official” GitHub repository and installed as a .zip file. Pity that the version of the library on the indicated repository is no longer the original 2.4.0-ALPHA, but some more advanced version — although already obsolete and stuck in 2022.

For this reason, anyone who tries to follow the tutorial instructions ends their experiment flooded with compilation errors.

If you want to try to travel back in time and experience the tutorial as if you were in 2019, you can get hold of the real 2.0.4-ALPHA version by downloading it from this link. By installing the original 2019 version — and not the one pointed to from the link inside the tutorial — the example sketches will compile correctly and the experiment will work.

If, instead, you’d like to start experimenting with a recent version of TensorFlow Lite Micro on Arduino, you can comfortably download our new library Chirale_TensorFlowLite directly from the IDE. The library comes with the example sketch “hello_world” that shows how to use TensorFlow Lite within a sketch. We generated a new project from the up-to-date master files and produced a new library in the style of the original Arduino_TensorFlowLite.

On Tiny Machine Learning we’ve made the last lesson of our new Arduino Unchained course available as a separately purchasable course. You can read all the details on the Accademia delle Arti Numeriche platform.

Tiny Machine Learning on Arduino course

The style of the Arduino_TensorFlowLite library — and therefore of our library — is not what Arduino enthusiasts are used to.

In the code there’s a very technical C++ syntax and pointers and other data types are used that are usually avoided in Arduino sketches.

Our project has just begun, and the next steps will be precisely to make this library intuitive and Arduino-standard-compliant, in order to facilitate access to this important technology for the entire community of creatives working with Arduino.