Smart Roberta - training Artificial Neural Networks yourself

In the project, the software framework AIfES (Artificial Intelligence for Embedded Systems) of the Fraunhofer IMS is used as a basis and further developed. AIfES is provided as an open-source software library for different models of the widely used "Arduino" platform and makes it possible to train and execute a ANN directly on a small embedded system.

As a user interface, the Fraunhofer experts extend the graphical programming interface of the Open Roberta platform with blocks for programming the KNN. They provide blocks to support the training and to use the trained network afterwards. Users can use the Open Roberta platform to build a full ANN application with a single Arduino that uses data from the hardware's own sensors to train and then use the ANN.

A gesture control system serves as a use case for the feasibility. The desired behavior is learned directly on the embedded system using a ANN. An "Arduino Nano 33 BLE Sense" development board, which has an integrated accelerometer, is used as the reference hardware platform. Training directly in the system is considered a particular innovation, as this is difficult to achieve with comparable systems. A feature extraction system specially developed for this purpose makes it possible: It extracts only the most necessary features from the raw signal of the acceleration sensor. The features of a gesture are forwarded as input to a ANN, which then enables the classification of the gesture. AIfES dynamically creates a suitable ANN at runtime. The network size depends on the number of gestures to be recognized, where each gesture corresponds to one output neuron of the ANN. The learned gestures can be stored permanently for later retrieval.

The visual approach of the Open Roberta platform allows, for example, teachers to introduce their students to computer science through the fascinating topic of robotics, especially with the help of drivable robots. In this field, ANN is particularly interested in reinforcement learning, a subfield of machine learning in which the robot interacts with its environment and learns independently based on rewards. Therefore, AIfES will be extended to include this model in a second step.

For the target group of Open Roberta, training materials will be developed that start at a simple level but can be increased in complexity in order to introduce young people to machine learning in a playful way using example applications. A demonstrator is to solve a task in a self-optimizing manner based on reinforcement learning. Users should be able to observe how the system learns and is able to master the task better and better. Preferably, a demonstrator is used that is based on a mobile robot, e.g., a robot that learns to balance on two wheels, similar to a "Segway".

The idea of reinforcement learning is not admittedly overly complex. However, applying it to the continuous surface on which the robot moves is quite mathematically challenging. A particular challenge of the project is to prepare material didactically in such a way that it is understandable for the target group of Open Roberta and encourages them to conduct their own experiments.

An external partner is to support the marketing of the results. Preliminary discussions have already taken place, and the final demonstrator will be coordinated with the partner. The overall goal is to develop a starter kit based on Arduino components that can be used for training purposes. The Fraunhofer Institutes IMS and IAIS are to share in the proceeds. Sales are to be handled by the partner.

The project aims to make the world of ANN understandable to pupils. The focus is on own experiments with small embedded systems using the software framework AIfES, without access to powerful external systems. Access to AIfES will be realized by graphical programming using the Open Roberta platform. We hope to have strong effects on the educational sector.