Wageningen University & Research Agro Food Robotics takes up challenge: making salad with smart robots'

Having a robot pick a bunch of grapes requires a lot of technology but having a robot make a complete, ready-to-eat salad can be even more challenging. Researchers in the Wageningen University & Research Agro Food Robotics program are facing this challenge head-on in the coming five years.

Typical main-course salads you can buy in a supermarket contain lettuce, raw vegetables, pasta, potatoes, fruit, nuts, eggs, cheese, meat and fish. To date, there is no option to automatically gather and package all these different products without the input of human labor. WUR experts in computer vision, robotics and AI want to change that in the coming five years in collaboration with industrial partners. Successful development of such a machine would save a lot of line workers, which, in turn, saves costs in an increasingly shallow labor pool.

To automatically prepare and package a main-course salad, robots must perform all tasks performed when single products are packaged. Cameras take pictures for advanced computer programs that learn to recognise products. Then, the same software is used to drive robotic arms that remove products rapidly without damaging them from the harvest crates and deposit them in custom packaging. The packaging is then sealed, and the product information is printed on it.

The technology that the researchers will develop further is built upon the results from the ‘Pick’nPack’ project led by WUR from 2011-2016. "At the time, producers were not exactly lining up to obtain the Pick’nPack technology, but that is beginning to change," said Paul Goethals, Business Development Manager in the WUR Agro Food Robotics program. Not only has the shortage of labour increased the demand, but the technology has also been developed further and is now cheaper, says Goethals. This is mainly due to the advancements in Artificial Intelligence (AI). "AI is making applications such as this more relevant for businesses."

Artificial intelligence helps robots identify and select the products directly in the harvest crates. "Before, we would have to feed the software with characteristics with which to analyze the camera images. Now, we use machine learning with training data. This serves as input for the software to autonomously recognize particular properties of vegetables," Goethals explained. "That then helps the robots recognize and grab the products directly from the harvest crates, without humans first having to deposit the goods on the production line. The stacked products are often easily bruised." This means that the robots must be very precise.

The ultimate goal is a flexible robot. After all, cucumbers are different from grapes. That requires different 'grippers' or suction cups to prevent soft fruit from being damaged. The robot should select the appropriate device and activate the matching computer program for the product in the crate. With the increasing variation in products – consider, for example, the many different types of tomatoes – a machine capable of rapidly switching saves time, money and labor. Moreover, different products are sometimes placed in the same package. "In the future, we will be able to package mixed vegetables for soup with this technology."

Goethals and his colleagues use the latest tools for artificial intelligence. "A sensor in the gripper is also interesting for quality control purposes. That makes it easy to instantly assess whether anything is wrong with the product. That information then enables the robot to sort products into various quality categories or a residue stream used for a different purpose." The data thus gathered makes it easier to retrace products back to their origin and print the product information on the packaging. Even cleaning the machine can be made ‘smarter’ using sensors that continuously measure how much dirt accumulates and that only clean the machine from within when needed.

In the coming five years, Agro Food Robotics researchers and industry partners will merge their work and recent developments to design one flexible state-of-the-art production robot for main-course salads. This new knowledge will then be shared with other businesses in the overarching Nxt Gen HighTech program. "With Pick’nPack, we were ahead of the developments in the field, but now, we can develop our prototypes and knowledge further and apply them. The time and technology are ripe."