In mid-September, a hackathon took place at Igus in Cologne. The goal of the hackathon was to create an automated method for packing pizzas using an AI-based Large Language Model (LLM). The event was held in collaboration with the Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS, the University of Bonn, the Lamarr Institute and the Fraunhofer Institute for Material Flow and Logistics IML.

During the hackathon, three teams worked on a real-world use case in which a ReBeL robot arm was placed in front of a conveyor belt with various pizza products. Using a webcam and an AI-based segmentation system (Segment Anything Model, SAM), the systems recognized the various products on the conveyor belt and identified their position. The language model assigned these objects to the desired products based on the natural language instructions. The robot then placed the products in the boxes by following the instructions of the language model.

The hackathon showed that many more automation possibilities are conceivable through the use of LLMs and robotics, such as packing several boxes at the same time, sorting products according to special instructions or checking ingredients for allergens using camera systems. The successful collaboration between Igus, Fraunhofer IAIS, the Lamarr Institute of the University of Bonn and Fraunhofer IML underlines the potential of these technologies for industrial automation.