The increasing automation and autonomy in production is key to addressing global economic and social challenges. While hardware, such as industrial robots, is already well developed, there is still potential to increase the flexibility and user-friendliness of such systems. At the Institute of Assembly Technology and Robotics (match) at Leibniz University Hannover, scientists are working to close this gap – through innovative approaches such as human-robot collaboration and intuitive programming.
Human-Robot Collaboration: Flexibility for SMEs
Industrial robots are already established in many areas, but their rigid programming and high safety requirements often make them unattractive for small and medium-sized enterprises (SMEs). This is where the match focuses on collaborative robots (cobots), which are also interesting for SMEs due to lower acquisition costs and simpler safety precautions.
A central aspect is user-friendly programming, for which SMEs do not need specialized programmers. In programming by demonstration, process experts demonstrate the desired task, which is then converted into a robot program. This “human-in-the-loop” approach uses human cognitive abilities to monitor the correctness of the process and make adjustments if necessary.
AR + 6D Pose Estimation = Intuitive and Flexible Automation
To make programming even more intuitive, the match is researching the use of Augmented Reality (AR). AR systems overlay the real field of vision with virtual content, enabling particularly simple and efficient interaction with the robot. However, for these technologies to reach their full potential, precise 6D pose estimation of objects is required.
6D pose estimation describes the determination of an object’s position and orientation in space. This information is crucial for robots to precisely grasp and place objects. Traditional methods were often limited to specific objects, but modern approaches based on machine learning can also recognize novel objects, provided a 3D model of the object is available.
A breakthrough in this area is foundation models, which are trained on a wide variety of objects and can therefore generalize. To generate the necessary training data, these rely on generative artificial intelligence (AI), which creates and adapts synthetic objects to improve generalizability.
Challenges and Solutions for Industrial Applications
Despite progress, challenges remain, particularly in evaluating methods for industrial use. Current benchmarks for 6D pose estimation are often based on objects that do not meet industry requirements. Metallic, nearly symmetrical, or scale-variant components are particularly underrepresented in test datasets.
To close this gap, scientists at the match have developed a robot-supported setup for automated recording and annotation of test datasets. This uses printed templates for positions on which objects are placed. Through the known transformation between template, robot, and camera, the data can be precisely annotated with the “ground truth”. This approach makes it possible to efficiently create domain-specific test datasets for various industrial applications in the future and to quantitatively evaluate the suitability of 6D pose estimation.
Extrinsic Calibration: Precision for AR Systems
Another research focus is the extrinsic calibration of AR headsets. When AR systems are used to program robots, the positions determined in the AR system must be precisely transferred to the robot’s coordinate system. Here, the match investigates the accuracy of such methods, especially for modern hardware like the Apple Vision Pro, and develops solutions for complete application cases.
Future Vision: Automation for All
The long-term goal of the match is to make automation accessible to companies of all sizes. Through intuitive programming, precise 6D pose estimation, and the use of AR systems, the scientists aim to increase the flexibility and efficiency of production processes. In ten years, robots should be commonplace not only in large industrial companies but also in SMEs – without the need for specialized personnel.
The match is working to make this vision a reality. The combination of human-robot collaboration, machine learning, and augmented reality creates the foundation for more flexible, efficient, and accessible automation.
