Researchers at the Institute of Electronics and Computer Science (EDI) have developed an artificial intelligence-based robotic system that independently detects bottles, determines how to grasp them, tracks a moving target and places the bottle into a conveyor slot. Latvian cosmetics manufacturer Madara Cosmetics has also expressed interest in the technology, assessing its potential from the perspective of real-world production.
In many production facilities, there are still tasks that require people to repeat the same technical action for hours – picking up a product, placing it, sorting it or passing it on to the next stage of production. Such tasks can be monotonous and physically tiring for people, but automating them with robots is not always easy. Production is changing: the same production line may need to handle products of different sizes and shapes, while packaging, production volumes and process configurations also change. A traditional robot is very good at repeating a single precisely programmed movement, but each product change may require new programming and system adjustments. Therefore, one of the challenges of modern robotics is to create systems that can adapt to change – taking over repetitive and technically precise tasks while allowing people to focus more on process supervision, quality and tasks that require experience and decision-making. This is precisely the kind of more flexible automation that EDI researchers are striving to develop.
From the outside, the task seems almost too simple. Picking up a bottle from a box and placing it in its designated position on a conveyor. A person does this almost without thinking.
For a Robot, the Situation Is Completely Different
A bottle in a box may be positioned at a different angle than the previous one. There may be several other bottles next to it. Their shape, size, colour and weight vary. The place where the bottle needs to be inserted is not stationary either – the slots move along the conveyor, may stop, start moving again or slightly change position.
Moreover, the Margin of Error Is Just a Few Millimetres
This is precisely the kind of problem that a team of researchers at the Institute of Electronics and Computer Science (EDI), led by Dr. sc. comp. Kārlis Freivalds, has been addressing in recent years, developing a reconfigurable, artificial intelligence-based robotic conveyor feeding system within the European research project AIMS5.0. EDI is the lead partner in this demonstrator, also collaborating with Harokopio University and Budapest University of Technology and Economics.
The idea is relatively simple, but technologically complex: to create a robot that does not require everything to be developed from scratch for each new bottle and production situation.
A Robot That Not Only Moves but Also Sees
In traditional industrial automation, a robot is exceptionally precise when the environment remains unchanged – the object is always in its designated position, the conveyor operates at a fixed speed, and movements can be programmed in advance.
However, modern manufacturing increasingly requires smaller production batches, more frequent packaging changes and the ability to use a single production line for multiple products. In such situations, a robot that can respond to what it “sees” at any given moment has an advantage over one that simply repeats a pre-programmed movement.
In the EDI demonstrator, this vision is provided by cameras and several computer vision algorithms. First, the system detects bottles in the image, separates the selected bottle from the surrounding background and uses depth information to create a spatial, or three-dimensional, representation of it. The bottle's position and orientation in space are then determined so that the robot can calculate a suitable grip.
Next, the robotic arm grasps the bottle and moves it towards the conveyor. But that alone is not enough. As the robot approaches the conveyor, the system must simultaneously identify an empty bottle slot and track its position. The developed vision algorithm can distinguish between empty and occupied slots, while their positions are repeatedly determined from successive camera images. This is important because the slots are not rigidly attached to the conveyor – they can move, stop and resume movement independently of the conveyor's theoretical speed.
When Two Millimetres Make a Big Difference
The final stage of the movement – inserting the bottle into the slot – is one of the most challenging. In the tested configuration, the bottle has a diameter of approximately 32 millimetres, while the slot measures 36 millimetres. This means that there is a clearance of only about two millimetres on each side. If the camera or robot determines the target position with an error of just a few millimetres, the bottle can no longer fit freely into the slot. This is precisely why it is not enough for the robot to look at the conveyor once, calculate a trajectory and then execute it “blindly”. It must be able to react while moving.
In EDI's solution, classical robotics methods are combined with machine learning. To pick up the bottle, the system uses pre-calculated, collision-aware motion planning, while in the final stage it switches to adaptive control. A neural network controller receives information about the robot's state and the position of the target slot and continuously adjusts the movement.
First, It Learns in the Virtual World
One of the methods used by EDI researchers is so-called differentiable physics. Put simply, a computer creates a physical model of the robot and its surrounding environment, allowing the algorithm to “try out” movements in a virtual environment. If the bottle does not end up in the correct position or its angle is not appropriate, the system can mathematically calculate how the preceding movements should have been adjusted. In this way, the neural network gradually learns how the robot should act in different situations.
Training involves both scenarios with a stationary target in different positions and situations in which the slot moves and stops at a location previously unknown to the robot. The advantage of this approach is not just speed. Some experiments can be carried out virtually, without stopping the factory production line or risking damage to the robot or products.
However, a simulation is not a perfect replica of the real world. Friction, camera errors, mechanical tolerances and even a tiny discrepancy between the virtual and physical environments can affect the result. Therefore, one of the researchers' most important tasks is addressing the so-called sim-to-real problem – ensuring that movements learned in simulation work just as reliably with a real robot.
In the EDI demonstrator, every command generated by the neural network is additionally checked by a safety layer before being sent to the physical UR5e robot. This layer controls the robot's movement speed and acceleration and checks whether the planned movement poses a collision risk. If a dangerous situation is detected, the command is not executed. In this way, the solution trained in simulation is supplemented with the safety mechanisms necessary for operating a real robot in a real production environment.
Tasks Change, but There Is No Need to Build a New System
However, perhaps the most important word in the name of the solution developed by EDI is not “robot” or “artificial intelligence”, but “reconfigurable”.
In manufacturing, products do not remain unchanged for years. New packaging is introduced, and bottle shapes, sizes or conveyor layouts change. If every such change requires new specialised equipment and extensive programming, automation can become economically unviable, particularly for smaller production batches.
EDI's architecture has therefore been designed to be modular. Object recognition, pose estimation, grasp planning, slot detection and robot control operate as separate components. When one aspect of a product changes, the relevant part of the system can be reconfigured without rebuilding the entire solution.
For example, in tests, the slot detection algorithm required less than 15 milliseconds to process a single image and achieved an overall detection accuracy of 98.7%. Adapting the system to a different slot type or camera angle, including image preparation and additional model training, took approximately three hours in the experiment.
This is not yet a universal robot that can be taken to any factory and begin production within five minutes. However, it demonstrates a direction – moving from rigidly programmed automation towards systems that can be adapted significantly faster. An additional safety layer checks commands sent to the physical UR5e robot. It controls the robot's speed and acceleration and checks for potential collisions.
“The value of the solution lies not in its ability to repeat a single movement perfectly, but in enabling the robot to adapt to a changing situation.”
From the Laboratory to Cosmetics Manufacturing
To ensure that the researchers' technology does not become a solution to a problem that exists only in the laboratory, dialogue with companies that could use such systems is essential. On 13 August, a representative of Madara Cosmetics visited EDI to see the demonstrator in person. During the visit, the researchers demonstrated how the system works and discussed its potential applications in real production processes. The company's representative praised the work carried out and shared practical observations on using the solution in an industrial environment. Potential opportunities for future cooperation and adapting the technology to specific production needs were also discussed.
The cosmetics industry provides a particularly good illustration of the need for flexible automation. A single manufacturer may use many different bottles, containers and packaging shapes, while its product range changes over time. In such an environment, a robot that can adapt to new packaging with relatively minor changes to its software and object models may be more valuable than a very fast automated machine designed for just one product.
Industry 5.0 – Not a Robot Replacing a Human, but More Flexible Manufacturing
The EDI demonstrator was developed as part of the international AIMS5.0 – Artificial Intelligence in Manufacturing leading to Sustainability and Industry 5.0 project. Its aim is to promote the use of artificial intelligence in European manufacturing while increasing its sustainability, flexibility and competitiveness.
EDI's task within the project is to develop a modular and reconfigurable robotic system capable of handling bottles of different sizes, weights and types, with individual technologies that can also be applied in other environments.
This is also part of the broader story of next-generation automation. Robots in factories are nothing new. What is new is that they increasingly no longer need to operate in a world where everything must be precisely positioned and programmed in advance.
A camera allows the robot to see. Artificial intelligence allows it to interpret what it sees. Simulation allows it to learn. And adaptive control enables it to adjust its actions when reality turns out to be slightly different from what was planned.