The essence of human-robot interaction

Fluently

AI-based technologies are difficult to implement as they require experience and special know-how from operators. The EU project Fluently aims to change this by developing a novel intelligent interface unit. The smart interface will allow machines to interpret speech content and tone more accurately and convert gestures into instructions, while also focusing on improved worker well-being. The project’s solution also includes the Robo-Gym, providing the first environment in Europe where human–robot interactive training can take place.

Fluently

Fluently leverages the latest advancements in AI-driven decision-making process to achieve true social collaboration between humans and machines while matching extremely dynamic manufacturing contexts.

The project results will be:
1) Fluently Smart Interface unit and
2) The Robo-Gym.

The Fluently Smart Interface unit features:
1) interpretation of speech content, speech tone and gestures, automatically translated into robot instructions, making industrial robots accessible to any skill profile;

2) assessment of the operator’s state through a dedicated sensors’ infrastructure that complements a persistent context awareness to enrich an AI-based behavioural framework in charge of triggering the generation of specific robot strategies;

3) modelling products and production changes in a way they could be recognized, interpreted and matched by robots in cooperation with humans. Robots equipped with Fluently will constantly embrace humans’ physical and cognitive loads, but will also learn and build experience with their human teammates to establish a manufacturing practise relying upon quality and wellbeing.

Learn more on The University of Southern Denmark's site >>

Objective

Fluently targets three large scale industrial value chains playing an instrumental role in the present and future manufacturing industry in Europe, that are:

1) lithium cell batteries dismantling and recycling (fully manual);

2) inspection and repairing of aerospace engines (partially automated);

3) laser-based multi-techs for complex metal components manufacturing, from joining and cutting to additive manufacturing and surface functionalization (fully automated in the equipment but strongly dependent upon human process assessment).

Challenges

Challenge 1:

Increase the automation level by 30% by introducing robotics in value chains that are heavily relying upon human-based activities, inference, and decision-making processes.

Target: 5 days training for people with no knowledge in automation to smoothly cooperate with a robot/cobot.

Challenge 2:

Enable zero scraps and production downtimes by reducing the physical and cognitive load of the human operator by 70%.

Target: A disruptive immersive training experience -fully application driven- in the Robo-Gym where robots are trained to build knowledge about the process and personalize their behaviour on the very human operator with whom they will work in the real field.

Challenge 3:

Reduce ramp-up time of 80% in matching production changes in part types, variants, volumes, and mix ratio.

Target: Artificial Intelligence (AI) / Artificial Empathy (AE) distributed platform trained in the Robo-Gym and deployed in the shop floor will enable recognition, classification, inference on new process recipes and strategies, thus enabling a fast reconfiguration of the production approach involving both people and robots

Use case 1 MEM - dismantling and recycling process

As one of the leading European companies in the recycling of lead-acid and Li-Ion high power batteries, MEM’s recycling process concerns the dismantling of the battery through the combination of complex, articulated, and unique mechanical processes chosen on the base of the type of battery pack. Due to the range of products together with the high dexterity requirements of the involved operations, the disassembly process is difficult to be automated and is entirely carried out by human operators. The exposure to toxic materials and the serious risks of the explosion during the recycling process push the battery industry to avoid recycling and to shred/burn batteries

Baseline: production planning driven by tooling distribution on manual dismantling stations; dispatching on manual stations driven by part types’ similarities. The average assembling/testing for a battery made of 6000 cells is 300 hours. The average dismantling time for the same battery configuration is 450 hours.

Fluently Automation targets: A) fully automate the tasks hazardous/dangerous for people (-70% cognitive load, Challenge 2); B) setting up cobotic cells for the other tasks demanding for: 1- human experience supporting product-driven process chains and 2- robots executing repetitive manipulation and trimming tasks; C) in case of product changes, robots should autonomously identify processing strategies based on acquired previous

Use case 2 CIM 4.0 - inspection and maintenance process

The turbojet engines are periodically subject to inspection and repairing processes to guarantee acceptance criteria and ensure high safety standards. One of the main inspection activities regards the blades of the first stage turbine. The variety of defects is large and demands multiple inspection technologies and also processing strategies.

Manufacturing challenges: extremely heterogeneous parts as a result of very diversified defects in terms of types and topological distribution.

Baseline: The inspection and repairing tasks are all manual and frequently iterative; the manufacturing recipes are typically resulting from the human operator experience who can only partially rely upon tools and devices.

Time to configure a process chain and collect tooling: on average 45 minutes for a single blade; the average time to execute the repairing is 2.5 hours.

Fluently Automation targets: A) Generalize inspection/repairing strategies to gain production flexibility by allocating very simple tasks to robots (teaching by imitation); B) complex shape parts demanding for high dexterity can be co-processed by people and robots (boost physical interaction, Challenge 1); C) structure human-based decision-making process to make it readable by robots.

Fluently Production targets: fully automated inspection by cobots run in parallel to specific repairing tasks (manual if very complex); overall time reduction for process recipe design and repairing execution is 65%.

Use case 3 Prima Additive - damaged metal part Inspection and repairing

Prima Additive relies upon a robotic work cells that can respond to different industrial needs by providing customized components manufactured through multiple laser technologies. The production approach consists of a complex and articulated sequence of operations (additive manufacturing, welding, surface polishing, cutting). The manufacturing chain, the process parameters the process strategies are strictly related to the part to be realized and driven by the experience of the operator who changes the part program for every product.

Manufacturing challenges: Oil&Gas and Aerospace parts present complex shapes and multiple techs in sequence (all custom processes). For specific part families, PA starts from a basic identical building block on the top of which solders/deposits/screws modular components.

Baseline: fully custom process chain programming for manufacturing and assembling. The average programming time is 21 hours for a single part.

Fluently Automation targets: A) systemize the process recipes in a set of modular part programs; B) avoid coding but triggering the part programs by voice; C) visual access to the robotic cell through cameras to run an intermediate quality inspection of parts and ask the robot to adapt the process chain (boost remote interaction).

Fluently Production targets: automatic generation of process chains and instantiation in part program modules; voice-based programming to interconnect part program modules. Reduction by 60% of process programming time to generate a new part program.

Funding

Learn more here >>

Partners

The project is supported by the European Union's Horizon 2020 programme and runs from 2023 to 2026.

Fluently is run by the University of Southern Denmark (SDU) in collaboration with Odense Robotics and 20 European partners, working together in a consortium.

See all project participants here >>