Learning to juggle is pretty difficult even for a human being to master. When first you try, the balls fly in every direction but where they should. Juggling is a complex motor skill that requires a high degree of accuracy in terms of arm movements, catching and watching the balls. Teaching this to a robot is even harder than learning it yourself.

Michael Lutter, a PhD student from the Intelligent Autonomous Systems Group at TU Darmstadt had to pick up around 500 balls from the floor of the laboratory before the robotic arm that he had programmed was able to launch them safely through the air. The system is now able to do it for up to 33 minutes in one go – and hence for around 4,500 throws. That is a difficult ask, even for circus artistes with many years of experience.

Lutter, and Kai Ploeger, whose outstanding Master’s thesis on the subject was supervised by the research associate, are not, of course, hoping to take the circus ring by storm. Their study, “High Acceleration Reinforcement Learning for Real-World Juggling with Binary Rewards” (opens in new tab) examines a complex problem in robotics: the application of machine learning methods under high mechanical acceleration.

“It is a challenge to develop learning robots that are able to react very quickly without damaging the mechanical systems in the process,” says Lutter. An incorrectly learned movement can destroy the joints – or in the worst case scenario, even cause parts of the robotic arm to fly across the laboratory. Nevertheless, quick and precise reactions are a prerequisite for a robot to be able to prove its worth outside of the laboratory in people’s everyday lives – for safety reasons, but also for it to be able to move more flexibly.

A paper has been produced from the Master's thesis and accompanying research that will be presented at the highly regarded “Conference on Robot Learning”, which this year will be held online. Lutter hit on the juggling idea two years ago, when the juggling fan and Computer Science student, Kai Ploeger, came into his office and asked him straightforwardly if it might not be possible to teach a robot to juggle. He thought it was a great idea and the new duo got cracking right away. They replaced the gripper on the Barrett WAM robot system with a conventional household funnel, which was ideal for safely catching balls.

The robot next set about learning, namely with one arm and two balls – a task more difficult than if it had three balls but two arms available to it. “The robot learns in a similar way to a human being,” says Lutter. To begin with, he defined a rough movement for the system by setting various waypoints, which the arm used to orient itself. After this the robot was left to its own devices. It had to gradually work out which movement worked best for it.

Lutter employed a machine learning method known as “reinforcement learning” that enabled the robot to learn, through a process of reward for every successful action that the algorithm mapped. With every reward, the system learns when it is on the right track. “Juggling is an interesting robotics problem for us, because the robot has to carry out fast movements with a very high degree of precision,” says Lutter. The arm – or manipulator – initially throws the first ball in the air, then it needs to move quickly to the right, catch the second ball and throw it back up in the air. It then moves back to the left – all with perfect timing. While following the circular juggling pattern the robot also has to find the perfect angle for throwing the ball.

However, until now a task of this kind was usually hindered by the limits of technology, first and foremost due to the large amount of acceleration required. “As a general rule, industrial robots cannot accelerate as suddenly as needed for juggling, due to the gear units that are used,” says Lutter. Moreover, acceleration movements can cause a large amount of wear and tear. “Juggling is therefore an ideal problem for testing the limits of existing learning algorithms on current physical systems,” says Lutter.

The complexity of the task is also demonstrated by the fact that the researchers cannot simply transfer what has been learned once on to another robot of the same construction. Minimal, unremarkable deviations in the movement ensure that what has been learned no longer works. Lutter says, “Now we are working on additional challenges such as how two arms that are independent of one another can learn to juggle together.”

The first videos of these mechanical jugglers are already a hit on social media.