How robots and AI outsmart humans to restore ancient Roman masterpieces

What if robots are the best at restoring ancient works of art to their former glory?

This is the question posed by a team of Italian researchers in Mestre, at the gates of Venice, as part of the REPAIR project.

The team collects broken frescoes and other artifacts from the ruins of Pompeii and attempts to restore them using an intelligent robotic system.

The fragments used in the experiment come from the frescoes in the Schola Armaturarum, destroyed by the building collapse in 2010.

The other frescoes were originally on the ceiling and walls of the House of Painters at Work in the Insula des Casti Amanti which was damaged during the Pompeii eruption in AD 79 and then destroyed by bombardment of World War II.

“One of the objectives of this project is to process very large frescoes. I mean that there is no need for our system to process small frescoes,” explained Marcello Pelillo, coordinator of the RePAIR project and professor of computer science at the ‘Ca’ Foscari University of Venice. .

“If we have a mural that is broken and only has dozens or maybe hundreds of pieces, maybe we can fix it manually so that we don’t need a robot, we don’t need artificial intelligence and stuff like that, so in this project we want to focus on very large frescoes, frescoes that no human team can really solve, can not really reconstruct, ”he added.

As part of the experiment, researchers from the REPAIR project will build a robot that uses hyperspectral technology to scan each fragment of the fresco and analyze its texture in three dimensions.

Hyperspectral imaging is a way to process information from the entire electromagnetic spectrum beyond what the human eye can see. This information can be used to identify materials and processes.

“We will use hyperspectral technology to detect traces of pigments that are no longer visible to the human eye, and these can, possibly, support our search for the right part to match the other part”, Arianna Travigle, coordinator of the IIT Center for Cultural Heritage Technology, said.

According to Travigle, the biggest challenge of the project is the poor state of conservation of the frescoes.

“There might be parts of the design that are on the fresco that are not easily visible,” Travigle explained.

“So we’re going to use this kind of technology to investigate what might have been there, before it gets washed away by temperature, or rain, or other factors that really change the structure of the rooms over time. “.

The plan is for thousands of fragments, like small pieces of a puzzle, to be rearranged using robotics.

The robot will be equipped with mechanical arms capable of scanning the fragments, recognizing them thanks to a 3D scanning system and putting them back in their place.

As the fragments are recognized and scanned, precision mechanical arms and hands will manipulate and move them using sensors to prevent any damage.

If that works, travigle suggests that the process will relieve archaeologists and environmentalists of this tedious and capricious task so that they can focus on other aspects of their work.

“Restoring an object that is destroyed like a fresco or any other archaeological object takes really a long time, it takes really time, and there is not always the time to do it, it does not increase knowledge either. or the understanding or expertise of the archaeologist or restorer – it’s just about putting pieces together, ”she said.

“The real work for the archaeologist comes after, because it is about the design or what is depicted or the type of pigments that were used, it is certainly not the piece to piece together and it is the same thing. for restorers therefore it will take a long time to reconstruct an object and give more time to archaeologists and restorers to do their real work “.

The project started in September 2021 and is expected to last four years.

To learn more about this story, watch the video in the media player above.

Previous Houston Goes For A Spacewalk: 'The Infinite' Launches VR Space Station Tours
Next Polytech Graphic Design students among the top 15 in the annual Art + Effect poster competition of the University of Cabrini