AI charts a course for HMS Victory

Artificial intelligence is an anchor for restoration work

Modern AI is more than chatbots and deepfakes. One UK museum is using it for something much bigger...and wetter.

The HMS Victory is the world's only surviving first-rate ship. Although now in dry dock in Portsmouth she is still classed as the world's oldest surviving naval vessel in commission, and is the official flagship of the First Sea Lord.

It's a heavy burden for a ship that turns 300 this century, and keeping the Victory - a 57m long, 104-gun galleon - in top shape is a full-time job.

"Her sheer size is already quite challenging just to maintain," says Dr Rodrigo Pacheco-Ruiz, who joined the National Museum of the Royal Navy two years ago specifically to work on Victory.

"As archaeologists - as a small team of archaeologists, there are only two of us - documenting her and recording her can be quite challenging. And this is where we use a lot of automation and a lot of AI documentation and processing, to understand the site in a much better way."

The Museum is working on the 10-year ‘Big Repair' project in partnership with the University of Southampton, where three Masters students - Siddhi Mahendra Pawar, Donheng Wang and Arundati Roy - have developed an AI-based algorithm to match images stored in different locations and add them to a digital 3D model of Victory, to ensure it's as accurate as possible.

Rebuilding Victory

Lord Nelson's former flagship has survived incredibly well through the years, but, inevitably, she won't be around forever. That's part of the reason the Museum is reconstructing her in 3D, using "very, very high-resolution" drone photography.

"We take a lot of pictures from different angles, and then clever software stitches them all together in a 3D space... Each image has a coordinate in space, so we have RTK [real-time kinematic] GPS, which will basically point out those pictures into a very accurate space of millimetres, so we know exactly where each image is taken. Once these images are stitched together the software builds a skeleton of points, which we call a point cloud, and that's the basis of our 3D model."

This photogrammetry process is Dr Pacheco-Ruiz's speciality. It allows the team to investigate everything from very, very small details like the texture of wood grain to an overview of the entire ship.

"If then we overlap these through time, we can quantify things like, ‘The repair has been done here,' but also things like erosion in certain sites. And in the case of Victory, we're attempting to do things like monitoring water ingress, and things like that. So, there's quite a lot of very useful information that comes out of it."

Where AI comes in is in stitching the images together, and also pulling in, storing and tagging new images from open data sources.

"There are some instances where we have additional information, or we have information that was not intended to be 3D; things like images from phones, visitors' images, the occasional selfie that people take on the ship."

These photos lack the quality and RTK information of the drone images, but are still useful as "a snapshot of the ship in time" to spot damage or erosion the professional scans might have missed. However, AI can help with that, too.

The AI system uses the open-source libraries OpenSIFT, Orb and ResNet-18 for image matching, comparing the high-res 3D surveys with metadata on the database and the visitor images.

"Depending on the percentage of the match, those images that were not intended to be part of the documentation will inherit the metadata of the ones that were."

This is a good example of how archaeology can help the computing industry

IT leaders don't often turn to archaeology to improve computer systems - practically never, in fact - but the work on Victory has the potential to enhance AI for the wider industry.

The images, as high as 6k resolution, were taking an unreasonable amount of time for the AI to process for image matching, so the team was forced to find efficiency improvements.

"Normally when they use image matching, they use really low-resolution images, because they want image matching to happen really quickly... [But] because of the sheer nature and size of our images, that was pretty challenging for them because the AI was behaving too well, and it was taking too much information from bigger images.

"They were reluctant to make them smaller and so they found ways of improving and trying to push the boundaries on image matching through a project like this."

Dr Pacheco-Ruiz has "no doubt" that projects and systems like this are "the way forward" for the museum sector.

"Archaeologists are always told that we just borrow techniques from everyone, but it's not always true. We sometimes send things back, and I think this is a nice example of that."