DebrisVision: Bridging the Synthetic-to-Real Gap for Enhanced Underwater Debris Analysis

Why underwater debris is, at its core, a data problem

Over 14 million tons of plastic enter the ocean every year, and a large share of it eventually sinks to the seafloor, forming debris fields that entangle marine life and break down into microplastics that move up the food chain. Autonomous underwater vehicles and computer vision are a natural fit for detecting and eventually cleaning up this debris, but the models behind them run into a familiar wall: there simply are not enough large, diverse, precisely annotated underwater datasets to train them on.

The datasets that do exist tend to be small, geographically narrow, and skewed toward whatever debris is easiest to photograph. Rare but important categories, discarded fishing gear, metal containers, electronics, are consistently underrepresented, which biases detection models against exactly the objects that are often the most hazardous. On top of that, underwater conditions themselves shift constantly: turbidity, lighting, and occlusion all change what a camera sees from one dive to the next, and models trained on one set of conditions frequently fail to generalize to another.

Building a hybrid, multi-modal dataset

DebrisVision combines 9,430 real-world underwater images with 15,570 synthetically generated ones, 25,000 images in total across 24 debris categories. Every image carries four kinds of annotation: bounding boxes, segmentation masks, depth maps, and a written text description, generated automatically using a pipeline of foundation models rather than hand-labeled one by one. Grounding DINO handles zero-shot detection from text prompts, Segment Anything produces the masks, Depth Anything V2 estimates depth, and InternVL2.5 writes the textual descriptions.

Real-world underwater debris data is scarce for a lot of categories, so we filled the gaps with synthetic images generated from text-to-image diffusion models. We evaluated three candidates, Flux.1-dev, Sana, and Janus Pro, on realism and text alignment, and settled on Flux.1-dev with LoRA photorealism, since it produced the most convincing debris textures and the lowest Frechet Inception Distance against our real image set.

Closing the synthetic-to-real gap

Synthetic images help with coverage, but a model trained purely on them tends to underperform on real footage, because even a photorealistic diffusion model does not perfectly reproduce underwater optics: light scattering, particulate haze, and the specific way water desaturates color at depth. To narrow that gap, we ran our synthetic images through FastCUT, an unpaired image-to-image translation method, which nudges the texture and lighting of synthetic debris toward what a real underwater camera would actually capture without needing paired synthetic-real examples to train on. This single step reduced the FID between our synthetic and real image distributions by 20 percent.

What the numbers show

We benchmarked DebrisVision by fine-tuning a range of detection and segmentation models, YOLOv8, YOLO11, DETR, RTDETR, and YOLO12 for detection, and YOLOv8, YOLO11, FPN, SegFormer, UNet, UNet++, and DeepLabV3+ for segmentation, across different combinations of real, synthetic, and domain-adapted synthetic data. The pattern was consistent: adding domain-adapted synthetic data on top of real data outperformed training on real data alone by a wide margin. YOLOv8 improved by 2.29 times on detection mAP50 and 3.03 times on segmentation mAP50; YOLO11 improved by 2.01 times and 2.62 times respectively. Synthetic data alone was not enough, and real data alone was not enough either. It was the combination, with the domain gap closed, that made the difference.

Teaching a model to describe debris, not just box it

Since every image in DebrisVision already carries a text description, we used that pairing to fine-tune CLIP into what we call DebrisCLIP. The result is a model that can classify debris from a free-text query without retraining, and retrieve images by description with noticeably better precision than the base CLIP model, particularly on the ambiguous, low-contrast cases where general-purpose CLIP tends to guess wrong. It is a smaller contribution than the dataset itself, but it is a fairly direct demonstration that multi-modal annotation pays for itself beyond just detection and segmentation.

What is still unresolved

Class imbalance is reduced by DebrisVision, not eliminated. Categories like electronics waste remain underrepresented even after synthetic augmentation, and the domain gap between synthetic and real images, while narrower, has not closed completely. Our depth maps also come from a monocular estimator without any sensor-based validation, so they should be read as reasonable approximations rather than ground truth. We see the current 25,000-image release as a starting point rather than a finished benchmark, and the next steps we have in mind, scaling toward 100,000 images and validating depth against real sensors, are meant to close exactly these gaps.

Why we built this

None of this is useful unless other people can build on it, so DebrisVision, the code, and the trained models are open-sourced. The goal was never just a benchmark number. Autonomous marine cleanup, robotic navigation around debris fields, and long-term ecological monitoring all depend on models that can actually recognize what they are looking at underwater, and that starts with a dataset that reflects what is really down there. This work was presented at the CVAUI & AAMVEM Workshop, ICCV 2025.