Challenging DINOv3 foundation model under low inter-class variability: a case study on fetal brain ultrasound

This study provides the first comprehensive evaluation of foundation models in fetal ultrasound (US) imaging under low inter-class variability conditions. While recent vision foundation models such as DINOv3 have shown remarkable transferability across medical domains, their ability to discriminate anatomically similar structures has not been systematically investigated. We address this gap by focusing on fetal brain standard planes–transthalamic (TT), transventricular (TV), and transcerebellar (TC)–which exhibit highly overlapping anatomical features and pose a critical challenge for reliable biometric assessment. To ensure a fair and reproducible evaluation, all publicly available fetal ultrasound datasets were curated and aggregated into a unified multicenter benchmark, FetalUS-188K, comprising more than 188,000 annotated images from heterogeneous acquisition settings. DINOv3 was pretrained in a self-supervised manner to learn ultrasound-aware representations. The learned features were then evaluated through standardized adaptation protocols, including linear probing with frozen backbone and full fine-tuning, under two initialization schemes: (i) pretraining on FetalUS-188K and (ii) initialization from natural-image DINOv3 weights. Models pretrained on fetal ultrasound data consistently outperformed those initialized on natural images, yielding weighted F1-score improvements of up to 21% (0.73 vs. 0.52 for ViT-B/16). This domain-adaptive pretraining proved critical for resolving low-margin class boundaries; while natural-image weights led to a representational collapse on the TV plane (F1-score 16%), our approach preserved the subtle echogenic and structural cues necessary for its accurate discrimination. These results demonstrate that while generic foundation models fail to generalize under low inter-class variability, domain-specific pretraining is a technical prerequisite for achieving the robust and clinically reliable representations required for fetal brain biometric assessment.