Leakage-Aware PCOS Diagnosis Across Independent Ultrasound and Clinical–Biochemical Data Sources

Abstract

Artificial-intelligence studies for polycystic ovary syndrome (PCOS) frequently report high discrimination, yet duplicated images, non-independent partitions, preprocessing leakage, and optimistic model selection can materially distort performance. This study evaluates two independent PCOS data sources through separate leakage-controlled branches: ovarian ultrasound images and clinical–biochemical records. The ultrasound audit began with 10,000 valid images. Exact and orientation-aware perceptual duplicate analysis identified 17 cross-label conflicting groups, removed 3,721 exact duplicate files after conflict exclusion, and retained 1,637 images in 718 analysis groups. A duplicate-group-safe 70/15/15 split was used for multi-seed transfer learning. ResNet50 was selected by mean validation PR-AUC, then Platt-calibrated and evaluated once at a locked threshold on 273 test images, achieving ROC-AUC 0.9926, PR-AUC 0.9975, sensitivity 0.9259, specificity 1.0000, F1 0.9615, and Brier score 0.0139. Cluster-bootstrap 95% confidence intervals were 0.9756–1.0000 for ROC-AUC and 0.8772–0.9976 for F1. The independent clinical branch contained 541 observations and 41 leakage-screened features. Global nested model selection across five outer folds yielded ROC-AUC 0.9501, PR-AUC 0.9091, sensitivity 0.8644, specificity 0.8984, F1 0.8338, and Brier score 0.0798. Grad-CAM visualized image-model attribution; clinical SHAP analysis was excluded because the executed notebook recorded a preprocessing-fit error. The results show that strong PCOS prediction can coexist with substantial data-integrity risks, making leakage control, locked evaluation, calibration, and uncertainty estimation central to defensible reporting. The two cohorts were not patient-linked; no image–clinical fusion was performed.