Temporal Vision Transformers for Generalizable Detection and Localization of AI-Generated Video Deepfakes in the Wild

Abstract

AI-generated video has become increasingly diverse, while many deepfake detectors remain sensitive to the manipulation types and acquisition conditions represented in their training data. We present TAV-Former, a temporal vision transformer that combines divided space–time self-attention with a learned cross-frame inconsistency token and three task heads for (i) real/fake detection, (ii) temporal localization of the manipulated interval, and (iii) spatial localization of the edited facial region. We evaluate the model on 1,200 video clips sampled across seven evaluation protocols, including FaceForensics++, Celeb-DF-v2, DFDC, DeeperForensics-1.0, WildDeepfake, DF40, and an AI-generated-video protocol. TAV-Former achieves a mean cross-domain AUC of 0.877, exceeding the spatial transformer baseline by 13.8 AUC points and the TimeSformer baseline by 4.0 points. The advantage over ViT-B/16 is significant across all cross-domain protocols, whereas the difference from TimeSformer is significant in aggregate but not on every individual protocol. TAV-Former also shows smaller performance degradation under compression, downsampling, and blur, with no significant advantage under heavy Gaussian noise. For localization, the model obtains a mean temporal IoU of 0.82 and [email protected] of 0.92. Spatial localization provides a smaller improvement over the Grad-CAM baseline and is significant on three of seven protocols. These results indicate that explicit temporal modeling improves cross-dataset generalization and temporal localization, while its benefit is more limited for spatial localization and strongly coherent AI-generated video.