Explainable Hybrid CNN–Transformer Framework for Crop Classification and Biotic Stress Detection under Real-World Field Variability

Abstract

In agricultural field applications, reliable image analysis is still challenging due to the variability of the crop appearance with illumination, background clutter, camera viewing angle, growth stages, partial occlusion and visible pest and disease symptoms. This study proposes a hybrid CNN–Transformer architecture to classify crops and analyze biotic stresses in such heterogeneous settings, with explainability. The framework relies on the ResNet-50 convolutional backbone to learn the local morphology and texture, translates high-level feature maps into tokens, and a Transformer encoder to capture the long range contextual dependencies. Fused local-global features are served to task-specific prediction heads, and visual explanations are given by Grad-CAM and Transformer attention maps. Experiments are performed on a well curated dataset of 30,204 RGB images from 15 different crop categories with standardized input size of 224×224, augmentation used only during training, stratified partitioning and imbalance-aware optimization. The obtained results of the implemented model were: the accuracy for crop classification was 97.31%, the weighted precision was 0.970, the weighted recall was 0.9731, the weighted F1-score was 0.971, the macro F1-score was 0.934 and the balanced accuracy was 0.940. The accuracy of multi-seed testing was 97.31±0.07%, which was relatively stable. The proposed model also achieved the best performance among the following models under the same evaluation protocol: VGG16, MobileNetV2, ResNet-50, DenseNet121, and EfficientNetB0. The findings are in support of the use of hybrid representation learning, with a local and a global representation, for robust and interpretable agricultural decision support. When exported from the implemented run, separate stress-head metrics should be reported.