Leveraging Multilingual BERT for Contextually Coherent Dialogue Systems in Low-Resource Settings

Abstract

Dialogue generation systems for Urdu are impeded by the language’s complex morphology, paucity of annotated corpora, and absence of standardized benchmarks. This study addresses these limitations through three contributions: curation and preprocessing of a novel parallel corpus comprising 12 000 Urdu–English conversational pairs;  implementation of two end-to-end architectures—a baseline sequence-to-sequence model with LSTM encoders and decoders, and a transformer-based model via fine-tuning multilingual BERT (mBERT); and comprehensive evaluation employing automatic metrics (BLEU, contextual coherence, response diversity) alongside subjective human judgments. Experimental results demonstrate that the mBERT-based system attains a BLEU score of 0.63 (5 % improvement over the LSTM baseline), a coherence score of 0.82 (7 % increase), and a 10.5 % enhancement in perceived response quality. These findings validate the efficacy of pretrained multilingual transformers for high-fidelity Urdu dialogue generation. The curated dataset and implementation have been released publicly to facilitate further research. Potential application domains include virtual assistants for Urdu speakers, automated customer-support chatbots, and contextualized language-learning platforms

Keywords: Dialog Generation, MBERT, Urdu, Deep Learning, High-Fidelity, Multilingual Transformers