Optimizing Cross-Lingual Transfer Learning with Transformer Architectures for Low-Resource Natural Language Processing

Abstract

Modern Natural Language Processing (NLP) faces a stark digital divide, leaving the vast majority of the world’s seven thousand languages digitally marginalized due to data scarcity and architectural biases. This review paper examines state-of-the-art cross-lingual transfer learning (CLTL) optimization methodologies designed to mitigate these resource constraints utilizing transformer architectures. We provide a comprehensive taxonomy categorizing transfer paradigms into instance, feature, and parameter-based strategies, highlighting advanced frameworks like colexification graphs and multilingual embedding alignment. Furthermore, we critically analyze structural bottlenecks such as the "curse of multilinguality," demonstrating how capacity competition and script barriers induce negative cross-lingual interference. To overcome the tight coupling of tokenizers to high-resource languages, this paper reviews vocabulary adaptation techniques that mitigate over-fragmentation and inference slowdown. We evaluate parameter-efficient fine-tuning (PEFT) configurations identifying the Houlsby+Inversion adapter as a top-performing structure alongside multi-stage curriculum alignment frameworks like Progressive Code-Switching and MERLIN. Finally, we expose systemic gaps in the current evaluation ecosystem, specifically looking at coverage and task equity disparities, while advocating for culturally grounded, native-authored benchmarks. Ultimately, this synthesis demonstrates that moving away from static translation paradigms toward modular architectures, optimized tokenization transplantation, and structured alignment curricula is vital for building equitable, globally inclusive language technology.