TY  - EJOU
AU  - Song, Shuangqing 
AU  - Chen, Yuan 
AU  - Hu, Xuguang 
AU  - Zhang, Juwei 

TI  - Domain-Aware Transformer for Multi-Domain Neural Machine Translation
T2  - Computers, Materials \& Continua

PY  - 2026
VL  - 86
IS  - 3
SN  - 1546-2226

AB  - In multi-domain neural machine translation tasks, the disparity in data distribution between domains poses significant challenges in distinguishing domain features and sharing parameters across domains. This paper proposes a Transformer-based multi-domain-aware mixture of experts model. To address the problem of domain feature differentiation, a mixture of experts (MoE) is introduced into attention to enhance the domain perception ability of the model, thereby improving the domain feature differentiation. To address the trade-off between domain feature distinction and cross-domain parameter sharing, we propose a domain-aware mixture of experts (DMoE). A domain-aware gating mechanism is introduced within the MoE module, simultaneously activating all domain experts to effectively blend domain feature distinction and cross-domain parameter sharing. A loss balancing function is then added to dynamically adjust the impact of the loss function on the expert distribution, enabling fine-tuning of the expert activation distribution to achieve a balance between domains. Experimental results on multiple Chinese-to-English and English-to-French datasets demonstrate that our proposed method significantly outperforms baseline models in both BLEU, chrF, and COMET metrics, validating its effectiveness in multi-domain neural machine translation. Further analysis of the probability distribution of expert activations shows that our method achieves remarkable results in both domain differentiation and cross-domain parameter sharing.
KW  - Natural language processing; multi-domain neural machine translation; mixture-of-expert; domain-aware gating mechanism

DO  - 10.32604/cmc.2025.072392