Open Access iconOpen Access

ARTICLE

crossmark

Integrating Deep Learning and Machine Translation for Understanding Unrefined Languages

HongGeun Ji1,2, Soyoung Oh1, Jina Kim3, Seong Choi1,2, Eunil Park1,2,*

1 Department of Applied Artificial Intelligence, Sungkyunkwan University, Seoul, 03063, Korea
2 Raon Data, Seoul, 03073, Korea
3 Department of Computer Science and Engineering, University of Minnesota, Minneapolis, 55455, MN, USA

* Corresponding Author: Eunil Park. Email: email

Computers, Materials & Continua 2022, 70(1), 669-678. https://doi.org/10.32604/cmc.2022.019521

Abstract

In the field of natural language processing (NLP), the advancement of neural machine translation has paved the way for cross-lingual research. Yet, most studies in NLP have evaluated the proposed language models on well-refined datasets. We investigate whether a machine translation approach is suitable for multilingual analysis of unrefined datasets, particularly, chat messages in Twitch. In order to address it, we collected the dataset, which included 7,066,854 and 3,365,569 chat messages from English and Korean streams, respectively. We employed several machine learning classifiers and neural networks with two different types of embedding: word-sequence embedding and the final layer of a pre-trained language model. The results of the employed models indicate that the accuracy difference between English, and English to Korean was relatively high, ranging from 3% to 12%. For Korean data (Korean, and Korean to English), it ranged from 0% to 2%. Therefore, the results imply that translation from a low-resource language (e.g., Korean) into a high-resource language (e.g., English) shows higher performance, in contrast to vice versa. Several implications and limitations of the presented results are also discussed. For instance, we suggest the feasibility of translation from resource-poor languages for using the tools of resource-rich languages in further analysis.

Keywords


Cite This Article

H. Ji, S. Oh, J. Kim, S. Choi and E. Park, "Integrating deep learning and machine translation for understanding unrefined languages," Computers, Materials & Continua, vol. 70, no.1, pp. 669–678, 2022.

Citations




cc This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 2606

    View

  • 1238

    Download

  • 0

    Like

Share Link