Open AccessOpen Access


Unsupervised Semantic Segmentation Method of User Interface Component of Games

Shinjin Kang1, Jongin Choi2,*

1 School of Games, Hongik University, Sejong, 30016, Korea
2 Department of Digital Media, Seoul Women’s University, Seoul, 01797, Korea

* Corresponding Author: Jongin Choi. Email:

Intelligent Automation & Soft Computing 2022, 31(2), 1089-1105.


The game user interface (UI) provides a large volume of information necessary to analyze the game screen. The availability of such information can be functional in vision-based machine learning algorithms. With this, there will be an enhancement in the application power of vision deep learning neural networks. Therefore, this paper proposes a game UI segmentation technique based on unsupervised learning. We developed synthetic labeling created on the game engine, image-to-image translation and segmented UI components in the game. The network learned in this manner can segment the target UI area in the target game regardless of the location of the corresponding component. The proposed method can help interpret game screens without applying data augmentation. Also, as this scheme is an unsupervised technique, it has the advantage of not requiring paring data. Our methodology can help researchers who need to extract semantic information from game image data. It can also be used for UI prototyping in the game industry.


Cite This Article

S. Kang and J. Choi, "Unsupervised semantic segmentation method of user interface component of games," Intelligent Automation & Soft Computing, vol. 31, no.2, pp. 1089–1105, 2022.

This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 934


  • 828


  • 0


Share Link

WeChat scan