Open Access

ARTICLE

Video Conference System in Mixed Reality Using a Hololens

Baolin Sun1,#, Xuesong Gao2,#, Weiqiang Chen2, Qihao Sun2, Xiaoxiao Cui1, Hao Guo1, Cishahayo Remesha Kevin1, Shuaishuai Liu2, Zhi Liu1,*
1 School of Information Science and Engineering, Shandong University, Qingdao, 266000, China
2 State Key Laboratory of Digital Multi-Media Technology, Hisense Co., Ltd., Qingdao, 266000, China
* Corresponding Author: Zhi Liu. Email:
(This article belongs to this Special Issue: Frontiers in New Generation Product Manufacturing and Quality Inspection Based on Smart Sensing, Image Diagnostics and Advanced Computational Dynamics)

Computer Modeling in Engineering & Sciences 2023, 134(1), 383-403. https://doi.org/10.32604/cmes.2022.020491

Received 26 November 2021; Accepted 16 March 2022; Issue published 24 August 2022

Abstract

The mixed reality conference system proposed in this paper is a robust, real-time video conference application software that makes up for the simple interaction and lack of immersion and realism of traditional video conference, which realizes the entire process of holographic video conference from client to cloud to the client. This paper mainly focuses on designing and implementing a video conference system based on AI segmentation technology and mixed reality. Several mixed reality conference system components are discussed, including data collection, data transmission, processing, and mixed reality presentation. The data layer is mainly used for data collection, integration, and video and audio codecs. The network layer uses Web-RTC to realize peer-to-peer data communication. The data processing layer is the core part of the system, mainly for human video matting and human-computer interaction, which is the key to realizing mixed reality conferences and improving the interactive experience. The presentation layer explicitly includes the login interface of the mixed reality conference system, the presentation of real-time matting of human subjects, and the presentation objects. With the mixed reality conference system, conference participants in different places can see each other in real-time in their mixed reality scene and share presentation content and 3D models based on mixed reality technology to have a more interactive and immersive experience.

Keywords

Mixed reality; AI segmentation; hologram; video conference; Web-RTC

Cite This Article

Sun, B., Gao, X., Chen, W., Sun, Q., Cui, X. et al. (2023). Video Conference System in Mixed Reality Using a Hololens. CMES-Computer Modeling in Engineering & Sciences, 134(1), 383–403.



This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 282

    View

  • 175

    Download

  • 0

    Like

Share Link

WeChat scan