Internet of Cultural Things: Current Research, Challenges and Opportunities

1  Introduction

Everything in the Internet of Things is considered to be interconnected intelligent objects, and the IoT can be regarded as a new version of Information and Communication Technology (ICT) [1]. Sensors have been gradually embedded in various systems in our lives through computer technologies, automation, and automatic signal processing and offer multiple solutions for various areas, such as healthcare, retail market, transportation, security, agriculture, etc. IoT has significantly influenced everything from smart appliances to smart cities. It has also opened a new challenge in the culture domain, which is called the Internet of Cultural Things (IoCT). IoCT utilizes the Internet, the Internet of Things, and other general platforms as the underlying technical framework for offering convenience and new experiences for users like obtaining cultural resources, cultural products, and cultural services. Furthermore, it can also help to enhance economic development. The application scenarios of IoCT include cultural heritage protection [2], public cultural service improvement [3], local cultural tourism promotion [4], digital classroom education [5], the entertainment industry of theme parks [6], information search [7], personalized recommendation [8], multimedia interaction [9], etc. The IoT can monitor the environment where users and cultural objects are located and collect different environmental parameters through context-awareness technology. It can fulfill user needs by connecting associated devices [10]. The IoT is gradually infiltrating into various scenarios in the cultural sector, activating the static cultural objects and traditional spaces through technological means and establishing a more intimate interconnection and interaction with people.

This research aims to investigate the current status quo of IoCT, including its development status, technology deployment, and relevant cases. The contributions of this article related to IoCT are as follows. Section 2 describes the development background of IoCT and the previous research work. Section 3 presents the components and technologies of IoCT. Section 4 mentions the current cases of the IoCT in smart areas, including tourism, museums, classrooms, and historic districts. In addition, this section also introduces some typical applications, sensors/devices, network communication protocols, and related technologies. Section 5 proposes a systematic architecture of IoCT system. Section 6 concludes the challenges and opportunities faced by IoCT in the future.

2  Background and Related Works

In the IoCT domain, there has been a breadth of research on IoCT covering the topic of public cultural institution service improvement, the development, and protection of cultural heritage, Smart City, sustainable development of the tourism industry, etc. In 2015, in partnership with King’s College London and the University of Southampton, the British Library launched a project named “Internet of Cultural Things: Creative Explorations of Data in Cultural Institutions”, funded by the Arts and Humanities Research Council (AHRC). It proposed that cultural institutions should be regarded not only as physical repositories of collections but also as living organisms of data. The project aimed to explore how to leverage cultural data from the British Library (BL) and promote the cultural literacy of the public [11]. It would give more chances to the public to have better participation and creation in cultural projects. Moreover, through artistic visualization projects such as Elastic System, BL could explore the convergence of digital art and the humanities. Based on the background of Smart City, learning city, and cultural heritage, McKenna [12] studied the proper development and utilization of significant cultural heritage elements or fragments by using IoCT and the development space of related applications. This paper also provided an operational environmental heritage framework for the city reconstruction in a more spontaneous, meaningful, and purposeful way. Chianese et al. [13] have published a series of articles in the area of smart museums and smart cultural heritage, describing the IoCT as the intersection of cultural heritage and IoT, by integrating intelligent objects, sensors, services, and applications in static cultural sites. They designed an “intelligent art exhibition of talking objects” under the architecture of the IoCT that they had proposed. Besides, they also presented the method of analyzing the IoCT by combining and analyzing the semantic technology of information from heterogeneous sources [14]. In [4], Lerario explored how the IoT is the key to a sensitive balance between the development needs of Italy’s heritage cities and sustainable conservation. The paper analyzed the interaction between the Internet of things and architectural heritage using the “People, Activities, Contexts, Technologies (PACT)” model in UX. Taking a broader view of culture, the IoT can make adjustments or regulations from an overall systemic perspective. It can serve as a tool to reposition economic growth and sustainable development from a macro perspective

The Internet of things has the ability to connect assets, objects, devices, and people. At present, lots of research has been carried out in culture based on IoCT. However, the primary studies are from a single background perspective. It is vital to have a systematic survey to collect, summarize, analyze and classify the most advanced technologies and application cases in this area.

3  Major Components and Key Technologies

3.1 Major Components of IoCT

The IoCT facilitates the connection of cultural objects, the Internet, and people. The following will analyze the major components from four aspects:

Data Acquisition: Deploying various sensors and devices to collect data related to humans, objects, and ambient spaces, as shown in Tab. 1.

Data Transmission: Connect sensors or sensor-based terminals to the IoT services via the network, including device access and data transmission among devices.

Data Processing: This refers to the collection, storage, retrieval, processing, transformation, and transmission of data.

Data Application: This refers to developing the related platforms and applications according to the specific scenario requirements.

3.2 Key Technologies of IoCT

3.2.1 Sensing and Recognition Technology

Wireless Sensor Network (WSN): WSN is a distributed sensor network, which communicates wirelessly. WSN can monitor the temperature and humidity of indoor and outdoor environments of cultural venues.

Human Presence Detection Technology: This is a summary of a series of technologies and methods to recognize complex human activity [16], including radar technology, human image recognition technology, fingerprint recognition technology, etc.

Quick Response (QR) Code: It can integrate multimedia information such as images, sound, and text. It is used for digital exhibition board descriptions of collections, voice guides, museum publicity, museum ticketing system, etc.

Radio Frequency Identification (RFID): RFID is wireless communication technology and has been widely used in the market. It can be used to store collection descriptions, which can realize the overall management of collection information and be applied in interactive exhibits.

3.2.2 Network Communication Technology

Lora: It is a long-range, energy consumption wireless standard. In a large space area, it can realize real-time monitoring and interval reporting of museums and galleries’ temperature, humidity, light, and other environmental parameters.

Zigbee: Wireless communication protocol with low complexity, short distance, low power consumption, and low cost can be used to track and identify moving objects in indoor spaces of cultural places [17].

Wi-Fi: Wi-Fi is the most widely used wireless LAN transmission technology. It can convert wired network signals into wireless signals, providing free Wi-Fi hotspots for visitors to use the Internet and conducting indoor positioning [18] and crowd analysis [19] for visitors in the intelligent exhibition hall.

Bluetooth: Bluetooth technology is a low-cost, short-range wireless connection technology. There is also an energy-saving version to support the use of wireless beacon devices-BLE. It can be deployed in the intelligent interactive museum to provide users with cultural information related to works of art.

5G: With the feature of fast transmission speed, high signal transmission broadband, and low network delay, combined with Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR) technologies, 5G can bring a more immersive and multimodal interactive experience to offline cultural places. For example, relevant literature introduces the overall framework of the cultural education scene based on the 5G network [20].

3.2.3 Computing and Service Technology

Cloud Computing and Edge Computing: Cloud computing can provide services such as context-awareness navigation system [21]. Edge computing refers to the mobile nodes close to the data source, such as smartphones, tablets, personal computers, cameras, and other smart devices, which have the capability of calculating, storing, and applying data. Cultural institutions can host AR services on the Mobile Edge Computing (MEC) platform to provide low-latency services [22].

Big Data and Artificial Intelligence: Cultural industry has a massive, high growth rate and diversified information assets and has a dynamic and open data ecosystem that keeps changing with the diversified development of cultural practices [23]. Through the semantic relationship analysis of unstructured text entities and other technologies, the knowledge base population, information retrieval, and question answering can be realized [24]. The cultural industry can use deep learning, knowledge computing, visualization, and other artificial intelligence technologies to improve data utilization and management. At present, many global cultural institutions at home and abroad have cooperated with technology companies to launch a series of related intelligent experience services.

4  Typical Applications in IoCT

IoCT has a wide range of application scenarios. This paper discusses four common scenarios, including Smart Tourism, Smart Museum, Smart Classroom, and Smart Historic District, as depicted in Fig. 1 below.

Figure 1: Application scenarios and functions of IoCT

4.1 Smart Tourism

Smart tourism can be built on the concept of Smart City, which is one of the representative forms of cultural experience. Combined with the Internet of Things technology, people can get services matching their desires, needs, and consumer demands in the tourism scene. Sensors embedded in the tourism environment provide information and knowledge of many complex sensory systems, providing information for tourism segmentation scenarios such as traffic, attractions, travel, shopping, and accommodation [25]. The specific applications are as follows: real-time journey assistant, tourism service tailoring, immersive tourism experience, and tourism service improvement.

4.1.1 Mobile Applications and Real-time Journey Assistant

From the aspects of consumers, technologies, and providers, the authors in [26] carried out relevant research has mobile technology and its application in smart tourism. Smart devices are very important tools that provide real-time data to visitors, such as smartphones with cameras, microphones and smart sensors, touchpads, watches, and more. At the same time, with the growing maturity of positioning technology, computer vision, and AR technology, users can obtain augmented reality services on mobile applications. And with the fusion of positioning technology, computer vision, and AR technology, users can access augmented reality services in mobile applications. The application assistant can be divided based on the way of information provision, including the visual application assistant and audio application assistant.

•   Visual Application Assistant

The Old Summer Palace Dashuifa Site employs Visual Positioning and Augmentation Service (VPAS) provided by the Baidu DuMix AR platform to restore the building’s original appearance and provide users with a real-world experience by Baidu Map APP. Visual service is mainly comprised of high-precision map construction, real-time visual positioning, and online fusion tracking. The effect of the rebirth of the remains is eventually achieved [27], as shown in Fig. 2.

Figure 2: Visual application assistant case-Baidu DuMix AR [27]

•   Audio Application Assistant

Boletsis and Costas proposed AudioNear in [28], as shown in Fig. 3, which aimed to support visitors’ exploration of Oslo’s urban attractions through Audio Augmented Reality (AR). Audio AR allows users to perceive the real world and virtual audio simultaneously. It offers motion tracking, interaction, and position awareness based on its radius-activated functionality. When visitors enter the radius of a tourist sight, the client sends getPOIs request to the AudioNear service provider and database to obtain basic AR content about AudioNearPOIs. Besides, it can also send external link requests (such as audio tracks, images/icons) directly, providing visitors with audio-visual information such as history, architecture, visiting time, etc.

Figure 3: Audio application assistant case–AudioNear [28]

The development of IoT technology has made it more inclusive and accessible to travelers, supporting visitors with mobility, visual, auditory, and cognitive impairments to cope with various obstacles. As an accessible travel application, TUR4all enables people with accessibility needs to plan their travel and holidays and access objective and accurate information about the destinations and tourism resources [29]. The United Nations World Tourism Organization (UNWTO) released a report in 2021 entitled “Accessibility and Inclusive Tourism Development in Nature Areas Compendium of Best,” which proposes an accessible framework based on Professor Buhails’ research. It proposed that the applications (along with application programming interfaces-APIs) are one of the significant aspects, which could provide travelers with real-time and personalized advice and support [30].

4.1.2 Context-awareness and Tourism Service Tailoring

Through sensors and associated technologies, context-awareness is the capability of computer devices to sense and react based on their environment by analyzing relevant data automatically. Travelers can now benefit from context-awareness technology to personalize and customize their travel services with information tailored to their tastes, interests, and needs from a variety of mobile app recommendations [31]. Some countries and regions have developed relevant products and services for tourism cultural resources. SCRABS [10] is a cultural environment-oriented smart context-aware browsing assistant, as shown in Fig. 4, collecting data from five different heterogeneous pervasive data sources. The basic feature of SCRABS is its portability, “each user can carry a personalized version of the system, in which the information changes dynamically with the context.” LEAL et al. [32] compared the relevant context-aware tourism applications and analyzed them from four technical dimensions: knowledge acquisition, representation, processing, and services. As one of the 12 “key concepts of intelligence,” Hamid et al. mentioned the frequency of its application in e-tourism systems in the paper [33].

Figure 4: SCRABS architecture [10]

4.1.3 Multi-devices Connectivity and Immersive Tourism Experience

The development of IoT technology provides a broader space for the interconnection of devices under multiple scenarios. It changes the problem that the scope and quality of services provided in the traditional tourism experience will be constrained by time and space and brings a more seamless and smooth immersive experience to tourists. As an example, to achieve a “more immersive, more seamless, more personalized experience,” Disney has released a series of IoT hardware and software products, and some of them are as follows:

•   Wearable Devices

MagicBands, released in 2013, can be used for hotel door unlocking, catering fee payment, and park activity reservations. The wrist strip contains a small antenna and a high-frequency radio. The antenna allows MagicBands to be detected at short contact points, while radio devices are used for long-range communication. Through RFID, MagicBands could also track customer behavior, analyze buying habits, and provide real-time data to Disney employees.

•   Applications

Disney launched the Disney Parks app in 2018. It offers enhanced adventures in “Star Wars: Galaxy’s Edge,” which allows visitors to interact with the environment using Bluetooth and GPS positioning while playing offline, as shown in Fig. 5 [34]. When the visitors scan the online code, they will receive instant rewards for the task after completing the puzzle-solving interactive games. In the future, this interaction may be extended into visitors’ homes.

Figure 5: The disney parks app-Star Wars: Galaxy’s Edge [34]

•   Inductive identification equipment in the park

Disney filed patents for sensor-sensitive identification through sensors and camera-carrying robots, such as shoes, to identify guests in multiple locations and identify marks on guests’ clothes [35].

•   Visitor interactive devices in the park

Disney conducted research on Mickey and Minnie’s robotic versions for interacting with visitors.

4.1.4 Data Analysis and Tourism Service Improvement

In the field of intelligent tourism, a massive amount of heterogeneous data from social networks, applications, and sensors are waiting to be mined appropriately and utilized with a large amount of free information from the user’s behavior (voluntarily provided, observed, or inferred). Smart tourism offers new ways to manage travel flows and better travel services to enhance the travel experience. Sensors in IoT gather information about travel services and habits and responses to disparate attractions, enabling accurate marketing and increasing tourist satisfaction. Sharma et al. [36] analyzed that “using big data to analyze real-time data” was one of the key components behind the concept of IoT-based travel. They proposed the concept of IoTeST in the paper. For example, during a trip, the footprints and interactions of visitors in each scene are recorded in real-time, helping tourism service providers to understand the tourists’ experiences and preferences.

4.2 Smart Museum

Smart Museum refers to the use of modern technologies such as the Internet of things, big data, artificial intelligence, and cloud computing to perceive, calculate and analyze the people, things, and activities related to the work of the museum, so as to form a new model of the museum and truly promote the comprehensive improvement of the communication, collection, research, exhibition, protection, scientific research, and management ability in the museum. The current IoCT research at the Museum of intelligence focuses on two main areas:

•   Focused on the key concepts and overall solutions of IoT in Smart Museum.

•   Focused on the specific services provided by the museum, including collection-centric service, tourist-centric service, and exhibition-centric service.

4.2.1 IoT Systems for Smart Museums

As one of the ways to digitize the traditional offline museum, IoT is being studied for its key concepts. Aristova et al. [37] proposed three key concepts of digitalization: digitalization of museum oeuvre and archives for collecting, storing, and organizing information; digital transformation of a museum for better communication with the audience and presentation; big data management for monitoring visitors and accumulating information.

Korzun et al. [38] introduced a design solution that implements the semantic layer, which enriches the collection stored in traditional Management Information System (MIS). The concept of information services is based on services of a high level of intelligence, not limited to directly providing record-based descriptions of exhibits. The exhibits are transformed into intelligent objects of the IoT, which can expand knowledge from internet services and social networking activities. The information of the exhibits and other historical data sources is semantically enriched and integrated. The results were applied to the Petrozavodsk State University.

Piccialli et al. [13] created an IoT platform for intelligent art exhibitions “Talking Art”, which is deployed at the sculpture exhibition in Naples, Italy. Every sculpture there was equipped with a sensor panel that actively presents itself to the visitor in the surrounding area, telling the story in different ways and languages.

Pierdicca et al. [39] described the “IoT will be a key player” in the evolution of conversation between individuals and the museum space. The museum can use the IoT implementation to obtain the visitors’ data, like the visitor attention span in different rooms of a curated exhibition, and analyze their interests and preferences. It is helpful for museums to change their configuration for providing the tailoring interaction with specific visitors, instead of using a “one size fits all” approach.

4.2.2 IoT Applications in Smart Museums

The IoT application scenarios of smart museums are broad and can be subdivided into multiple application levels. In this paper, applications are classified into three dimensions:collection-centric service, tourist-centric service, and exhibition-centric service. The following Tab. 2 shows the typical cases in these applications.

4.3 Smart Classroom

Classrooms have a significant role in the dissemination of cultural information. Current research has highlighted the advantages of smart classrooms, such as facilitating interaction and sharing learning content and materials easier for students to understand. The SMART company introduced a series of interactive whiteboard tools and Lumio classroom learning software that makes it easy for teachers to integrate teaching resources from different sources and edit them while enabling students to participate in learning in more interesting ways [44]. SAINI and GOEL [45] summarized the software and hardware and related technical support from four main applications of the smart classroom, including content and presentation, interaction and participation, evaluation, and physical environment. Memos et al. present the concept of Revolutionary Interactive Smart Classroom (RISC) that provides a virtual environment for enhancing the learning experience based on 5G [46]. With the normalization of the Covid-19 epidemic, more research will emerge on smart classrooms for distance education, such as comparing the classroom experience with face-to-face, emergency distance learning [47]. The applications are as shown in Tab. 3 below.

4.4 Smart Historical District

Under the construction of Smart City, the evolving IoT technology interconnects tangible objects and intangible information from historical and cultural neighborhoods, so that the historical value can be more fully highlighted, making neighborhoods more attractive, sustainable, inclusive, and have the balancing development.

4.4.1 Participatory Cultural Interaction

Community residents have more opportunities to make their voices heard to the outside world with IoT technology. Based on the diverse immigrant culture of the Southhall community in the UK, the project “The Lost Stories Collective” provides a platform for community participation in historical knowledge sharing. Residents own the right to interpret local culture. The designers worked with residents to design the QR code and bring the AR experience into the block, which changed the traditional one-way flow of historical knowledge by authoritative museums and enhanced the cohesion of the community [48], as shown in Fig. 6.

Figure 6: The project of “The Lost Stories Collective” [48]

4.4.2 Historic Building Preservation

The urban historical districts have the old building tilt, cracking, overload, and other structural anomalies. Building damage, such as unlawful knocking on the wall, ornamentation, and other violations, can be monitored in real-time by placing sensors. In this way, the relevant institutions can implement digital management of building safety.

4.4.3 Dynamic District Management

At present, IoT has been used to monitor the environmental data of historical and cultural blocks, as well as the urban behavior data such as traffic flow and the state data of public facilities. For example, conservation districts of historic sites in Beijing Baita Temple apply the fine sensor network name “CITY GRID” in urban space, which can monitor the real-time traffic flow, passenger flow, and other multi-level dynamic situations. The multi-dimensional result can be presented on the visualization platform [49], as shown in Fig. 7.

Figure 7: The visualization platform of CityGrid [49]

5  Architecture of IoCT

Different from the IoT in the agriculture and industry domain, the IoCT platform architecture enables a place where cultural objects are linked to the internet and humans. The overall architecture of the IoCT is shown in Fig. 8. The architecture of IoCT is composed of five layers, detailed in the following: sensor layer, core layer, network layer, application layer, and user layer.

Sensor Layer: As the bottom layer of the IoT infrastructure, this layer is composed of a variety of sensors. The sensing and recognition technology is the core support of this layer. Sensors can perceive things like people, objects, devices, and the environment.

Core Layer: Collecting data from the sensor network, digital repository, social network, web, etc., and storing them in the Knowledge Base according to certain norms and standards.

Network Layer: Managing data transfer between different networks and applications. For example, data collected by the sensor layer and in the database are transmitted in real-time over 4G/5G, broadband networks, self-organizing networks, WSN, and so on.

Application Layer: It includes the four main applications mentioned in this paper. The applications can provide digital management, environmental monitoring, personalized recommendation, information query, evaluation, and so on.

User Layer: It includes the government, cultural institutions such as “galleries, libraries, archives, museums (GLAM)”, schools, enterprises, and individuals with a need for cultural resources and looking forward to obtaining services from various hardware and software.

Figure 8: The system architecture of IoCT

6  Research Challenges and Opportunities

6.1 Challenges

Privacy and Data Security. When collecting user data, devices can get a large number of “digital footprints” of users, such as online browsing and search behavior, geographical location and moving track, etc. An increasing number of individuals and institutions upload content to cloud servers, and user-sensitive data may be leaked. Therefore, technical solutions related to data sharing need to be optimized from the perspective of security [50]. It is easy to put the user security and privacy at risk when the devices are being attacked, including the leakage of personal information of family members and so on. On the other hand, when collecting user data, the device may not explain the content, scope, and form of the collected data to the user and be authorized.

Operation and maintenance costs. The traditional IoT solutions involve transforming the traditional static spaces into intelligent spaces. Each added function might require additional software and hardware, resulting in the external cost. For example, we can create an affordable IoT solution for making an intelligent classroom and reduce infrastructure requirements by using distributed computing technology and crowdsourcing hardware equipment. In addition, the standardization of intelligent classroom technology is also what the school expects, like some general integration technologies, and can be accepted to modify in a certain range.

Multi-device collaboration. The applications and services of the IoCT need to cooperate and synchronize with multiple sensor devices and the Internet or with other devices. It is necessary to consider how to make these heterogeneous devices and data sources work effectively. At the same time, it’s necessary to consider the rapidly changing technical environment and user attitudes to ensure the feasibility of the system.

Dilemma of “intelligent traps”. Personalized recommendation tools can analyze and predict user behavior and interests by integrating information, products, and services in cultural scenes. The contextual data (time, location, weather conditions, etc.) is the crucial information source for user modeling in personalized content services. It can affect user perception and decision-making behavior. Driven by commercial interests or other intentions, whether the cultural information obtained by people is manipulated or induced to fall into the “intelligent trap” needs to be discussed in future research.

6.2 Opportunities

Real-time personalized response. With the development of edge computing, developers of the IoCT can deploy terminals with high computing performance in near-field environments. Combined with context-awareness technology, the terminal can realize adaptive change according to the specific needs and emotions of users and provide more real-time [51], comprehensive, and efficient cultural information recommendations and services. With the advent of the era of 5G, edge computing makes the culture scenario of terminal equipment have the ability to handle huge amounts of data.

High-quality experience. Emerging technologies like Extended Reality (XR) and the latest concepts such as “Metaverse” provide technical support, bring more real-time personalized multimodal immersive experience, and deliver more positive feedback to participants. It can also promote relevant hardware equipment research and production. At the same time, combined with the development of AI technologies such as computer vision, speech recognition, natural language processing, machine learning, and big data, people can get a more flexible, smooth, and natural interactive experience on mobile applications.

Promotion of cultural communication. IoT technologies make the traditional cultural space dynamic. The process of people acquiring cultural knowledge is more diverse, interesting, and seamless, which promotes the sharing and exchange of knowledge. For example, through the community effect, the cultural exhibitions would have greater influence and dissemination.

In addition, with people’s increasingly high requirements and expectations for cultural learning and experience, research on the dissemination of cultural heritage knowledge, barrier-free cultural experience designed for special groups, and personalized cultural activity management needs to be improved.

7  Conclusion

In this paper, we have presented a systematic survey on the state-of-the-art for IoT in culture. IoCT is an exploratory and promising area. With the advent of cloud computing, edge computing, and other technologies, more and more literature has focused on the management, transmission, and value transformation of data. However, there is a lack of a systematic analysis of the technologies and applications of the IoCT. The purpose of this survey is to analyze the current research progress, technical support, and application scenarios, especially focusing on smart tourism, smart museum, smart classroom, and smart historic district. Finally, the architecture of IoCT is proposed and some research challenges and opportunities are discussed in the end.

In the future, IoCT will cover the whole scene of the cultural industry. Cultural resources can be transferred across scenes, platforms, and terminals. For instance, cultural resources can be transferred from GLAM and other cultural institutions to individuals or school institutions, reducing the difficulty and cost of acquiring knowledge. IoCT promotes the integration of cultural industry with other areas, forms more innovative elements, improves the development level of social-cultural productivity, and enhances cultural confidence.

Funding Statement: This work is supported by National Natural Science Foundation of China (62172155)and The National Key Research and Development Program of China (2019YFB1405702)

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.