Open Access iconOpen Access

ARTICLE

crossmark

A Resource-constrained Edge IoT Device Data-deduplication Method with Dynamic Asymmetric Maximum

Ye Yang1, Xiaofang Li2, Dongjie Zhu3,*, Hao Hu3, Haiwen Du4, Yundong Sun4, Weiguo Tian3, Yansong Wang3, Ning Cao1, Gregory M.P. O’Hare5

1 School of Artificial Intelligence, Wuxi Vocational College of Science and Technology, Wuxi, 214028, China
2 Department of Mathematics, Harbin Institute of Technology, Weihai, 264209, China
3 School of Computer Science and Technology, Harbin Institute of Technology, Weihai, 264200, China
4 School of Astronautics, Harbin Institute of Technology, Harbin, 150001, China
5 School of Computer Science, University College Dublin, Dublin, Dublin4, Ireland

* Corresponding Author: Dongjie Zhu. Email: email

Intelligent Automation & Soft Computing 2021, 30(2), 481-494. https://doi.org/10.32604/iasc.2021.019201

Abstract

Smart vehicles use sophisticated sensors to capture real-time data. Due to the weak communication capabilities of wireless sensors, these data need to upload to the cloud for processing. Sensor clouds can resolve these drawbacks. However, there is a large amount of redundant data in the sensor cloud, occupying a large amount of storage space and network bandwidth. Deduplication can yield cost savings by storing one data copy. Chunking is essential because it can determine the performance of deduplication. Content-Defined Chunking (CDC) can effectively solve the problem of chunk boundaries shifted, but it occupies a lot of computing resources and has become a bottleneck in deduplication technology. This paper proposes a Dynamic Asymmetric Maximum algorithm (DAM), which uses the maximum value as the chunk boundaries and reducing the impact of the low-entropy string. It also uses the perfect hash algorithm to optimize the chunk search. Experiments show that our solution can effectively detect low-entropy strings in redundant data, save storage resources, and improve sensor clouds system throughput.

Keywords

Internet of Things; Sensor clouds; Intelligent transportation; Data deduplication

Cite This Article

APA Style
Yang, Y., Li, X., Zhu, D., Hu, H., Du, H. et al. (2021). A Resource-constrained Edge IoT Device Data-deduplication Method with Dynamic Asymmetric Maximum. Intelligent Automation & Soft Computing, 30(2), 481–494. https://doi.org/10.32604/iasc.2021.019201
Vancouver Style
Yang Y, Li X, Zhu D, Hu H, Du H, Sun Y, et al. A Resource-constrained Edge IoT Device Data-deduplication Method with Dynamic Asymmetric Maximum. Intell Automat Soft Comput. 2021;30(2):481–494. https://doi.org/10.32604/iasc.2021.019201
IEEE Style
Y. Yang et al., “A Resource-constrained Edge IoT Device Data-deduplication Method with Dynamic Asymmetric Maximum,” Intell. Automat. Soft Comput., vol. 30, no. 2, pp. 481–494, 2021. https://doi.org/10.32604/iasc.2021.019201



cc Copyright © 2021 The Author(s). Published by Tech Science Press.
This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 1910

    View

  • 1070

    Download

  • 0

    Like

Share Link