Table of Content

Open Access

ARTICLE

SMK-means: An Improved Mini Batch K-means Algorithm Based on Mapreduce with Big Data

Bo Xiao1, Zhen Wang2, Qi Liu3,*, Xiaodong Liu3
Jiangsu Key Laboratory of Atmospheric Environment Monitoring and Pollution Control, Jiangsu Collaborative Innovation Centre of Atmospheric Environment and Equipment Technology, School of Environmental Science and Engineering, Nanjing University of Information Science and Technology, 219 Ningliu Road, Nanjing, 210044, China.
School of Computer and Software, Nanjing University of Information Science and Technology, 219 Ningliu Road, Nanjing, 210044, China.
School of Computing, Edinburgh Napier University, 10 Colinton Road, Edinburgh, EH10 5DT, UK
* Corresponding Author: Qi Liu. Email: .

Computers, Materials & Continua 2018, 56(3), 365-379. https://doi.org/ 10.3970/cmc.2018.01830

Abstract

In recent years, the rapid development of big data technology has also been favored by more and more scholars. Massive data storage and calculation problems have also been solved. At the same time, outlier detection problems in mass data have also come along with it. Therefore, more research work has been devoted to the problem of outlier detection in big data. However, the existing available methods have high computation time, the improved algorithm of outlier detection is presented, which has higher performance to detect outlier. In this paper, an improved algorithm is proposed. The SMK-means is a fusion algorithm which is achieved by Mini Batch K-means based on simulated annealing algorithm for anomalous detection of massive household electricity data, which can give the number of clusters and reduce the number of iterations and improve the accuracy of clustering. In this paper, several experiments are performed to compare and analyze multiple performances of the algorithm. Through analysis, we know that the proposed algorithm is superior to the existing algorithms

Keywords

Big data, outlier detection, SMK-means, Mini Batch K-means, simulated annealing

Cite This Article

B. . Xiao, Z. . Wang, Q. . Liu and X. . Liu, "Smk-means: an improved mini batch k-means algorithm based on mapreduce with big data," Computers, Materials & Continua, vol. 56, no.3, pp. 365–379, 2018.



This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 1274

    View

  • 657

    Download

  • 0

    Like

Share Link

WeChat scan