Table of Content

Open Access

ARTICLE

SNES: Social-Network-Oriented Public Opinion Monitoring Platform Based on ElasticSearch

Chuiju You1, Dongjie Zhu2,*, Yundong Sun2, Anshan Ye3, Gangshan Wu4, Ning Cao1, Jinming Qiu1, Helen Min Zhou5
1 College of Information Engineering, Sanming University, Sanming, 365004, China.
2 School of Computer Science and Technology, Harbin Institute of Technology, Weihai, 264209, China.
3 College of Mathematics and Computer Science, Xinyu University, Xinyu, 338004, China.
4 School of Information Engineering, Jiangsu Polytechnic College of Agriculture and Forestry, Jurong, 212400, China.
5 School of Engineering, Manukau Institute of Technology, Auckland, 2241, New Zealand.
* Corresponding Author: Dongjie Zhu. Email: zhudongjie@hit.edu.cn.

Computers, Materials & Continua 2019, 61(3), 1271-1283. https://doi.org/10.32604/cmc.2019.06133

Abstract

With the rapid development of social network, public opinion monitoring based on social networks is becoming more and more important. Many platforms have achieved some success in public opinion monitoring. However, these platforms cannot perform well in scalability, fault tolerance, and real-time performance. In this paper, we propose a novel social-network-oriented public opinion monitoring platform based on ElasticSearch (SNES). Firstly, SNES integrates the module of distributed crawler cluster, which provides real-time social media data access. Secondly, SNES integrates ElasticSearch which can store and retrieve massive unstructured data in near real time. Finally, we design subscription module based on Apache Kafka to connect the modules of the platform together in the form of message push and consumption, improving message throughput and the ability of dynamic horizontal scaling. A great number of empirical experiments prove that the platform can adapt well to the social network with highly real-time data and has good performance in public opinion monitoring.

Keywords

Social network, public opinion monitoring, elasticsearch, scrapy-redis.

Cite This Article

C. You, D. Zhu, Y. Sun, A. Ye, G. Wu et al., "Snes: social-network-oriented public opinion monitoring platform based on elasticsearch," Computers, Materials & Continua, vol. 61, no.3, pp. 1271–1283, 2019.

Citations




This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 3452

    View

  • 1011

    Download

  • 0

    Like

Share Link

WeChat scan