Open Access
ARTICLE
Design and Implementation of Log Data Analysis Management System Based on Hadoop
Dunhong Yao1,2,3,*, Yu Chen4
1 School of Computer Science and Engineering, Huaihua University, Huaihua, 418000, China
2 Key Laboratory of Wuling-Mountain Health Big Data Intelligent Processing and Application in Hunan Province Universities, Huaihua, 418000, China
3 Key Laboratory of Intelligent Control Technology for Wuling-Mountain Ecological Agriculture in Hunan Province, Huaihua, 418000, China
4 School of Computer Science and Engineering, Yulin Normal University, Yulin, 537000, China
* Corresponding Author: Dunhong Yao. Email:
Journal of Information Hiding and Privacy Protection 2020, 2(2), 59-65. https://doi.org/10.32604/jihpp.2020.010223
Received 12 July 2020; Accepted 25 July 2020; Issue published 11 November 2020
Abstract
With the rapid development of the Internet, many enterprises have
launched their network platforms. When users browse, search, and click the
products of these platforms, most platforms will keep records of these network
behaviors, these records are often heterogeneous, and it is called log data. To
effectively to analyze and manage these heterogeneous log data, so that
enterprises can grasp the behavior characteristics of their platform users in time,
to realize targeted recommendation of users, increase the sales volume of
enterprises’ products, and accelerate the development of enterprises. Firstly, we
follow the process of big data collection, storage, analysis, and visualization to
design the system, then, we adopt HDFS storage technology, Yarn resource
management technology, and gink load balancing technology to build a Hadoop
cluster to process the log data, and adopt MapReduce processing technology and
data warehouse hive technology analyze the log data to obtain the results. Finally,
the obtained results are displayed visually, and a log data analysis system is
successfully constructed. It has been proved by practice that the system
effectively realizes the collection, analysis and visualization of log data, and can
accurately realize the recommendation of products by enterprises. The system is
stable and effective.
Keywords
Cite This Article
D. Yao and Y. Chen, "Design and implementation of log data analysis management system based on hadoop,"
Journal of Information Hiding and Privacy Protection, vol. 2, no.2, pp. 59–65, 2020. https://doi.org/10.32604/jihpp.2020.010223