Table of Content

Open Access iconOpen Access

ARTICLE

crossmark

Core – An Optimal Data Placement Strategy in Hadoop for Data Intentitive Applications Based on Cohesion Relation

Vengadeswaran, Balasundaram

Department of Computer Applications, National Institute of Technology, Tiruchirappalli 620015, India

* Corresponding Author: E-mail: email

Computer Systems Science and Engineering 2019, 34(1), 47-60. https://doi.org/10.32604/csse.2019.34.047

Abstract

The tremendous growth of data being generated today is making storage and computing a mammoth task. With its distributed processing capability Hadoop gives an efficient solution for such large data. Hadoop’s default data placement strategy places the data blocks randomly across the nodes without considering the execution parameters resulting in several lacunas such as increased execution time, query latency etc., Also, most of the data required for a task execution may not be locally available which creates data-locality problem. Hence we propose an innovative data placement strategy based on dependency of data blocks across the nodes. Our strategy dynamically analyses the history log and establishes relationship between various tasks and blocks required for each task through Block Dependency Graph (BDG). Then Our CORE-Algorithm re-organizes the HDFS layout by redistributing the data blocks to give an optimal data placement, resulting in improved performance for Big Data sets in distributed environment. This strategy is tested in 20-node cluster with different real-world MR applications. The results conclude that proposed strategy reduces the query execution time by 23%, improves the data locality by 50.7%, compared to default.

Keywords


Cite This Article

APA Style
Vengadeswaran, , Balasundaram, (2019). Core – an optimal data placement strategy in hadoop for data intentitive applications based on cohesion relation. Computer Systems Science and Engineering, 34(1), 47-60. https://doi.org/10.32604/csse.2019.34.047
Vancouver Style
Vengadeswaran , Balasundaram . Core – an optimal data placement strategy in hadoop for data intentitive applications based on cohesion relation. Comput Syst Sci Eng. 2019;34(1):47-60 https://doi.org/10.32604/csse.2019.34.047
IEEE Style
Vengadeswaran and Balasundaram, “Core – An Optimal Data Placement Strategy in Hadoop for Data Intentitive Applications Based on Cohesion Relation,” Comput. Syst. Sci. Eng., vol. 34, no. 1, pp. 47-60, 2019. https://doi.org/10.32604/csse.2019.34.047

Citations




cc Copyright © 2019 The Author(s). Published by Tech Science Press.
This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 1837

    View

  • 1625

    Download

  • 2

    Like

Share Link