Open Access iconOpen Access


Optimizing Big Data Retrieval and Job Scheduling Using Deep Learning Approaches

Bao Rong Chang1, Hsiu-Fen Tsai2,*, Yu-Chieh Lin1

1 Department of Computer Science and Information Engineering, National University of Kaohsiung, Kaohsiung, Taiwan
2 Department of Fragrance and Cosmetic Science, Kaohsiung Medical University, Kaohsiung, Taiwan

* Corresponding Author: Hsiu-Fen Tsai. Email: email

(This article belongs to the Special Issue: Hybrid Intelligent Methods for Forecasting in Resources and Energy Field)

Computer Modeling in Engineering & Sciences 2023, 134(2), 783-815.


Big data analytics in business intelligence do not provide effective data retrieval methods and job scheduling that will cause execution inefficiency and low system throughput. This paper aims to enhance the capability of data retrieval and job scheduling to speed up the operation of big data analytics to overcome inefficiency and low throughput problems. First, integrating stacked sparse autoencoder and Elasticsearch indexing explored fast data searching and distributed indexing, which reduces the search scope of the database and dramatically speeds up data searching. Next, exploiting a deep neural network to predict the approximate execution time of a job gives prioritized job scheduling based on the shortest job first, which reduces the average waiting time of job execution. As a result, the proposed data retrieval approach outperforms the previous method using a deep autoencoder and Solr indexing, significantly improving the speed of data retrieval up to 53% and increasing system throughput by 53%. On the other hand, the proposed job scheduling algorithm defeats both first-in-first-out and memory-sensitive heterogeneous early finish time scheduling algorithms, effectively shortening the average waiting time up to 5% and average weighted turnaround time by 19%, respectively.


Cite This Article

APA Style
Chang, B.R., Tsai, H., Lin, Y. (2023). Optimizing big data retrieval and job scheduling using deep learning approaches. Computer Modeling in Engineering & Sciences, 134(2), 783-815.
Vancouver Style
Chang BR, Tsai H, Lin Y. Optimizing big data retrieval and job scheduling using deep learning approaches. Comput Model Eng Sci. 2023;134(2):783-815
IEEE Style
B.R. Chang, H. Tsai, and Y. Lin "Optimizing Big Data Retrieval and Job Scheduling Using Deep Learning Approaches," Comput. Model. Eng. Sci., vol. 134, no. 2, pp. 783-815. 2023.

cc This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 1755


  • 830


  • 0


Share Link