Yi Liang1,*, Shaokang Zeng1, Xiaoxian Xu2, Shilu Chang1, Xing Su1
CMC-Computers, Materials & Continua, Vol.66, No.2, pp. 1697-1717, 2021, DOI:10.32604/cmc.2020.012513
Abstract Spark is the most popular in-memory processing framework for big data analytics. Memory is the crucial resource for workloads to achieve performance acceleration on Spark. The extant memory capacity configuration approach in Spark is to statically configure the memory capacity for workloads based on user’s specifications. However, without the deep knowledge of the workload’s system-level characteristics, users in practice often conservatively overestimate the memory utilizations of their workloads and require resource manager to grant more memory share than that they actually need, which leads to the severe waste of memory resources. To address the above issue, SMConf, an automated memory… More >
Login
Register
