Intelligent Automation & Soft Computing

An Enhanced Decentralized Virtual Machine Migration Approach for Energy-Aware Cloud Data Centers

R. Jayamala*and A. Valarmathi

University College of Engineering, BIT Campus, Anna University, Tiruchirappalli, 620024, India
*Corresponding Author: R. Jayamala. Email: rjayamala10@gmail.com
Received: 30 August 2020; Accepted: 13 October 2020

Abstract: Cloud computing is an increasingly important technology to deliver pay-as-you-go online computing services. In this study, the cloud service provider permits the cloud user to pay according to the user’s needs. Various methods have been used to reduce energy utilization in the cloud. The rapid increase of cloud users has led to increased energy consumption and higher operating costs for cloud providers. A key issue in cloud data centers is their massive energy consumption to operate and maintain computing services. Virtual machine (VM) migration is a method to reduce energy consumption. This study proposes enhanced decentralized virtual machine migration (EDVMM), based on a linear prediction model, to decrease energy utilization in cloud data centers, reduce service-level agreement violations with a minimum number of migrated VMs, and enhance resource utilization. The enhanced decentralized approach is used to select the appropriate VMs, and prediction is used to determine the VMs for a host. This EDVM algorithm uses virtualization technology and migrates VMs from overloaded and under-loaded hosts to physical machines (PMs). The work was implemented and evaluated using CloudSim with 10 days of real workload trace provided by PlanetLab.

Keywords: VM consolidation; VM migration; data center; energy efficiency; service level agreement

1  Introduction

Cloud computing is increasingly important in the digital era [1], and it uses numerous services to satisfy users’ demands. It includes online data storage, infrastructure, and applications, implementing hardware and software tools through platform-as-a-service (PAAS), networking, virtualization, storage through infrastructure-as-a-service (IAAS), and access to software through the internet as software-as-a-service (SAAS) [2]. The data center forms the vital computing infrastructure in the cloud environment, and it encompasses a number of physical machines (PMs) each connected to a larger number of virtual machines (VMs). In distributed data centers, servers and networking equipment collect, store, and process data, which can be accessed by users. The continually increasing number of user applications available through cloud computing [3] has made efficient energy utilization a challenge in cloud data centers. VM migration, by which data are transmitted between hosts, is a promising technique to optimize energy efficiency.

Virtualization is the creation of a VM connected to a physical host [4], where a VM can be transferred between PMs using live migration. Data centers use VM consolidation to maximize energy saving. VMs can be migrated based on overloaded and underloaded conditions of physical machines to effectively improve resource utilization and quality of service (QoS) in SLAs [5]. When a PM’s load is higher or lower than capacity, the host is considered to be overloaded or underloaded, respectively. An overloaded host tends to cause performance degradation, and an underloaded host leads to poor resource utilization.

VM selection policy is used to determine VM migration based on the overload and underload conditions of physical hosts [6]. When a PM is considered underloaded, the VM is migrated and the active PM goes into sleep mode. This reduces the number of active PMs, which decreases energy utilization. VM allocation is a popular technique to reduce resource usage and increase energy efficiency, and VM allocation policy can be used to allocate VMs to suitable PMs [7]. Fig. 1 depicts the architecture of a typical VM migration model in the cloud environment.


Figure 1: Architecture of VM migration model

The proposed work is based on workload traces of the PlanetLab dataset. The remainder of this paper is organized as follows. Section 2 discusses related work. Section 3 describes VM migration, VM allocation, and selection policy. Section 4 explains the proposed enhanced decentralized virtual machine migration (EDVMM). Section 5 describes performance analysis. Section 6 presents our conclusions.

2  Related Work

Various methods and techniques have been used to reduce energy utilization in the cloud. Soltanshahi et al. [8] described cloud computing as a rapidly emerging technology in the online world. The rapid increase of cloud users has led to more energy consumption and higher operating costs. The krill herd algorithm has been used to optimize energy utilization and reduce the number of SLA violations in data centers. Zhou et al. [9] proposed an energy efficiency optimization of VM migration (EEOM) algorithm to reduce the number of active PMs, thereby reducing energy utilization through the choice of a trigger time, VM, and position of the host. The algorithm sets a double threshold (DT) value. EEOM algorithm consumes 7% energy and reduces the number of SLA violations by 13%. Shaw et al. [10] examined load balancing in cloud computing. Active load balancing and VM load balancing algorithms have been used to balance the VM load by distributing it equally to all VMs, which can reduce the response and processing time of a task.

Babu et al. [11] explained energy planning strategies, focusing on reducing energy and resource usage. Peak loads can cause scheduling errors, which affect the energy efficiency of scheduling algorithms. At peak time, scheduling is a complex task because there is no model to predict future resource utilization. The proposed iterative fractal prediction algorithm focuses on asset management while reducing energy consumption and maintaining QoS.

Zhu et al. [12] examined the traffic-aware migration problem, with a focus on reducing communication costs during VM migration. The VM migration process consists of two parts: the VM selection algorithm is based on greedy selection to select an appropriate VM, and it achieves the mappings between VMs and underutilized hosts. Here, the VM migration double auction mechanism is applied. Pietri et al. [13] discussed dynamic VM placement in view of both CPU and memory resources. A genetic algorithm was used to dynamically reallocate VMs, which reduces overload and underload of PMs to minimize SLA violations (SLAVs) and improve overall resource utilization. Monil et al. [14] used a reduced bin packing method and multi-pass optimization in VM placement to test for effective VM consolidation. To sustain energy QoS balance, an SLA-aware VM selection strategy was used to maintain the performance level. An underload detection method using an optimization phase and single pass and double pass (SPDP) optimizer function was used to further improve performance.

Malhotra et al. [15] used cloud, migration, and directory agents for efficient VM migration. This agent classification helps reduce migration time. A mobile agent retains the record of every occurrence of a VM in the cloud. The directory agent contains a list of all mobile cloud agents, and details on all VMs in various clouds. The migration agent initiates the migration process when a request is received from a cloud mobile user. To save energy, Akhter et al. [16] used an energy-aware algorithm to select a VM from an overloaded host. This helps to determine the maximum time frame for migration. The algorithm dynamically acquires a VM’s allocation, deallocation, and reallocation action from the physical server. Xu et al. [17] identified the migration cost (MC)-aware VM consolidation (MVC) problem as a multi-constraint optimization model by analyzing migration costs and long-lasting runtimes of VMs. The algorithm can reduce migration costs and ensure low energy consumption.

Shakya et al. [18] proposed a hybrid migration approach to achieve optimal VM migration performance. They used a mix of pre- and post-copy approaches based on the push-and-pull process to migrate a VM. During the migration cycle, both overloaded and underloaded servers help to move and pull a VM. The overloaded server moves the VM to the least loaded server, which often supports the process by migrating the VM. The migration process was replicated, and the result was compared to live migration based on CPU usage, network, and memory. A K-order mixed Markov model [19] was used to predict the load, and the results showed a reduction in energy use, along with fewer SLAVs and fewer VM migrations. The results were compared to those of traditional approaches, including a threshold-based load detection algorithm (classical-Markov) and a local regression robust (LRR) algorithm. The author used a cloud simulator tool for implementation.

3  Enhanced Decentralized Virtual Machine Migration Approach

EDVMM with a prediction approach has been used to reduce energy utilization in data centers. The enhanced decentralized approach [20] is used to select appropriate VMs, for which hosts are predicted. The algorithm uses virtualization technology and migrates VMs from overloaded hosts to underloaded hosts. An idle host can be swapped to low-power mode or shut down to reduce energy consumption. The approach uses a VM selection algorithm to select hosts and a VM allocation algorithm to migrate VMs to desired physical hosts. Fig. 2 shows the VM migration process.


Figure 2: VM migration process

3.1 Linear Regression (LR)

Linear regression is an arithmetic model that explains the relationship between variables [21], and has the form


where images is the expected vector and βu is the current capacity vector used for the PM. The regression coefficients are



where x is the mean of imagesimages, and y is the mean of images.

VM resource utilization prediction is based on a linear function that represents the relationship between the currently used capacity vector images and predicted used capacity vector images of the VM, i.e.,


3.2 Load Calculation

Load calculation is one of the most complex tasks in the cloud. A PM’s load depends on CPU usage, bandwidth utilization, and memory utilization [22]. We calculate the load based on CPU utilization. The algorithm seeks to eliminate the least-loaded PMs to reduce energy consumption. It migrates all VMs from the least-loaded to the most-loaded PMs to increase energy utilization. The proportionality usage of CPU is considered for VM allocation in each PM. Therefore, PM load is modeled as the summation of the resource utilization images [23],


where imagesis the ratio of used capacity to the PM’s total capacity of PM,


3.3 Threshold Calculation

Threshold values are used to calculate the overload and underload conditions of PMs. If the load on a physical machine exceeds the higher threshold value, then the host is perceived as over-utilized. If the load on a PM is less than the lower threshold value, then it is underloaded. The threshold value can be static or dynamic. The upper threshold value is the average load of all PMs available in the data center, and the lower threshold value is fixed at 0.1 [24], i.e.,



The upper threshold value is meant to save the surplus utilization of a CPU, which will help to reduce the number of SLAVs. The lower threshold is meant to encourage the switching of PMs to full sleep mode, which will decrease energy consumption.

3.4 VM Selection Policies

Incorrect VM selection can lead to a significant increase in overall migration time and downtime.

3.4.1 Minimum Migration Time (MMT)

The VM with the least migration time is picked for migration. The time period required for the migration of a VM between two PMs depends on the available network bandwidth and the VM’s memory utilization [25,26].

3.4.2 Maximum Load (MAXL)

Each PM has many VMs. Once the load of each PM and VM is calculated, the VM with the maximum load is migrated to another PM.

3.4.3 Minimum Load (MINL)

Each PM has many VMs. Once the load of each PM and VL is calculated, the VM with the minimum load is migrated to other PMs.

3.5 VM Migration

Each physical node sustains a load index in order to send and receive a load index from other peer nodes. In Case A, if the PM usage is less than the lower threshold value, then we migrate the VM. In Case B, if the PM utilization is more than the upper threshold value, then we migrate the VM. In Case C, if the PM is idle for a long time, then we move the PM to sleep mode. Fig. 3 shows how VM selection policy is used to pick the appropriate VM for migration.


Figure 3: VM selection algorithm

3.6 VM consolidation

VM consolidation is used to enhance energy efficiency [27]. It migrates running VMs from underutilized resources to other resources to reduce energy consumption, as shown in Fig. 4.


Figure 4: VM consolidation algorithm

4  Experimental Setup

In this experiment, 800 heterogeneous PMs were simulated. The simulated cloud data center consists of two dual-core servers: an HP ProLiant ML 110 G4 and an HP ProLiant ML 110 G5. The host configurations are listed in Tab. 1.

Table 1: Host configuration


Four types of single-core VMs were simulated in real-world scenarios: high-CPU medium instances, extra-large instances, small instances, and micro instances. The VM configuration information is listed in Tab. 2. This experiment used the workload from the CoMon project [28], which is described in Tab. 3.

Table 2: Virtual machine configuration


Table 3: Properties of PlanetLab dataset


5  Performance Evaluations

The proposed model aims to decrease the numbers of VM migrations and SLA violations. The performance calculation of the proposed work was calculated by the following metrics.

5.1 SLA Violations

A service-level agreement (SLA) is a commitment between a customer and service provider [29] for services as regards quality, availability, and responsibilities. SLA(V) measures the SLAs delivered to VMs in clouds, capturing SLAVs with over-utilization (SLAV(O)) and with migration (SLAV(M)) [30],


SLAV(O) is the amount of time that CPU use was encountered by active PMs as 100%,


where n is the number of physical machines and images is the total time.

images is CPU utilization leading to SLA violations.

images is Total images being the active state

SLAV(M) is the overall degradation of VM migration,


where S is the number of VMs, images is the performance estimate of degradation caused by migration VMm, and images is the total CPU capacity requested by the VMm.

Fig. 5 shows the SLA violations of the proposed EDVMM algorithm compared to the minimum migration time (MMT), maximum load (MAXL), and minimum load (MINL). The figure clearly shows that the algorithm results in a significant decrease in SLA violations.


Figure 5: SLA violations

5.2 Energy Consumption

A PM’s resource utilization is typically expressed by its CPU usage. When a PM is underutilized, it enters sleep mode, which saves energy [3133]. Fig. 6 shows that the proposed EDVMM algorithm is associated with less energy consumption than MMT, MAXL, and MINL.


Figure 6: Energy consumption

5.3 Number of Host Shutdowns

When a host is underutilized, VM migrations occur from the source to the destination host, and the host becomes idle and is moved to sleep mode. The host shutdown rate can be calculated as


where images is the number of active hosts at time I, and H is host shutdowns from time to time.

Fig. 7 depicts the number of host shutdowns. The proposed EDVMM model has fewer host shutdowns compared to MMT, MAXL, and MINL.


Figure 7: No. of host shutdowns

5.4 Number of Virtual Machine Migrations

VM migrations can be performed during VM placement, and the migrated number of VMs is


where N is the number of hosts, G is VMs placement at host I, and Migi(G) is the number of host migration i from time t1 to time t2. Fig. 8 shows that the proposed EDVMM algorithm has fewer VM migrations than MMT, MAXL, and MINL.


Figure 8: No. of VM migrations

6  Conclusion

In this paper, the proposed enhanced decentralized VM migration algorithm selects the most appropriate VM from a different host to perform an efficient VM migration using a threshold value. The proposed algorithm was compared to VM selection methods such as MMT, MINL, and MAXL, using the CloudSim simulation toolkit. The results show that the proposed algorithm achieves better energy savings, reduces the number of SLA violations with minimum VM migrations, and improves resource utilization. The proposed work was implemented using the real-world PlanetLab workload by considering only the CPU utilization. Future work will involve energy consumption by considering other parameters such as memory and disk space, and performing comparative analyses to assess our system against other methods using real traces.

Funding Statement: The author(s) declares that no specific funding was received for this research work.

Conflicts of Interest: The author(s) declares no known financial interests or personal relationships that could appear to influence the work reported in this paper.


  1.  1.  H. A. Khattak, H. Arshad, S. U. Islam, G. Ahmed and S. Jabbar. (2019). “Utilization and load balancing in fog servers for health applications,” EURASIP Journal on Wireless Communications and Networking, vol. 2019, no. 1, pp. 1–12.
  2.  2.  L. Mohan, R. Pandey, S. Bisht and J. Pant. (2017). “A comparative study on SaaS, PaaS and IaaScloud delivery model in cloud computing,” International Journal on Emerging Technologies, vol. 8, no. 1, pp. 158–160.
  3.  3.  R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg and I. Brandic. (2009). “Cloud computing and emerging it platforms: Vision, hype, and reality for delivering computing as the 5th utility,” Future Generation Computer Systems, vol. 25, no. 6, pp. 599–616.
  4.  4.  X. M. Huang, R. Yu, J. W. Kang, Z. Q. Xia and Y. Zhang. (2018). “Software defined networking for energy harvesting Internet of Things,” IEEE Internet of Things Journal, vol. 5, no. 3, pp. 1389–1399.
  5.  5.  H. X. Li, W. J. Li, S. G. Zhang, H. D. Wang and Y. Pan. (2019). “Page-sharing-based virtual machine packing with multi-resource constraints to reduce network traffic in migration for clouds,” Future Generation Computer Systems, vol. 96, pp. 462–471.
  6.  6.  A. Motawani, V. Patel and M. V. Patel. (2015). “Power and QoS aware virtual machine consolidation in green cloud data,” International Journal of Electrical, Electronics and Computer Engineering, vol. 4, pp. 93–96.
  7.  7.  T. P. Shabeera, S. D. Madhu Kumar, S. M. Salam and K. Murali Krishnan. (2017). “Optimizing VM allocation and data placement for data-intensive applications in cloud using ACO metaheuristic algorithm,” Engineering Science and Technology, an International Journal, vol. 20, no. 2, pp. 618–628.
  8.  8.  M. Soltanshahi, R. Asemi and N. Shafiei. (2019). “Energy-aware virtual machines allocation by krill herd algorithm in cloud data centers,” Science Direct Heliyon, vol. 5, no. 7, pp. 1–6.
  9.  9.  Z. Zhou, J. Yu, F. Li and F. Yang. (2018). “Virtual machine migration algorithm for energy efficiency optimization in cloud computing,” Concurrency and Computation: Practice and Experience, vol. 30, no. 24, pp. 1–10.
  10. 10. S. B. Shaw. (2017). “Balancing load of cloud data center using efficient task scheduling algorithm,” International Journal of Computer Applications, vol. 159, no. 5, pp. 1–5.
  11. 11. G. P. Babu and A. K. Tiwari. (2019). “Energy efficient scheduling algorithm for cloud computing systems based on prediction model,” International Journal of Advanced Networking and Applications, vol. 10, no. 5, pp. 4013–4018.
  12. 12. J. Zhu, J. Wang, Y. Zhang and Y. Jiang. (2019). “Virtual machine migration method based on load cognition,” Soft Computing, vol. 23, no. 19, pp. 9439–9448.
  13. 13. I. Pietri and R. Sakellariou. (2016). “Mapping virtual machines onto physical machines in cloud computing: A survey,” ACM Computing Surveys (CSUR), vol. 49, no. 3, pp. 1–30.
  14. 14. M. A. H. Moniland and A. D. Malony. (2017). “QoS-aware virtual machine consolidation in cloud data center,” in IEEE Int. Conf. on Cloud Engineering, Canada, pp. 81–87.
  15. 15. G. Singh, M. Malhotra and A. Sharma. (2019). “An agent based virtual machine migration process for cloud environment,” in 4th Int. Conf. on Internet of Things: Smart Innovation and Usages (IoT-SIUGhaziabad, India, pp. 1–4.
  16. 16. N. Akhter, M. Othman and R. K. Naha. (2018). “Energy-aware virtual machine selection method for cloud data center resource allocation”, arXiv preprint arXiv:1812.08375, pp. 1–16.
  17. 17. H. Xu, Y. Liu, W. Wei and Y. Xue. (2019). “Migration cost and energy-aware virtual machine consolidation under cloud environments considering remaining runtime,” International Journal of Parallel Programming, vol. 47, no. 3, pp. 481–501.
  18. 18. D. Garg and P. C. Nayak. (2018). “Hybrid live VM migration: An efficient live VM migration approach in cloud computing,” in Springer Int. Conf. on Advanced Informatics for Computing Research, Singapore, vol. 955, pp. 600–611.
  19. 19. Y. Zhang, H. Wen, S. Zhou, Z. Wang, R. Wang et al. (2018). , “EEDVMI: Energy-efficient dynamic virtual machines integration,” Mobile Networks and Applications, vol. 25, no. 3, pp. 997–1007.
  20. 20. X. Wang, X. Liu, L. Fanand and X. Jia. (2013). “A decentralized virtual machine migration approach of data centers for cloud computing,” Mathematical Problems in Engineering, vol. 2013, no. 878542, pp. 1–10.
  21. 21. L. Li, J. Dong, D. Zuo and J. Wu. (2019). “SLA-aware and energy-efficient VM consolidation in cloud data centers using robust linear regression prediction model,” IEEE Access, vol. 7, pp. 9490–9500.
  22. 22. C. M. S. Magurawalage, K. Yang, L. Hu and J. Zhang. (2014). “Energy-efficient and network-aware offloading algorithm for mobile cloud computing,” Computer Networks, vol. 74, pp. 22–33.
  23. 23. F. Farahnakian, T. Pahikkala, P. Liljeberg, J. Plosila, N. T. Hieu et al. (2019). , “Energy-aware VM consolidation in cloud data centers using utilization prediction model,” IEEE Transactions on Cloud Computing, vol. 7, no. 2, pp. 524–536.
  24. 24. D. Patel, R. K. Guptaand and R. K. Pateriya. (2019). “Energy-Aware prediction-based load balancing approach with VM migration for the cloud environment,” Springer in Data, Engineering and Applications, Singapore, pp. 59–74.
  25. 25. S. Namdev, N. Sain and A. K. Rai. (2015). “Improved minimum migration time VM selection policy for cloud data center,” International Journal of Application or Innovation in Engineering and Management, vol. 4, no. 4, pp. 157–160.
  26. 26. H. X. Li, W. J. Li, H. D. Wang and J. X. Wang. (2018). “An optimization of virtual machine selection and placement by using memory content similarity for server consolidation in cloud,” Future Generation Computer Systems, vol. 84, pp. 98–107.
  27. 27. F. Farahnakian, T. Pahikkala, P. Liljeberg, J. Plosila and H. Tenhunen. (2015). “Utilization prediction aware VM consolidation approach for green cloud computing,” in IEEE 8th Int. Conf. on Cloud Computing, New York, pp. 381–388.
  28. 28. M. G. Arani, A. A. Rahmanian, M. Shamsi and A. RasouliKenari. (2018). “A learning-based approach for virtual machine placement in cloud data centers,” International Journal of Communication Systems, vol. 31, no. 8, pp. 1–18.
  29. 29. Y. S. Luo, K. Yang, Q. Tang, J. M. Zhang, P. Li et al. (2016). , “An optimal data service providing framework in cloud radio access network,” EURASIP Journal on Wireless Communications and Networking, vol. 2016, pp. 1–11.
  30. 30. A. Mosaand and R. Sakellariou. (2018). “Dynamic tuning for parameter-based virtual machine placement,” in 17th Int. Sym. on Parallel and Distributed Computing (ISPDCpp. 38–45.
  31. 31. A. Beloglazov, J. Abawajy and R. Buyya. (2012). “Energy-aware resource allocation heuristics or efficient management of data centers for cloud computing,” in Future Generation Computer Systems (FGCS), vol. 28. The Netherlands: Elsevier Science, pp. 755–768.
  32. 32. H. Huang, S. F. Zeng, A. K. Sangaiah and J. Wang. (2019). “Beamforming aided SSK modulation for MIMO system with energy harvesting,” Journal of Intelligent & Fuzzy Systems, vol. 36, no. 5, pp. 4017–4023.
  33. 33. M. Long and X. Xiao. (2018). “Outage performance of double-relay cooperative transmission network with energy harvesting,” Physical Communication, vol. 29, pp. 261–267.
images This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.