Multi-Time Scale Optimization Scheduling of Data Center Considering Workload Shift and Refrigeration Regulation

1  Background and Motivation

Artificial Intelligence (AI) has experienced explosive growth in recent years and plays a pivotal role in improving productivity and driving economic development. Under this background, data has increasingly emerged as a primary factor of production, and society is rapidly transitioning into the digital economy era that stimulates unprecedented demand for computational capacity. Data centers, functioning as the backbone for data processing and storage, are thus emerging as vital socio-economic infrastructure during this transition [1].

The data center industries are facing huge energy challenges due to their high power consumption and carbon emissions. Worldwide, data centers consume approximately 460 TWh of electricity annually, comprising nearly 2% of the global electricity use [2]. With world’s over 8000 data centers, the United States, Europe, and China host a significant portion of these facilities, which are 33%, 16% and 10%, respectively. In 2022, US data centers consumed 200 TWh of electricity, accounting for about 4% of the national electricity consumption. China’s data center electricity consumption reached 216.6 TWh, roughly 2.6% of the societal electricity consumption in 2021. The CO2 emissions from data centers reached 135 million tons, approximately 1.14% of the national emissions [3]. Emerging AI models with billions of parameters demand enormous energy during their training, and as AI technology evolves, data centers’ power consumption and carbon emissions will continue to grow [4]. In the pursuit of carbon neutrality, the integration of data centers with renewable energy sources has emerged as an inevitable trend [5]. China’s ‘Channeling computing resources from the east to the west’ policy was introduced against this backdrop, aiming to optimize the data-center layout and promote green energy utilization by leveraging the abundant wind and solar resources of the western regions [6].

To effectively integrate the intermittent renewable energy, it is essential to extensively analyze the operation flexibility of data centers [7]. Fig. 1 illustrates the structural overview of a green data center micro-grid. The information technology (IT) equipment mainly includes the servers that provide computing resource for workloads. The refrigeration system provides cooling resources to extract the heat generated from the IT equipment. The power infrastructure connects the wind, photovoltaic (PV) power system and main-grid in supply side and IT equipment, refrigeration system in power demand side.

Figure 1: Structure of green data center

Data centers manage an exponentially increasing volume of computational workloads driven by continuous enhancements in processing capabilities. As computing workloads migrate across the network in temporal and spatial domains, the associated power consumption of the data center correspondingly varies. Thus, computing workloads exhibit significant spatiotemporal adjustable characteristics [8]. Additionally, computer room temperature can fluctuate within predefined limits, and the temperature change exhibits a temporal delay relative to the cooling power change. By dynamically modulating the refrigeration power within acceptable temperature bounds, fluctuations in renewable energy generation can be effectively managed [9]. Consequently, the refrigeration system functions as an auxiliary flexible resource for data center micro-grid operations.

This study concentrates on formulating power regulation models for delay-tolerant computing workloads and refrigeration system within data centers, and incorporating these flexible regulation models into power scheduling framework of data centers to increase the operational economy and maximize the utilization of renewable energy. Section 2 reviews pertinent literature to delineate the existing state of research and to identify critical knowledge gaps.

2  Literature Review

In recent studies, leveraging the spatiotemporal flexibility of computing workloads in data centers to promote renewable energy utilization has emerged as a new research focus. Chen et al. [10] analyzed the topology of computing power networks from an energy perspective, and used price leverage to guide the spatial shift of workloads from interconnected data centers to promote the renewable energy utilization and improve the grid resilience. Yang et al. [11] proposed a spatial migration mechanism of computing workloads based on the spatiotemporal distribution complementarity of renewable energy in multiple regions around the world, and conducted simulations using Google’s interconnected multiple data centers to minimize carbon emissions. However, the aforementioned research focuses mainly on the spatial redistribution of computing workloads across multiple data centers, with limited attention to the temporal shift of workloads within individual data centers.

The flexibility of computing workloads within an individual data center primarily manifests in their temporal transferability. According to the length of response delay, workloads can be categorized into delay-sensitive and delay-tolerant types [12]. Delay-sensitive workloads primarily refer to ‘online tasks’, or ’interactive workloads’ that provide real-time feedback to users. Delay-tolerant workloads, by contrast, refer to ‘offline tasks’ or ‘batch processing workloads’ that execute a series of jobs based on a computer program without human intervention; Their maximum response time can range from several minutes to days [13]. By dynamically adjusting the delay-tolerant workloads’ start time and the amount of computation load for each time interval within the response deadline, the power consumption of workloads can be controlled. This study concentrates on the temporal flexibility of workloads within an individual data center.

Studies regarding the temporal flexibility modeling of delay-tolerant workloads within an individual data center have been conducted. Liu et al. [14] proposed a power consumption model for the delay-tolerant workloads and integrated it with a renewable power supply model. The optimal plan of generator units’ outputs and workloads scheduling was obtained with the objective of minimizing operation cost of data centers. Cupelli et al. [15] simulated the power consumption of delay-tolerant workloads considering their arrival, queuing, and execution process. By coordinating workloads scheduling with cooling systems and energy-storage devices, they demonstrated reductions in overall data-center energy costs. Kwon [16] analyzed the time-shift characteristics of a simple delay-tolerant workload and developed a flexible mechanism model to shift workloads power consumption towards periods of high renewable generation. The studies above highlight the potential of delay-tolerant workloads to provide flexible regulation in individual data centers.

However, existing studies have primarily focused on modeling single, simple time-shiftable workload and haven’t comprehensively accounted for multiple time-shiftable workload types with distinct operational characteristics and shifting behaviors. Comprehensive modeling of multiple delay-tolerant workloads, and systematic integration of these models into data center power dispatch frameworks, remain insufficiently explored.

Furthermore, the refrigeration system serves as an additional flexible resource in data center operations. Data centers commonly employ refrigeration systems to maintain suitable thermal conditions. With advancements in technology, the temperature requirements for server clusters have become less stringent. The ASHRAE introduced the ‘Data Center Environmental Thermal Guidelines’ in 2014, extending the design ranges for maximum allowable computer room temperatures to 32°C, 35°C, 40°C, and 45°C across A1–A4 classifications. Considering the temperature fluctuation range and thermal inertia, existing research investigated the feasibility of utilizing refrigeration modulation to optimize data center operation. Tang et al. [17] established the first-order equivalent thermal parameter model to analyze the thermal load dynamics of the air condition system in base stations. By incorporating temperature variation boundaries, they employed day-ahead demand response strategies utilizing start-stop control to manage cooling and heating loads. Zhu et al. [18] formulated a temperature response model considering thermal inertia within data center environments and proposed dynamic temperature regulation strategies to optimize real-time renewable energy utilization. These investigations demonstrate the potential of using temperature and refrigeration modulation to enhance power scheduling efficiency in data centers. However, they don’t address the coordination of computing workloads time-shift and refrigeration modulation in data center operational management.

In light of the deficiencies, this research aims to provide a power scheduling framework for data center by incorporating the computing workloads shift and refrigeration regulation. To this aim, this work first analyzes the time-shift characteristics of three kinds of typical delay-tolerant computing workloads and establishes the corresponding time-shift models. Then, this work quantitatively examines the heat transfer process of data center and builds the time-variant model of computer room temperature. Last, this work proposes a two-stage multi-time scale optimization scheduling framework for the green data center, where the time-shift models of workloads are incorporated into the day-ahead optimization scheduling, and the refrigeration power is set as the control variable in the intraday scheduling. The proposed multi-time scale energy management method is applied to a data center to verify its effectiveness.

3  Methods

3.1 Time-Shift Models of the Delay-Tolerant Workloads

This section introduces three kinds of delay-tolerant workloads, i.e., Long-running non-interruptible, long-running interruptible, and short-running workloads, and models the time-shifting processes of the workloads.

3.1.1 Workloads Classification and Time-Shift Characteristics Analysis

Time-shiftable workloads are categorized into long-running and short-running workloads, distinguished by their processing time length. Long-running workloads include continuous non-interruptible and interruptible workloads based on their interruptibility.

Analysis of large internet data centers shows that processor resource consumption of computing workloads follows a heavy-tailed distribution, indicating that a small portion of long-running workloads consume a significant amount of processor resources [19]. These energy-intensive long-running workloads present opportunities for power regulation. Additionally, analysis reveals that most workloads in Google’s cluster last only a few minutes [19], while over 90% of batch jobs in Alibaba’s data cluster run for less than 15 min [20]. Short-running computing tasks, when combined, can effectively help mitigate fluctuations in renewable energy.

•   Long-running non-interruptible workload

Some workloads, e.g., continuous integration and continuous deployment jobs (CI/CD), test suites, compilation jobs, database migration, and backups, can’t be paused or interrupted once execution begins and must run continuously until completion. Workload with this working characteristic are identified as ‘long-running non-interruptible workload’.

The temporal mitigation pattern for this kind of workload is to shift the overall workload within a certain time range while keeping the original shape of the computing power load curve unchanged.

•   Long-running interruptible workload

Certain workloads, e.g., machine learning training, block-chain mining, protein folding, or long-running scientific simulations, can be paused or interrupted during execution and resumed at an appropriate time. Workload with this working characteristic is regarded as ‘long-running interruptible workload’.

According to its time-shift characteristics, long-running interruptible workload can be viewed as a holistic task comprised of multiple sequential subtasks. The shift strategy is to adjust each subtask forward or backward to a certain time point within a specified time frame, while ensuing the sequential order dependency among the subtasks.

•   Short-running workloads

Some workloads, including function as a service, scheduled data backups, log cleaning, report generation, email or notification sending, etc., typically require brief execution time and don’t demand immediate response. These workloads can tolerate delays ranging from minutes to several hours, classifying them as short-running workloads.

Short-running workloads exhibit two defining characteristics: short execution duration and high task volume. When numerous short-running workloads aggregate, they form a short-running workload cluster, which presents significant potential for power regulation. The time-shifting mechanism for managing these workloads involves rescheduling all or part of the short-running tasks from their original time slots to alternative periods in future time window for execution.

3.1.2 Modelling of Long-Running Non-Interruptible Workload

Set the day-ahead scheduling period to 24 h, and define the scheduling time step as Δt and the number of daily scheduling periods as T. Introduce Faori to denote the original computing power demand time series of the long-running non-interruptible workload. Faori is a constant vector with dimensions of 1*T, represented as follows:

Faori=[xa1,xa2,xa3,…xat,…,xaT−1,xaT](1)

where xa1, xa2, xat are the computing power demand values of the long-running non-interruptible workload at the time point of 1, 2, and t. In vector Faori, there exists a continuous period of computing power demand values, and the elements before and after these continuous periods are zero. Thus, the vector Faori can be more specifically written as:

Faori=[0,0,…,fa1,fa2,…fak,…,faK,…,0,0](2)

where fa1 represents the computing power demand value at the starting node of the long-running continuous workload; fa2, fak, faK, is the computing power demand value at the 2nd, kth, Kth time points following the starting node, respectively. K represents the processing time of the long-running non-interruptible workload execution, which is a constant.

The mitigation pattern for the long-running non-interruptible workload is to shift the entire workload along the time axis. Once the start time of the workload is determined, the computing power demand for each subsequent time point can be uniquely ascertained. Therefore, the pivotal parameter for the time-shift modelling of long-running non-interruptible workload is the time-shift vector of start node.

Use Ua to represent the time-shift vector of start node of the long-running non-interruptible workload. Ua is the variable with a dimension of 1*T, represented as Eq. (3):

Ua=[ua1,ua2,ua3…,uat,…,uaT−1,uaT](3)

where the element uat represents whether the start node of the long-running non-interruptible workload is located at a certain time point. When uat is 1/0, it signifies that the start node of the workload is/isn’t positioned within the t time point.

For Ua, only one element in the vector can have a value of 1, thus the sum of all elements is constrained to 1, denoted by:

∑t=1Tuat=1(4)

The time at which the start node is positioned is denoted as Uaindex, which is expressed as Eq. (5):

Uaindex=[1,2,3,…,T]transpose∗Ua(5)

The long-running non-interruptible workload has the earliest start time point te and the latest start time point tl, allowing the start node of the workload to shift within [te, tl]. The constraint is expressed as:

te≤Uaindex≤tl(6)

The relation between the computing power demand time series of workload after shifting Famov and the time-shift vector of the workload start node Ua is expressed as Eq. (7):

Famov=Ua∗Mamat(7)

where the dimension of Famov is 1*T; Mamat is a constructed auxiliary constant matrix, with a dimension of T*T, as shown in Eq. (8).

Mamat=[ma1,1ma1,2ma1,3…ma1,t…ma1,T−1ma1,Tma2,1ma2,2ma2,3…ma2,t…ma2,T−1ma2,Tma3,1ma3,2ma3,3…ma3,t…ma3,T−1ma3,T……………………mat,1mat,2mat,3…mat,t…mt,T−1mat,T……………………maT−1,1maT−1,2maT−1,3…maT−1,t…maT−1,T−1maT−1,TmaT,1maT,2maT,3…maT,t…maT,T−1maT,T](8)

Mamat can be more specifically written as Eq. (9). In this matrix, elements from ‘column k, row k’ to ‘row k, column k + K − 1’ (k = 1, 2, …, T + 1 − K) are filled with the computing power demand values corresponding to the start node and subsequent nodes of the long-running uninterruptible workload, and the elements at the remaining positions are 0.

Mamat=[fa1fa2…faK0…000fa1…faK−1faK…0000…………0000…………00……………………00…………0000…fa1fa2…faK000…0fa1…faK−1faK](9)

3.1.3 Modelling of Long-Running Interruptible Workload

Introduce Fbori to denote the original time series of computing power demand of long-running interruptible workload. Fbori is a constant vector with a dimension of 1*T, represented as follows:

Fbori=[yb1,yb2,yb3,…ybt,…ybT−1,ybT](10)

where ya1, ya2, yat are the computing power demand values of the long-running interruptible workload at the time point of 1, 2, and t.

According to the time-shift characteristics of long-running interruptible workload, this study considers the long-running interruptible workload as a holistic task composed of multiple subtasks in sequence. The duration of each subtask is consistent with the scheduling time step Δt. Record the start node of the long-running interruptible workload as the 1st subtask, and the subtasks in subsequent time points as the 2nd,…, kth and Kth subtask in sequence, and the corresponding computing demand value as fb1, fb2, …, fbk, …, fbK. K represents the processing time of long-running interruptible workload execution, which is a constant. The constant vector Fbmat can be constructed from fb1 to fbK, as illustrated below:

Fbmat=[fb1,fb2,…fbk,…,fbK](11)

Hence, the original computing demand time series of the long time interruptible workload Fbori can be more specifically written as:

Fbori=[0,0,…,fb1,fb2,…,fbk,…,fbK,…,0,0](12)

In this vector, there exists a period of time with computing power demand value, and the elements before and after this period are zero.

The mitigation pattern for the long-running interruptible workload is scheduling individual subtasks to new time points within a specified time frame while preserving their sequential dependencies. Once the execution time of each subtask is determined, the computing power demand for the workload can be uniquely ascertained. Thus, the key parameter for time-shift modelling of long-running interruptible workload is the time-shift matrix of its subtasks.

Use ubt,k to indicate whether the kth subtask is moved to time point t, with a value of 0 or 1. The time shift matrix variable Ub with a dimension of T*K is formed, which is represented as:

Ub=[ub1,1ub1,2⋯ub1,k⋯ub1,Kub2,1ub2,2⋯ub2,k⋯ub2,Kub3,1ub3,2⋯ub3,k⋯ub3,K⋯⋯⋯⋯⋯⋯ubt,1ubt,2⋯ubt,k⋯ubt,K⋯⋯⋯⋯⋯⋯ubT−1,1ubT−1,2⋯ubT−1,k⋯ubT−1,KubT,1ubT,2⋯ubT,k⋯ubT,K](13)

For each column in matrix Ub, the sum of the element values in each row is 1, represented by Eq. (14), implying that a subtask can only transfer from one time point to another, not to multiple time points.

∑t=1Tubt,k=1(k=1,2,…,K)(14)

The matrix of time points where subtasks locate after shifting is denoted as Ubindex, which is a variable with a dimension of 1*K, expressed as Eq. (15). Ubindex is calculated using Eq. (16). ubindex,k represents the time point where each subtask locates after shifting. Eq. (17) enforces that the time position at which the (k + 1)-th subtask moves to should be greater than the time position at which the kth subtask moves to.

Ubindex=[ubindex,1,ubindex,2,…,ubindex,k,…,ubindex,K](15)

Ubindex=[1,2,3,…,T]∗Ub(16)

ubindex,k+1−ubindex,k≥1(17)

The computing power demand time series of each subtask after shifting is denoted as Fbmov, which is a variable with a dimension of 1*T, expressed as below:

Fbmov=[Ub∗(Fbmat)transpose]transpose(18)

3.1.4 Modelling of Short-Running Workloads

Introduce Fcori to denote the original computing power demand time series of short-running workloads. Fcori is a constant vector with a dimensions of 1*T, represented as Eq. (19). Use Fcmov to denote the computing power demand time sequence of the short-running workloads after shift. Fcori is a variable with a dimension of 1*T, represented Eq. (20):

Fcori=[fc1,fc2,…fck,…,fcT](19)

Fcmov=[fc∗1,fc∗2,…fc∗t,…,fc∗T](20)

where fc1, fc2, fck are the computing power demand values of short-running workloads at original status at time point 1, 2, and k; fc∗1, fc∗2, fc∗t are the computing power demand values of short-running workloads after shifting at time point 1, 2, and t.

The time shifting pattern for short-running workloads is shifting all or part of the tasks from the original time periods to new time periods. The key parameter for the time shift modelling of short-running workloads is the time shift matrix of the inflow and outflow amount of computing power demand at each time point.

Use uct,k to represent the amount of computing power demand of short-running workloads that is transferred from the original time point k to another time point of t. The time shift matrix variable of the inflow and outflow amount of computing power demand Uc with a dimension of T*T is formed, which is expressed as:

uc=[uc1,1uc1,2…uc1,k…uc1,T−1uc1,Tuc2,1uc2,2…uc2,k⋯uc2,T−1uc2,T…………………uct,1uct,2…uct,k…uct,T−1uct,Tuct+1,1uct+1,2…uct+1,k…uct+1,T−1uct+1,Tuct+2,1uct+2,2…uct+2,k…uct+2,T−1uct+2,Tuct+3,1uct+3,2…uct+3,k…uct+3,T−1uct+3,Tuct+4,1uct+4,2…uct+4,k…uct+4,T−1uct+4,T…………………ucT−1,1ucT−1,2…ucT−1,k…ucT−1,T−1ucT−1,TucT,1ucT,2…ucT,k…ucT,T−1ucT,T](21)

Uc should meet the following constraint:

0≤Uc≤Ccmat∗M(22)

In this formula, Ccmat is a constructed constant matrix with the same dimension as the variable Uc. Ccmat is expressed by:

Ccmat=[10⋯⋯⋯1111⋯⋯⋯11⋯⋯⋯⋯⋯⋯…11⋯⋯⋯1111⋯⋯⋯0111⋯⋯⋯0001⋯⋯⋯0000⋯⋯⋯00⋯⋯⋯⋯⋯⋯⋯00⋯⋯⋯1000⋯⋯⋯11](23)

To illustrate the specific form of Ccmat, we first introduce a constant value H to represent the maximum deferrable time periods of short-running workloads. The short-running workloads of the original time points are allowed to be fully or partially transferred to the 1st to Hth time point thereafter. In Ccmat, when k = 1, 2, …, T − H, the elements from ‘column k, row k,’ to ‘column k, row k + H’, and when k = T − H + 1, T − H + 2, …, T, the elements from ‘column k, row k’ to ‘column k, row T’ and from ‘column k, row 1’ to ‘column k, row H − T + k’, are filled with the value of 1, and the elements at the remaining positions are 0. Set the maximum deferrable time periods H to be 11, the Ccmat can be specifically written as Eq. (23). The vector Ccmat can be changed as the parameter H varies.

M is a constant, equal to the peak computing power demand fcmax of the short-running workloads. Eq. (22) indicates that the transferred computing power demand from the original time point to the new time period should be less than fcmax and greater than 0.

Another constraint for Uc is that, for the kth column, the sum of all the elements equals to the original computing power demand fck at time point k. For the tth column, the accumulated elements of all the rows equals to the computing power demand at time point t of workloads after shifting fc∗t.

fck=∑t=1Tuct,k(t=1,2,…,T)(24)

fc∗t=∑k=1Tuct,k(k=1,2,…,T)(25)

To prevent invalid shifting, a penalty cost coefficient matrix Dcmat with dimension of T*T is established, as expressed in Eq. (26). Setting the unit penalty cost coefficient for the transfer amount of computing power demand at all possible time points to be 0.05 ¥/kWh, when k = 1, 2, …, T − H, the elements in positions of Dcmat from the ‘column k, row k + 1’ to ‘column k, row k + H’, and when k = T – H + 1,T – H + 2, …, T − 1, elements from the ‘column k, row k + 1’ to ‘column k, row T’ and from ‘column k, row 1,’ to ‘column k, row H − (T − k)’, and when k = T, elements from the ‘column k, row 1’ to ‘column k, row H’ are filled with 0.05, and the other elements are set to be 0. One can adjust the matrix Dcmat according to concrete parameter value of the unit penalty cost coefficient.

Dcmat=[00………0.050.050.050………0.050.05…………………0.050.05………0.050.050.050.05………00.050.050.05………0000.05………0000………00…………………00………0000………0.050](26)

The penalty cost function Cdata is expressed as:

Cdata=∑t=1T∑k=1TUc.∗DcmatFcpumax∗Ecpumax∗Δt(27)

where Fcpumax represents the maximum computing power of the CPU chip, which is 1600 GFLOPS in this paper, as indicated in Table 1. Ecpumax represents the maximum power consumption of the CPU used in this study, which is 150 W, as indicated in Table 1.

3.1.5 Relation of Computing Power and Electricity Power

Computing power refers to a device’s capability to perform data processing and generate specific outputs. The computing power is implemented through various computing chips such as CPU, GPU, FPGA, ASIC, which are carried by computers, servers, and other systems. The standard measurement for computing power is the number of FLoating-point Operations executed Per Second (FLOPS).

The computing power of a chip is determined by three key factors: the number of computing cores of the chip Ncore, the core frequency fcore, and the double-precision floating-point operands per clock cycle Zcore of the core. The computing power Fchipmax of the chip follows the formula:

Fchipmax=Ncore∗fcore∗Zcore(28)

The computing power capacity of the data center Fdcmax using CPU chips is expresses as:

Fdcmax=Nrack∗Nserver∗NCPU∗Fcpumax(29)

where Nrack is the number of racks in the datacenter. Nserver is the number of servers per rack. NCPU is the number of CPU chips per server. Fcpumax is the maximum computing power of the CPU chip studied in this paper.

Based on the assumption that: (1) all servers in the data center are homogeneous; (2) all servers in the computer room are turned on. The power consumption of servers can be modelled as Eq. (30) [21,22]:

Edata=Eidle+(Epeak−Eidle)∗ucpu(30)

where ucpu is the processor utilization rate. Eidle represents the power of servers when ucpu is zero, Epeak represents the power of servers when ucpu reaches 100%.

The formula for calculating CPU utilization is ucpu, expressed as:

ucpu=FdataFdcmax(31)

where Fdata is the computing power demand of workloads.

This model provides a quantitative approach to estimate the electricity power consumption of workloads based on their computing power demand.

3.2 Time-Variant Model of Computer Room Temperature

Data center computer room temperature is influenced by multiple factors including server cluster heat generation, environmental maintenance structure, heat load, cooling power, and outdoor temperature. To simplify the problem-solving process when constructing a thermal model for the data center, the heat transfer through the walls is treated as a one-dimensional problem, and the physical properties of the walls is assumed to be remain constant over time. In addition, the indoor temperature is represented as a single node. The simplified heat transfer process for the data center room is depicted in Fig. 2.

Figure 2: Schematic diagram of the equivalent model of heat transfer process in data center computer room

The energy conservation equation for the indoor air and the walls are constructed as Eqs. (32) and (33).

Cin∗dθintdt=Pheatt−Pcoldt−θint−θwalltR1(32)

Cwall∗dθwalltdt=θint−θwalltR1−θwallt−θouttR2(33)

In the formula, t represents the current time point; Cin is the equivalent heat capacity of indoor air; Cwall is the equivalent heat capacity of the wall; R1 is the equivalent thermal resistance of indoor air and the inner side of the wall; R2 is the equivalent thermal resistance of the outer wall and outdoor air; θint is the indoor temperature at time t; θwallt is the wall temperature at time t; θoutt is the outdoor temperature at time t; Pheatt is the heat generation of the servers at time t; Pcoldt is the refrigeration power at time t.

In order to calculate the indoor temperature changes at different discrete time periods within a day, differential equations of Eqs. (32) and (33) are transformed into differential equations, as expressed as Eqs. (34) and (35), with a time step of Δτ.

Cinθint+Δτ−θintΔt=(Pheatt−Pcoldt)−θint−θwalltR1(34)

Cwallθwallt+Δτ−θwalltΔt=θint−θwalltR1−θwallt−θouttR2(35)

Transforming Eqs. (34) and (35) yields Eqs. (36) and (37):

θint+Δτ=[(Pheatt−Pcoldt)−θint−θwalltR1]∗ΔτCin+θint(36)

θwallt+Δτ=(θint−θwalltR1−θwallt−θouttR2)∗ΔτCwall+θwallt(37)

Eqs. (36) and (37) can be used to update the indoor temperature and wall temperature at each time point. Furthermore, according to the practical engineering situation where the variation amplitude of θwallt is very small compared to that of θint, θwallt can be regarded as its mean value θwallstable, and Eq. (38) can be derived from Eq. (37).

θwallt=R1∗θoutt+R2∗θintR1+R2(38)

Regarding the determination of the time step Δτ, it is crucial to balance accuracy with computational complexity. In this context, Δτ is set as 1 min. Combining Eqs. (36) and (38), the equilibrium constraints of the indoor temperatures at adjacent time points of θint+1 and θint can be expressed as:

θint+1=θint+Pheatt−PcoldtCin−θint−θoutt(R1+R2)∗Cin(39)

Combined with the initial indoor temperature θin0, the dynamic indoor temperature throughout the day can be calculated using the time-varying temperature model of computer room. Through the temperature time-variant model, dynamically adjusting the refrigeration power within the temperature range can achieve the regulation of electricity, thereby further promoting the consumption of renewable energy.

4  Two-Stage Multi-Time Scale Optimization Model for Data Center1

4.1 Two-Stage Multi-Time Scale Scheduling Framework

4.1.1 Day-Ahead and Intra-Day Optimization Scheduling Diagram

The multi-time scale optimization scheduling includes a two-stage day-ahead and intraday rolling optimization scheduling, as illustrated in Fig. 3. The day-ahead scheduling operates on a 24-h optimization scale with 15 min as the time step. Based on the day-ahead forecasts of renewable power and load, the scheduling plan for next 24 h can be formulated with the goal of minimizing the daily operating cost. The intraday rolling optimization scheduling takes 15 min as a rolling round, 60 min as the optimization scale, and 1 min as the time step for each round of optimization. Each round of optimization scheduling is based on 60-min ahead predictions of renewable power and load, aiming to minimize the operating costs of each round of optimization and minimize the fluctuations of exchange power between day-ahead and real-time plans.

Figure 3: Diagram of the two-stage multi-time scale optimization scheduling framework of data center

4.1.2 Regulation Characteristics at Different Time Scales of Computing Workloads and Refrigeration System

The time step of day-ahead scheduling is generally set as 15 min, which is consistent with sampling interval of power supply and demand. On the one hand, the fluctuations trend of renewable power throughout a day typically exhibit periodic changes. The 15-min interval allows for capturing the primary fluctuation characteristics of wind and PV power, while mitigating the computational burden caused by overly frequent data points. On the other hand, the workloads power consumption is relatively steady throughout the day. Using 15 min as the interval for the measuring of power consumption and as the minimum unit for its movement offers sufficient accuracy to illustrate the load shift mechanism while simplifying data processing. Consequently, the time-shifting model of workloads is suitable to be incorporated into the day-ahead optimization scheduling model. By shifting these workloads along the time axis, the fluctuations of wind and PV power can be offset.

However, the volatilities of wind and PV power at the 1-min level are notably faster and more irregular compared to the daily fluctuations forecasted within a 15-min interval. To address this problem, intra-day real-time scheduling are needed. In this study, the time step in the real-time scheduling stage is 1min, which is consistent with the time interval of the time-variant model of the computer room temperature. Given that refrigeration power serves as the only controllable variable for regulating computer room temperature, by embedding the time-varying temperature model into the intra-day optimization scheduling model, adjusting the refrigeration power every 1 min can effectively cope with the instantaneous fluctuations of wind and PV power. Thus, the refrigeration power can be used as a control variable for intra-day optimization scheduling.

4.2 Day-Ahead Scheduling Optimization Objective

The day-ahead optimization scheduling is designed to minimize operational costs within the dispatching day, expressed as:

OBJ=minCgrid+Ccur+Cdata(40)

The optimization function comprises the net cost of electricity transactions with the main-grid Cgrid (¥), penalties for wind and PV power curtailment Ccur, and penalty costs for the computing workloads scheduling Cdata(¥), which are expressed as Eqs. (41)–(43).

Cgrid=∑t=1TKbuyEbuytΔt−KsellEselltΔt(41)

Ccur=∑t=1TKcurEcurtΔt(42)

Cdata=∑t=1T∑k=1TUc.∗DcmatFcpumax∗Ecpumax∗Δt(43)

where Kbuy/Ksell is the unit electricity purchase/sale price from/to the main grid during time period t, which is 0.8 and 0.4 ¥/kWh, respectively2; Ebuyt/Esellt is the day-ahead planned purchased/sold electricity power during time period t; Kcur is the unit penalty price for wind and PV power curtailment, which is set as 0.6 ¥/kWh; Ecurt is the day-ahead planned renewable power curtailment in time period t. ∆t is the duration of time period t, set at 15 min in this paper.

By considering system operation constraints and combining day-ahead forecasts of renewable generation and electricity consumption of fundamental interactive computing workloads, this model can optimize the time shift plan of various delay-tolerant workloads, refrigeration output plan, exchange power plan, wind and PV power utilization plan under the goal of minimizing the operating costs of data center micro-grid during the scheduling day.

4.3 Constraints of Day-Ahead Optimization Scheduling

(1) Constraints of delay-tolerant batch processing workloads

The control variables of the long-running non-interruptible workloads, long-running interruptible workloads, and short-running workloads are Ua, Uaindex, Famov; Ub, Ubindex, Fbmov and Uc, Fcmov. Constraints of the control variables are detailed in Sections 3.1.2–3.1.4.

In addition to the above constraints, the other constraints are listed as follows.

(2) Computing power demand balance constraint

This study considers two long-running non interruptible workloads A1 and A2, two long-running interruptible workloads B1 and B2, and short-running workloads C. The total computing power demand of all the workloads at the original state Fdataori and after shifting Fdatamov should follow the below constraints.

Fdataori=Fa1ori+Fa2ori+Fb1ori+Fb2ori+Fcori+Fmgori(44)

Fdatamov=Fa1mov+Fa2mov+Fb1mov+Fb2mov+Fcmov+Fmgori(45)

where Fa1ori, … , FCori are the computing power demand of delay-tolerant workloads A1, … , C at the original state. Fa1move, …, Fcmove are the computing power demand of delay-tolerant workloads A1, … , C after shift. Since the delay-sensitive workloads don’t shift, their computing power demand are denoted as Fmgori.

(3) Relation of computing power demand and electricity consumption

Edatat=Eidle+(Epeak−Eidle)∗FdatatFdcmax(46)

where Fdatat is the computing power demand of all workloads during time period t.

(4) Indoor temperature constant constraint

θint=25∘C(47)

If day-ahead forecasted indoor temperature are set to be variable, aligning with the 1-min time step of the time-variant computer room temperature model, day-ahead scheduling would require matching this granularity, which would result in each day-ahead decision variable increasing to 1440 and significantly increasing computational burden. Thus, indoor temperature is fixed at 25°C during day-ahead scheduling. Updates are applied via intraday optimization, which incorporates a refined 1-min temperature variation model.

(5) Refrigeration constraint

The refrigeration power at time point t should meet the upper and lower limits constraints:

Pcoldmin≤Pcoldt≤Pcoldmax(48)

In this research, Pcoldmin is 0, Pcoldmax is 500 kW. Detailed analysis can be found in Section 5.1.3.

The relation of Pcoldt and the electricity power consumption of refrigeration system Ecoldt is:

Ecoldt=Pcoldt/3.5(49)

(6) Upper and lower limit of renewable power utilization

The utilized wind and PV power at time point t. Ewindt and Epvt have the constraints as below.

0≤Ewindt≤Ewind,maxt(50)

0≤Epvt≤Epv,maxt(51)

where Ewind,maxt and Epv,maxt are the forecasts of maximum wind and PV power generation at time t.

(7) Exchange power constraint

The day-ahead planned purchased power and sold power Ebuyt and Esellt, as well as the state of power purchase and sell Bbuyt and Bsellt have the following constraint.

Bbuyt∗Ebuymin≤Ebuyt≤Bbuyt∗Ebuymax(52)

Bsellt∗Esellmax≤Esellt≤Bsellt∗Esellmax(53)

Bbuyt+Bsellt=1(54)

where Ebuymin is 0, Ebuymax is 360 kW, Esellmin is 0, Esellmax is 200 kW.

(8) Power supply and demand balance constraint

The total power at supply side of data center equals to the power on demand side:

Ewindt+Epvt+Ebuyt=Edatat+Ecoldt+Esellt(55)

4.4 Intraday Rolling Optimization Objective

Due to the different time steps of intra-day and day-ahead scheduling, in order to distinguish, the time point in intra-day scheduling is recorded as τ; The time step is recorded as Δτ; The number of time points of a scheduling day is denoted as Γ. For each round of real-time optimization, the objective is to minimize the fluctuation of power purchase and sale and to optimize the operating cost during the optimization time frame, expressed as:

obj=minVgridVgridmax+CopCopmax(56)

In the formula, there is:

Vgrid=∑τ=160|Ebuy∗τ−Ebuyτ|+|Esell∗τ−Esellτ|(57)

Cop=∑τ=160Ebuy∗τKbuyΔτ−Esell∗τKsellΔτ+∑τ=1ΓKcurEcur∗τΔτ(58)

where Vgrid is the fluctuation of the purchased and sold power between day-ahead plan and real-time plan for each round. Cop is the optimized operation cost for each round. Vgridmax is the fluctuation of the purchased and sold power between day-ahead and real-time plan with the minimization of operation costs as the only goal for each round of optimization. Copmax is the operating costs with the minimization of fluctuation of the purchased and sold power between day-ahead and real-time plan as the only goal for each round of optimization. Ebuyτ/Esellτ is the day-ahead planned purchased/sold electricity power at time point τ. Ebuy∗τ/Esell∗τ is the real-time purchased/sold electricity power at time period τ. Ecur∗τ is the real-time renewable power curtailment at time period τ.

For each round, the control variables are real-time refrigeration power Pcold∗τ, refrigeration electricity consumption Ecold∗τ, utilized wind power Ewind∗τ, utilized solar power Epv∗τ, purchased power Ebuy∗τ, sold power Esell∗τ, and indoor temperature θin∗τ. The workloads aren’t suitable to be the control variable in the rolling optimization. The day-ahead time-shift plan of workloads Edatat are only adopted as boundary conditions in the intra-day scheduling. By considering the system operation constraints and combining real-time forecasts of renewable power generation and server electricity consumption (Ewind∗,mpptτ, Epv∗,mpptτ, Edataτ), the control variables are solved using the rolling optimization model.

With reference to Fig. 3, for each round optimization, the first 15-min time period of Ecold∗τ, Ewind∗τ, Epv∗τ, Ebuy∗τ, Esell∗τ, θin∗τ will be saved. After multiple rounds of rolling optimization scheduling, the results of the control variables in 1440 min are obtained.

4.5 Constraints of Intraday Rolling Optimization

Constraints regarding these variables are described in the following section.

(1) Indoor temperature constraint

For each round, the real-time indoor temperature θin∗τ need to satisfy the upper and lower limit:

5∘C≤θin∗τ≤30∘C(59)

The adjacent indoor temperature θin∗τ+1 and θin∗τ in the intra-day scheduling satisfies the equilibrium constraint:

θin∗τ+1=θin∗τ+Pheatτ−Pcold∗τCin−θin∗τ−θout∗τR1∗Cin+R2∗Cin(60)

where the indoor temperature at the start time θin∗0 is 25°C. For each round of optimization thereafter, θin∗0 is equal to the optimized indoor temperature at the 15th time point of the last optimization round. Pheatτ is equal to Edataτ, which is a known boundary condition derived from day-ahead scheduling plan.

The refrigeration power should satisfy the ramp constraint.

θin∗τ+15−θin∗τ≤Δθin∗max15(61)

θin∗τ+1−θin∗τ≤Δθin∗max1(62)

where θin∗τ+1 and θin∗τ are the indoor temperature at time τ+1 and τ. In this paper, Δθin∗max15 and Δθin∗max1 is the max fluctuation of indoor temperature within 15 min and 1minute, which are set to be 15°C and 0.33°C [24].

(2) Refrigeration power

The refrigeration power should satisfy the ramp constraint.

Pcold∗τ+1−Pcold∗τ≤ΔPcold∗max(63)

where Pcold∗τ and Pcold∗τ+1 are the refrigeration power at time point τ+1 and τ. In this paper, ΔPcold∗max is set to be 100 kW.

The relationship of Pcold∗τ with the refrigeration electricity consumption Ecold∗τ is:

Ecold∗τ=Pcold∗τ3.5(64)

(3) Upper and lower limit of renewable power utilization

Real-time utilized wind and PV power at time τ Ewind∗τ and Epv∗τ have the constraints as below.

0≤Ewind∗τ≤Ewind∗,mpptτ(65)

0≤Epv∗τ≤Epv∗,mpptτ(66)

where Ewind∗,mpptτ and Epv∗,mpptτ are the 60-min ahead forecasts of maximum wind and PV power generation at time τ.

(4) Exchange power constraint

Real-time electricity purchase and sale Ebuy∗τ and Esell∗τ, as well as the state of electricity purchase and sale Bbuy∗τ and Bsell∗τ have the following constraint.

Bbuy∗τ∗Ebuy∗min≤Ebuy∗τ≤Bbuy∗τ∗Ebuy∗max(67)

Bsell∗τ∗Esell∗min≤Esell∗τ≤Bsell∗τ∗Esell∗max(68)

Bbuy∗τ+Bsell∗τ=1(69)

where Ebuy∗min is 0, Ebuy∗max is 360 kW; Esell∗min is 0, Esell∗max is 200 kW.

(5) Power supply and demand balance constraint

For each optimization round, the power balance equation constraint at time τ is expressed as:

Ewind∗τ+Epv∗τ+Ebuy∗τ=Edataτ+Ecold∗τ+Esell∗τ(70)

5  Boundary Conditions and Multi-Time Scale Optimization Scheduling Results

5.1 Boundary Conditions and Parameters of the Data Center

5.1.1 Parameters of the CPU, Server, Rack and Data Center

This study selected a medium-sized data center room in northern China as the example. The data center room occupies an area of 200 m2, with a height of 4 m. The external wall area of the computer room is 240 m2, and the roof area is 200 m2. The data center room is equipped with 60 42U racks, each housing 10 4-way 4U servers. Detailed parameters for the CPU chip, server and rack are outlined in Table 1.

Based on Table 1, in this paper, Fdcmax is 3.84 ∗ 106 GFLOPS, Epeak is 360 kW. The Eidle is 216 kW, which is calculated by multiplying Epeak with a coefficient of 0.6 [25].

The thermal parameters of the computer room is adapted from literature [26] in accordance with the standards and specifications of data center room construction. The thermal parameters for the equivalent model of heat transfer process in the data center room are given in Table 2.

5.1.2 Parameters of Delay-Sensitive and Delay-Tolerant Workloads

The study considers two long-running non-interruptible workloads (A1 and A2), two long-running interruptible workloads (B1 and B2), and short-running workloads (C). The original computing power demand time series curves for the delay-sensitive interactive workload and delay-tolerant batch processing workloads are depicted in Fig. 4.

Figure 4: The original computing power demand time series curves of the delay-sensitive and various delay-tolerant workloads

The shifting time range [te, tl] for the start node of the long-running non-interruptible workload A1 and A2 is [33, 73] and [1, 87]. The short-running workloads are allowed to be fully or partially transferred to the 1st–11th time points after the initial time point.

5.1.3 Configuration of Refrigeration Power System

The heat load of data center mainly stems from the heat generated by servers and environmental maintenance structure. The rated refrigeration capacity of refrigeration system is determined based on the server equipment power and the area of data center room, as described by the formula:

Pcoldrated=Pheatmax+βSa(71)

where Pheatmax represents the maximum heat generation of the server cluster, which is equal to Epeak [17]. Sa denotes the area of the computer room, set as 200 m2 for this study. The empirical coefficient β is assigned with a value of 0.7 [27]. Hence, the rated refrigeration capacity Pcoldrated is determined to be 500 kW for the computer room. With a comprehensive refrigeration performance coefficient of 3.5, the corresponding rated electricity consumption of the refrigeration system Ecoldrated is calculated to be 143 kW.

5.1.4 Configuration of Wind and PV Power System

The installed capacity of wind and PV power system, denoted as Evremax, is determined by formula (72) as follows:

Evremax∗ηvre=Ecoldrated+Epeak(72)

In this formula, the average power generation efficiency of renewable energy ηvre is assumed to be 0.5. With Ecoldrated and Epeak having the values of 143 and 360 kW, Evremax is calculated to be 1000 kW. This study stochastically set the proportion of wind and PV power capacity to be 3:7, resulting in capacities of 300 and 700 kW for wind and PV power system, respectively. This study focuses on developing a scheduling method considering the time-shiftable workloads and refrigeration system to facilitate the renewable energy utilization, while the economic analysis of allocating renewable energy is not the main concern. It is noted that varying the capacity ratios of wind and PV doesn’t affect the effectiveness of the control strategy and conclusions drawn in this paper.

Wind and PV power generation data used in the study are sourced from a reliable database [28]. To align with the configured capacities of wind and PV power systems, the data is scaled to generate 24h-ahead forecasts of wind and PV power generation used in this study, as depicted in Fig. 5.

Figure 5: 24 h-ahead forecasts of wind and PV power generation for data center

5.2 Results of Multi-Time Scale Optimization Scheduling

This paper formulates both day-ahead scheduling and intra-day real-time scheduling optimization as Mixed Integer Programming (MIP) optimization problems. The optimization involves continuous variables, e.g., key time-shift parameter variables for various batch computing workloads, and binary variables, e.g., the status of power purchases and sales. Based on the decision variables, equality and inequality constraints, and objective function established in previous sections, the optimization model is solved using Gurobi solver in MATLAB R2024a. The Gurobi solver, widely recognized in academia and industry, employs techniques such as the interior point method, branch and bound method, and cut plane method to find optimal solutions.

This section presents the scheduling results after multi-time scale optimization. Section 5.2.1 details the operational costs and renewable power curtailment results after day-ahead scheduling and verifies its effectiveness by comparing with a simple scheduling scenario. Additionally, it analyzes the comparison of workloads at original status and after scheduling to illustrate the feasibility of the proposed time-shift modelling. Section 5.2.2 reports the operational costs and power exchange volatility results after real-time optimization scheduling. A comparison between the optimization scheduling and a plain real-time scheduling is also conducted to highlight the necessity of incorporating refrigeration regulation in real-time scheduling to cope with the instantaneous power volatility from renewable energy and load demand.

5.2.1 Results after Day-Ahead Optimization Scheduling

To quantitatively verify the effect of considering workloads time-shift, this paper sets up a simple scheduling scenario where the workloads don’t shift over time. In the simple scheduling scenario, the power balance results of utilized wind power Ewind#, utilized PV power EPV#, purchased power Ebuy#, server power consumption Edata#, refrigeration electricity consumption Ecold# and sold power Esell# are illustrated in Fig. 6. By considering the 24 h-ahead forecasts of wind and PV power generation and the system operation constraints, the daily operation plan in this scenario is obtained. Fig. 6 indicates that in this scenario, electricity sales mainly occur from the 39th–66th time periods, while electricity purchases predominantly occur from the 1st–38th and 67th–96th time periods. Considering the purchase and sale electricity prices, as well as the penalties for wind and PV power curtailment, the daily operating cost in this simple scheduling scenario amounts to ¥2980.6, and the planned curtailment of wind and solar power is 294.6 kWh (3.8% of the forecasted daily renewable generation).

Figure 6: The power balance results in the day-ahead simple scheduling scenario

In the day-ahead optimization scheduling scenario where the time-shiftable workloads are involved, the power balance results of Ewind, EPV, Ebuy, Edata, Ecold, and Esell for the scheduling day are presented in Fig. 7. A comparison of wind and PV power curtailment between the day-ahead simple scheduling scenario and the optimization scheduling scenario is depicted in Fig. 8. The computing power demand time series of workloads at the original state and after the temporal shift are shown in Figs. 9 and 10, respectively. Upon observing Figs. 9 and 10, it is evident that after optimization, long-running non-interruptible workload A1 and A2, long-running interruptible workload B2 all moves to the 43rd–66th time periods, and long-running interruptible workload B1 and short-running workloads C partially shift to these time periods. Consequently, the renewable power curtailment during the 43rd–66th periods is reduced, while the electricity purchases during the 1st–38th periods and the 67th–96th periods decreases. The optimized operating cost plan is ¥1885.3 for the scheduling day, and the planned wind and PV power curtailment is 145.9 kWh (1.9% of the daily renewable generation), which respectively represents a 36.7% and 50.5% decrease compared to simple scheduling scenario. These comparison results prove that considering workload time-shift for the day-ahead optimization scheduling is advantageous in increasing the economic benefits of data center operation and promoting the consumption of renewable power.

Figure 7: The power balance results in the day-ahead optimization scheduling scenario

Figure 8: Comparison of wind and PV power curtailment in the day-ahead simple scheduling scenario and the optimization scheduling scenario

Figure 9: The computing power demand time series of various delay-tolerant workloads at the original state

Figure 10: The computing power demand time series of various delay-tolerant workloads after temporal shift

The temporal shift details for the various delay-tolerant workloads are visualized in Figs. 11–15. Specifically, Figs. 11 and 12 display the computing power demand of long-running non-interrupted workloads A1 and A2 before and after shifting. Figs. 13 and 14 illustrate the computing power demand of long-running interruptible workloads B1 and B2, demonstrating that the sequential order of subtasks at different time points and the total computing power demand remain consistent before and after shifting. Additionally, Fig. 15 depicts the computing power demand of short-running workloads C before and after shifting. The results indicate a redistribution of computing power demand as short-running workloads transfer from initial time points to new time points, revealing a decrease in demand during the initial time periods and an increase at the new time periods. The total computing power demand stays unchanged before and after the shifting process. These findings provide evidence supporting the feasibility of workloads’ time shifting, demonstrating the adaptability and efficiency of the workloads scheduling strategy.

Figure 11: The computing power demand curve of long-running non-interruptible workload A1 at original state and after moving

Figure 12: The computing power demand curve of long-running non-interruptible workload A2 at original state and after moving

Figure 13: The computing power demand curve of long-running interruptible workload B1 at original state and after moving

Figure 14: The computing power demand curve of long-running interruptible workload B2 at original state and after moving

Figure 15: The computing power demand curve of short-running workloads C at original state and after moving

Among the three kinds of workloads, the short-running workloads have a special setting—a penalty coefficient in their time shift model. This paper discusses the rationale behind this setting by comparing the scenarios with and without the penalty coefficient. The time shift result of the various delay-tolerant workloads after the day-ahead optimization scheduling without the penalty coefficient is shown in Fig. 16.

Figure 16: The computing power demand of delay-tolerant workloads after shifting in the scenario without the penalty coefficient setting for short-running workloads

(1)   In the scenario with the penalty coefficient setting, most of the short-running workloads C are reallocated to the 43rd–66th time periods, where the surplus wind and solar power is utilized. The reason of this behavior is that, the cost of reducing the wind and PV power sales volume (0.4 + 0.05 = 0.45 ¥/kWh) is lower than the cost of purchasing electricity from other periods (1st–42nd and 67th–96th periods) (0.8 ¥/kWh). For another, this behavior may also lead to electricity revenue (0.6 − 0.05 = 0.55 ¥/kWh) due to reducing penalties for wind and PV power curtailment. Hence, operators are willing to pay lower costs or gain profits by using wind and PV power, rather than purchasing electricity from the main grid.

(2)   In the scenario without the penalty coefficient setting, there is still an inflow and aggregation of short-running workloads in the 43rd−66th time periods. Meanwhile, there are also some workloads transferring from the 70th−96th period to the 1st−33rd time period. The mutual transfer of short-running workloads between the 1st−33rd and the 70th−96th time periods is an invalid time shift, since the unit price of power purchase in these two time frames is the same. Thus, the comparison indicates that setting the penalty coefficient for the time-shift model of short-running workloads can avoid invalid movement.

5.2.2 Results after Intra-Day Rolling Optimization

Considering the uncertainty of the day-ahead forecasts of power supply and demand, the day-ahead scheduling plans are updated and corrected by using the intra-day rolling optimization scheduling model. In order to carry out this model, the ultra-short-term 60 min-ahead forecasts of the wind, PV power generation, and server power consumption are needed. According to literature, the 7.5%, 5%, 2.5% mean absolute percentage error of day-ahead forecasts of wind, PV power and server electricity load compared to real-time forecasts are reasonable values [29,30]. Thus, the real-time forecasts of wind power, PV power generation, and server power consumption with a fluctuation ranges of 15%, 10%, and 5% compared to day-ahead forecasts are generated, as shown in Fig. 17a–c. The real-time outdoor temperature is given in Fig. 18.

Figure 17: (a–c) Real-time forecasts (plans) and day-ahead forecasts (plans) of wind and PV power generation, and server power consumption. (a) The 60 min-ahead and 24 h-ahead forecasts of wind power generation (step length: 1 min). (b) The 60 min-ahead and 24 h-ahead forecasts of PV power generation (step length: 1 min). (c) Real-time server electricity consumption and day-ahead planned server electricity consumption (step length: 1 min)

Figure 18: Real-time outdoor temperature (step length: 1 min)

Through the intra-day rolling optimization, the power balance results of utilized wind power Ewind∗, utilized PV power EPV∗, purchased power Ebuy∗, server power consumption Edata∗, refrigeration electricity consumption Ecold∗ and sold power Esell∗ are as shown in Fig. 19. The fluctuation of real-time optimized exchange power relative to the day-ahead planed exchange power is shown in Fig. 20. Results show that the daily operating cost after real-time rolling optimization scheduling is ¥1895.9, and the deviation of real-time exchange power compared to day-ahead plan is 0.30 kW/min. Furthermore, optimized real-time indoor temperature, refrigeration power and heat generation, and net heat are shown in Fig. 21a–c. It can be observed that the indoor temperature fluctuates within the range of 15°C–30°C, and the temperature change rate is within the specified limits. The refrigeration power also meets the ramping constraint. Results indicate that the intra-day optimization scheduling can not only obtain the refined operation plan updates, but can also minimize the impact of uncertainty on the exchange power volatility, proving that the intra-day optimization scheduling plays an indispensable role in data center power scheduling.