Moving Deferrable Big Data to the Cloud by Adopting an Online Cost-Minimization Approach

Moving Deferrable Big Data to the Cloud by Adopting an Online Cost-Minimization Approach

As cloud computing gets popular in recent years, the bandwidth cost of data centers becomes a hot research topic. For the analysis jobs based on MapReduce framework, locally generated big data usually does not need uploading immediately. Instead, certain delay is tolerable. Therefore, we can use the allowable delay time to optimize the bandwidth usage and minimize the cost.

In this paper, we discuss how to use the allowable delay window that a given workload has and propose two algorithm to reduce peak volume by increasing the maximum transmission of early stages. The experiments show that the peak value can be reduced by choosing a larger initial value.

Besides, we also discuss how to assign workloads among data centers in the cloud scenario. We point out that the total bandwidth cost of data centers will be minimal when the maximum transmission capacity of these data centers are generally equal to each other.