閱讀全文 | |
篇名 |
ACO-HCO: Heuristic Performance Tuning Scheme for the Hadoop MapReduce Architecture
|
---|---|
並列篇名 | ACO-HCO: Heuristic Performance Tuning Scheme for the Hadoop MapReduce Architecture |
作者 | Chiang-Lung Liu、Hsiang-Fu Lo、Wei-Tsong Lee |
英文摘要 | Hadoop MapReduce is a widely-used cloud computing technology for big data processing. However, the Hadoop configuration parameters settings can significantly change the execution performance. Manual adjustment of the Hadoop parameters will be a time consuming and difficult task. In this paper, we propose ACO-HCO, a Hadoop configuration tuning scheme for MapReduce applications. We use MapReduce applications job history records to generate specific job profiles. Based on these profiles, an objective function for execution time is constructed with gene expression programming algorithm by mining the correlation among the core Hadoop configuration parameters and input data size. Leveraging the objective function, an ACO-based configuration optimizer is able to heuristically search for the optimal configuration for a given application. Experimental results show that ACO-HCO enhances the performance of Hadoop significantly compared with the default configuration. Moreover, ACO-HCO performs better than heuristic approach and the cost-based model in Hadoop performance tuning. |
起訖頁 | 1151-1160 |
關鍵詞 | Hadoop performance tuning、Ant colony optimization algorithm、Gene expression programming |
刊名 | 網際網路技術學刊 |
期數 | 202007 (21:4期) |
出版單位 | 台灣學術網路管理委員會 |
DOI |
|
QR Code | |
該期刊 上一篇
| An Improved RSU-based Authentication Scheme for VANET |
該期刊 下一篇
| Research on Recurrent Neural Network Based Crack Opening Prediction of Concrete Dam |