Project Description: Firstly we evaluate HBase with some real applications, and then study the HBase implementation to seek for optimization or better design.
Current Status: HBase falsely sequetializes irrelative put operations and thus causes long lock acquiring time. We’re attempting to parallelize these operations.
(03/2008 ~ Now) MapReduce performance on cluster with multi-core nodes
Project Description: We studied the MapReduce performance on multi-core cluster. We tried to exploit the power of multi-core machines to maximize MapReduce efficiency.
Results: The poor performance of Hadoop mainly comes from object creation, large memory footprint, naive data structure and lack of good multithreading support. We have implemented a prototype system Azwraith, which outperforms the original MapReduce implementation from 1.4X to 3.5X.
Current Status: Identify the interrupt handling issues in Hadoop. Propose a federated schedule model for MapReduce.
(09/2009 ~ 11/2009) Worked on understanding the characteristics of IO in Hadoop and surveyed applying SSD to Hadoop and HBase
Results: Mixed workloads incur a random disk access pattern in Hadoop. Using SSD as a disk cache and intermediate storage would gain a good speedup.
(03/2009 ~ 04/2009) Evaluated parallel shared-memory applications with MapReduce cluster
Results: Frequent communications in this kind of applications cause significant overhead on cluster. We proposed some extensions to the MapReduce model in our APPT’09 paper.
Zhiwei Xiao, Haibo Chen, Binyu Zang and Bo Huang. A Hierarchical Approach to Maximizing MapReduce Efficiency. In the Proceedings of the 20th International Conference on Parallel Architectures and Compilation Techniques (PACT’11, poster). 2011. [pdf]
Chao Zhang, Chenning Xie, Zhiwei Xiao, and Haibo Chen. Evaluating the Performance and Scalability of MapReduce Applications on X10. In the Proceedings of the 9th International Symposium on Advanced Parallel Processing Technologies. 2011.
Shengkai Zhu, Zhiwei Xiao, Haibo Chen, Rong Chen, Weihua Zhang and Binyu Zang. Evaluating SPLASH-2 Applications Using MapReduce. In the Proceedings of the 8th International Symposium on Advanced Parallel Processing Technologies(APPT'09). 2009. [pdf]