Technical Reports
HPL-2012-92
Click here for full text:
Overcoming Web Server Benchmarking Challenges in the Multi-Core Era
Hashemian, Raoufehsadat; Krishnamurthy, Diwakar; Arlitt, Martin
HP Laboratories
HPL-2012-92
Keyword(s): Software Performance; Performance Testing; Multi-Core Servers
Abstract: Web-based services are used by many organizations to support their customers and employees. An important consideration in developing such services is ensuring the Quality of Service (QoS) that users experience is acceptable. Recent years have seen a shift toward deploying Web services on multi-core hardware. Leveraging the performance benefits of multi-core hardware is a non-trivial task. In particular, systematic Web server benchmarking techniques are needed so organizations can verify their ability to meet customer QoS objectives while effectively utilizing such hardware. However, our recent experiences suggest that the multi-core era imposes significant challenges to Web server benchmarking. In particular, due to limitations of current hardware monitoring tools, we found that a large number of experiments are needed to detect complex bottlenecks that can arise in a multi-core system due to contention for shared resources such as cache hierarchy, memory controllers and processor inter-connects. Furthermore, multiple load generator instances are needed to adequately stress multi-core hardware. This leads to practical challenges in validating and managing the test results. This paper describes the automation strategies we employed to overcome these challenges. We make our test harness available for other researchers and practitioners working on similar studies.
6 Pages
Additional Publication: Published in the proceedings of LT 2012: 1st International Workshop on Load Testing of Large Software Systems & the 5th IEEE International Conference on Software Testing, Verification and Validation.
External Posting Date: May 6, 2012 [Fulltext]. Approved for External Publication
Internal Posting Date: May 6, 2012 [Fulltext]