Program performance is always a concern, even in this era of high-performance hardware. This article, the first in a two-part series, guides you around the many pitfalls associated with benchmarking Java code. Part 2 covers the statistics of benchmarking and offers a framework for performing Java benchmarking. Because almost all new languages are virtual machine-based, the general principles the article describes have broad significance for the programming community at large.
A. Born de Oliveira, S. Fischmeister, A. Diwan, M. Hauswirth, and P. Sweeney. Proceeding of the 18th international conference on Architectural support for programming languages and operating systems, volume 48 of ASPLOS '13, page 207--218. ACM, (March 2013)
G. Richards, S. Lebresne, B. Burg, and J. Vitek. Proceedings of the 31st ACM SIGPLAN Conference on Programming Language Design and Implementation, page 1--12. ACM, (June 2010)
A. Phansalkar, A. Joshi, L. Eeckhout, and L. John. IEEE International Symposium on Performance Analysis of Systems and Software, 2005. ISPASS 2005., page 10--20. (March 2005)
R. Roberts, S. Marr, M. Homer, and J. Noble. 33rd European Conference on Object-Oriented Programming, volume 134 of ECOOP'19, page 5:1--5:28. Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik, (Jul 15, 2019)
D. Aumayr, S. Marr, E. Gonzalez Boix, and H. Mössenböck. Proceedings of the 16th ACM SIGPLAN International Conference on Managed Programming Languages and Runtimes, page 157--171. ACM, (October 2019)
C. Cummins, P. Petoumenos, Z. Wang, and H. Leather. Proceedings of the 2017 International Symposium on Code Generation and Optimization, page 86–99. IEEE Press, (2017)
A. Goens, A. Brauckmann, S. Ertel, C. Cummins, H. Leather, and J. Castrillon. Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages, page 38–46. New York, NY, USA, Association for Computing Machinery, (2019)
M. Yasugi, Y. Matsuda, and T. Ugawa. Proceedings of the 11th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering - PASTE \textquotesingle13, ACM, (2013)