Program performance is always a concern, even in this era of high-performance hardware. This article, the first in a two-part series, guides you around the many pitfalls associated with benchmarking Java code. Part 2 covers the statistics of benchmarking and offers a framework for performing Java benchmarking. Because almost all new languages are virtual machine-based, the general principles the article describes have broad significance for the programming community at large.
M. Yasugi, Y. Matsuda, and T. Ugawa. Proceedings of the 11th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering - PASTE \textquotesingle13, ACM, (2013)
A. Goens, A. Brauckmann, S. Ertel, C. Cummins, H. Leather, and J. Castrillon. Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages, page 38–46. New York, NY, USA, Association for Computing Machinery, (2019)
C. Cummins, P. Petoumenos, Z. Wang, and H. Leather. Proceedings of the 2017 International Symposium on Code Generation and Optimization, page 86–99. IEEE Press, (2017)
D. Aumayr, S. Marr, E. Gonzalez Boix, and H. Mössenböck. Proceedings of the 16th ACM SIGPLAN International Conference on Managed Programming Languages and Runtimes, page 157--171. ACM, (October 2019)
R. Roberts, S. Marr, M. Homer, and J. Noble. 33rd European Conference on Object-Oriented Programming, volume 134 of ECOOP'19, page 5:1--5:28. Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik, (Jul 15, 2019)
A. Phansalkar, A. Joshi, L. Eeckhout, and L. John. IEEE International Symposium on Performance Analysis of Systems and Software, 2005. ISPASS 2005., page 10--20. (March 2005)
G. Richards, S. Lebresne, B. Burg, and J. Vitek. Proceedings of the 31st ACM SIGPLAN Conference on Programming Language Design and Implementation, page 1--12. ACM, (June 2010)
A. Born de Oliveira, S. Fischmeister, A. Diwan, M. Hauswirth, and P. Sweeney. Proceeding of the 18th international conference on Architectural support for programming languages and operating systems, volume 48 of ASPLOS '13, page 207--218. ACM, (March 2013)