Comparing benchmarks using key microarchitecture-independent characteristics
- Author
- Kenneth Hoste (UGent) and Lieven Eeckhout (UGent)
- Organization
- Abstract
- Understanding the behavior of emerging workloads is important for designing next generation microprocessors. For addressing this issue, computer architects and performance analysts build benchmark suites of new application domains and compare the behavioral characteristics of these benchmark suites against well-known benchmark suites. Current practice typically compares workloads based on microarchitecture-dependent characteristics generated from running these workloads on real hardware. There is one pitfall though with comparing benchmarks using microarchitecture-dependent characteristics, namely that completely different inherent program behavior may yield similar microarchitecture-dependent behavior. This paper proposes a methodology for characterizing benchmarks based on microarchitecture-independent characteristics. This methodology minimizes the number or inherent program characteristics that need to be measured by exploiting correlation between program characteristics. In fact, we reduce our 47-dimensional space to an 8-dimensional space without compromising the methodology's ability to compare benchmarks. The important benefits of this methodology are that (i) only a limited number of microarchitecture-independent characteristics need to be measured, and (ii) the resulting workload characterization is easy to interpret. Using this methodology we compare 122 benchmarks from 6 recently proposed benchmark suites. We conclude that some benchmarks in emerging benchmark suites are indeed similar to benchmarks from well-known benchmark suites as suggested through a microarchitecture-dependent characterization. However, other benchmarks are dissimilar based on a microarchitecture-independent characterization although a microarchitecture-dependent characterization suggests the opposite to be true.
Downloads
-
Hoste 2006 ISWC 83.pdf
- full text
- |
- open access
- |
- |
- 3.13 MB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-411847
- MLA
- Hoste, Kenneth, and Lieven Eeckhout. “Comparing Benchmarks Using Key Microarchitecture-Independent Characteristics.” Proceedings of the IEEE International Symposium on Workload Characterization, IEEE, 2006, pp. 83–92, doi:10.1109/IISWC.2006.302732.
- APA
- Hoste, K., & Eeckhout, L. (2006). Comparing benchmarks using key microarchitecture-independent characteristics. Proceedings of the IEEE International Symposium on Workload Characterization, 83–92. https://doi.org/10.1109/IISWC.2006.302732
- Chicago author-date
- Hoste, Kenneth, and Lieven Eeckhout. 2006. “Comparing Benchmarks Using Key Microarchitecture-Independent Characteristics.” In Proceedings of the IEEE International Symposium on Workload Characterization, 83–92. New York, NY, USA: IEEE. https://doi.org/10.1109/IISWC.2006.302732.
- Chicago author-date (all authors)
- Hoste, Kenneth, and Lieven Eeckhout. 2006. “Comparing Benchmarks Using Key Microarchitecture-Independent Characteristics.” In Proceedings of the IEEE International Symposium on Workload Characterization, 83–92. New York, NY, USA: IEEE. doi:10.1109/IISWC.2006.302732.
- Vancouver
- 1.Hoste K, Eeckhout L. Comparing benchmarks using key microarchitecture-independent characteristics. In: Proceedings of the IEEE International Symposium on Workload Characterization. New York, NY, USA: IEEE; 2006. p. 83–92.
- IEEE
- [1]K. Hoste and L. Eeckhout, “Comparing benchmarks using key microarchitecture-independent characteristics,” in Proceedings of the IEEE International Symposium on Workload Characterization, San José, CA, USA, 2006, pp. 83–92.
@inproceedings{411847,
abstract = {{Understanding the behavior of emerging workloads is important for designing next generation microprocessors. For addressing this issue, computer architects and performance analysts build benchmark suites of new application domains and compare the behavioral characteristics of these benchmark suites against well-known benchmark suites. Current practice typically compares workloads based on microarchitecture-dependent characteristics generated from running these workloads on real hardware. There is one pitfall though with comparing benchmarks using microarchitecture-dependent characteristics, namely that completely different inherent program behavior may yield similar microarchitecture-dependent behavior.
This paper proposes a methodology for characterizing benchmarks based on microarchitecture-independent characteristics. This methodology minimizes the number or inherent program characteristics that need to be measured by exploiting correlation between program characteristics. In fact, we reduce our 47-dimensional space to an 8-dimensional space without compromising the methodology's ability to compare benchmarks. The important benefits of this methodology are that (i) only a limited number of microarchitecture-independent characteristics need to be measured, and (ii) the resulting workload characterization is easy to interpret. Using this methodology we compare 122 benchmarks from 6 recently proposed benchmark suites. We conclude that some benchmarks in emerging benchmark suites are indeed similar to benchmarks from well-known benchmark suites as suggested through a microarchitecture-dependent characterization. However, other benchmarks are dissimilar based on a microarchitecture-independent characterization although a microarchitecture-dependent characterization suggests the opposite to be true.}},
author = {{Hoste, Kenneth and Eeckhout, Lieven}},
booktitle = {{Proceedings of the IEEE International Symposium on Workload Characterization}},
isbn = {{1-4244-0508-4}},
language = {{eng}},
location = {{San José, CA, USA}},
pages = {{83--92}},
publisher = {{IEEE}},
title = {{Comparing benchmarks using key microarchitecture-independent characteristics}},
url = {{http://doi.org/10.1109/IISWC.2006.302732}},
year = {{2006}},
}
- Altmetric
- View in Altmetric
- Web of Science
- Times cited: