Advanced search
1 file | 558.02 KB

ArchExplorer.org: a methodology for facilitating a fair comparison of research ideas

Author
Organization
Abstract
While reproducing the experimental results of research articles is standard practice in mature domains of science, such as physics or biology, it has not yet become mainstream in computer architecture. However, recent research shows that the lack of a fair and broad comparison of research ideas can be significantly detrimental to the progress, and thus the productivity, of research. At the same time, the complexity of architecture simulators and the fact that simulators are not systematically disseminated with novel ideas are largely responsible for this situation. While this methodology has a fundamental impact on research, it is by essence a practical issue. In this article, we present and set up an a typical approach to overcome this practical methodology issue, which takes the form of an open and continuous exploration through ArchExplorer, a server-side web infrastructure, that can significantly ease the process of fairly and quantitatively comparing research ideas. The web infrastructure ArchExplorer.org is now publicly open, and we demonstrate the approach with a set of data cache mechanisms. We show that this broad exploration can challenge some earlier assessments about data cache research, and even challenges the conclusions of an earlier but less thorough study on data cache comparison.
Keywords
cache storage, architecture simulator complexity, ArchExplorer.org, data cache mechanisms, Internet, server-side Web infrastructure, computer architecture

Downloads

    • full text
    • |
    • UGent only
    • |
    • PDF
    • |
    • 558.02 KB

Citation

Please use this url to cite or link to this publication:

Chicago
Desmet, Veerle, Sylvain Girbal, and Olivier Temam. 2010. “ArchExplorer.org: a Methodology for Facilitating a Fair Comparison of Research Ideas.” In 2010 IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS 2010), 45–54. Piscataway, NJ, USA: IEEE.
APA
Desmet, V., Girbal, S., & Temam, O. (2010). ArchExplorer.org: a methodology for facilitating a fair comparison of research ideas. 2010 IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS 2010) (pp. 45–54). Presented at the 2010 IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS - 2010), Piscataway, NJ, USA: IEEE.
Vancouver
1.
Desmet V, Girbal S, Temam O. ArchExplorer.org: a methodology for facilitating a fair comparison of research ideas. 2010 IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS 2010). Piscataway, NJ, USA: IEEE; 2010. p. 45–54.
MLA
Desmet, Veerle, Sylvain Girbal, and Olivier Temam. “ArchExplorer.org: a Methodology for Facilitating a Fair Comparison of Research Ideas.” 2010 IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS 2010). Piscataway, NJ, USA: IEEE, 2010. 45–54. Print.
@inproceedings{1006886,
  abstract     = {While reproducing the experimental results of research articles is standard practice in mature domains of science, such as physics or biology, it has not yet become mainstream in computer architecture. However, recent research shows that the lack of a fair and broad comparison of research ideas can be significantly detrimental to the progress, and thus the productivity, of research. At the same time, the complexity of architecture simulators and the fact that simulators are not systematically disseminated with novel ideas are largely responsible for this situation. While this methodology has a fundamental impact on research, it is by essence a practical issue. In this article, we present and set up an a typical approach to overcome this practical methodology issue, which takes the form of an open and continuous exploration through ArchExplorer, a server-side web infrastructure, that can significantly ease the process of fairly and quantitatively comparing research ideas. The web infrastructure ArchExplorer.org is now publicly open, and we demonstrate the approach with a set of data cache mechanisms. We show that this broad exploration can challenge some earlier assessments about data cache research, and even challenges the conclusions of an earlier but less thorough study on data cache comparison.},
  author       = {Desmet, Veerle and Girbal, Sylvain and Temam, Olivier},
  booktitle    = {2010 IEEE International Symposium on Performance Analysis of Systems \& Software (ISPASS 2010)},
  isbn         = {9781424460229},
  keyword      = {cache storage,architecture simulator complexity,ArchExplorer.org,data cache mechanisms,Internet,server-side Web infrastructure,computer architecture},
  language     = {eng},
  location     = {White Plains, NY, USA},
  pages        = {45--54},
  publisher    = {IEEE},
  title        = {ArchExplorer.org: a methodology for facilitating a fair comparison of research ideas},
  url          = {http://dx.doi.org/10.1109/ISPASS.2010.5452068},
  year         = {2010},
}

Altmetric
View in Altmetric
Web of Science
Times cited: