Ghent University Academic Bibliography

Advanced

ArchExplorer.org: a methodology for facilitating a fair comparison of research ideas

Veerle Desmet UGent, Sylvain Girbal and Olivier Temam (2010) 2010 IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS 2010). p.45-54
abstract
While reproducing the experimental results of research articles is standard practice in mature domains of science, such as physics or biology, it has not yet become mainstream in computer architecture. However, recent research shows that the lack of a fair and broad comparison of research ideas can be significantly detrimental to the progress, and thus the productivity, of research. At the same time, the complexity of architecture simulators and the fact that simulators are not systematically disseminated with novel ideas are largely responsible for this situation. While this methodology has a fundamental impact on research, it is by essence a practical issue. In this article, we present and set up an a typical approach to overcome this practical methodology issue, which takes the form of an open and continuous exploration through ArchExplorer, a server-side web infrastructure, that can significantly ease the process of fairly and quantitatively comparing research ideas. The web infrastructure ArchExplorer.org is now publicly open, and we demonstrate the approach with a set of data cache mechanisms. We show that this broad exploration can challenge some earlier assessments about data cache research, and even challenges the conclusions of an earlier but less thorough study on data cache comparison.
Please use this url to cite or link to this publication:
author
organization
year
type
conference (conferencePaper)
publication status
published
subject
keyword
cache storage, architecture simulator complexity, ArchExplorer.org, data cache mechanisms, Internet, server-side Web infrastructure, computer architecture
in
2010 IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS 2010)
pages
45 - 54
publisher
IEEE
place of publication
Piscataway, NJ, USA
conference name
2010 IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS - 2010)
conference location
White Plains, NY, USA
conference start
2010-03-28
conference end
2010-03-30
Web of Science type
Conference Paper
Web of Science id
11259915
ISBN
9781424460229
9781424460236
DOI
10.1109/ISPASS.2010.5452068
language
English
UGent publication?
yes
classification
C1
copyright statement
I have transferred the copyright for this publication to the publisher
id
1006886
handle
http://hdl.handle.net/1854/LU-1006886
date created
2010-07-09 08:37:40
date last changed
2017-01-02 09:52:38
@inproceedings{1006886,
  abstract     = {While reproducing the experimental results of research articles is standard practice in mature domains of science, such as physics or biology, it has not yet become mainstream in computer architecture. However, recent research shows that the lack of a fair and broad comparison of research ideas can be significantly detrimental to the progress, and thus the productivity, of research. At the same time, the complexity of architecture simulators and the fact that simulators are not systematically disseminated with novel ideas are largely responsible for this situation. While this methodology has a fundamental impact on research, it is by essence a practical issue. In this article, we present and set up an a typical approach to overcome this practical methodology issue, which takes the form of an open and continuous exploration through ArchExplorer, a server-side web infrastructure, that can significantly ease the process of fairly and quantitatively comparing research ideas. The web infrastructure ArchExplorer.org is now publicly open, and we demonstrate the approach with a set of data cache mechanisms. We show that this broad exploration can challenge some earlier assessments about data cache research, and even challenges the conclusions of an earlier but less thorough study on data cache comparison.},
  author       = {Desmet, Veerle and Girbal, Sylvain and Temam, Olivier},
  booktitle    = {2010 IEEE International Symposium on Performance Analysis of Systems \& Software (ISPASS 2010)},
  isbn         = {9781424460229},
  keyword      = {cache storage,architecture simulator complexity,ArchExplorer.org,data cache mechanisms,Internet,server-side Web infrastructure,computer architecture},
  language     = {eng},
  location     = {White Plains, NY, USA},
  pages        = {45--54},
  publisher    = {IEEE},
  title        = {ArchExplorer.org: a methodology for facilitating a fair comparison of research ideas},
  url          = {http://dx.doi.org/10.1109/ISPASS.2010.5452068},
  year         = {2010},
}

Chicago
Desmet, Veerle, Sylvain Girbal, and Olivier Temam. 2010. “ArchExplorer.org: a Methodology for Facilitating a Fair Comparison of Research Ideas.” In 2010 IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS 2010), 45–54. Piscataway, NJ, USA: IEEE.
APA
Desmet, V., Girbal, S., & Temam, O. (2010). ArchExplorer.org: a methodology for facilitating a fair comparison of research ideas. 2010 IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS 2010) (pp. 45–54). Presented at the 2010 IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS - 2010), Piscataway, NJ, USA: IEEE.
Vancouver
1.
Desmet V, Girbal S, Temam O. ArchExplorer.org: a methodology for facilitating a fair comparison of research ideas. 2010 IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS 2010). Piscataway, NJ, USA: IEEE; 2010. p. 45–54.
MLA
Desmet, Veerle, Sylvain Girbal, and Olivier Temam. “ArchExplorer.org: a Methodology for Facilitating a Fair Comparison of Research Ideas.” 2010 IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS 2010). Piscataway, NJ, USA: IEEE, 2010. 45–54. Print.