Advanced search
1 file | 1.27 MB Add to list

Data-analytical stability of cluster-wise and peak-wise inference in fMRI data analysis

Sanne Roels (UGent) , Han Bossier (UGent) , Tom Loeys (UGent) and Beatrijs Moerkerke (UGent)
Author
Organization
Abstract
Background: Carp (2012) demonstrated the large variability that is present in the method sections of fMRI studies. This methodological variability between studies limits reproducible research. New method: Evaluation protocols for methods used in fMRI should include data-analytical stability measures quantifying the variability in results following choices in the methods. Data-analytical stability can be seen as a proxy for reproducibility. To illustrate how one can perform such evaluations, we study two competing approaches for topological feature based inference (random field theory and permutation based testing) and two competing methods for smoothing (Gaussian smoothing and adaptive smoothing). We compare these approaches from the perspective of data-analytical stability in real data, and additionally consider validity and reliability in simulations. Results: There is clear evidence that choices in the methods impact the validity, reliability and stability of the results. For the particular comparison studied, we find that permutation based methods render the most valid results. For stability and reliability, the performance of different smoothing and inference types depends on the setting. However, while being more reliable, adaptive smoothing can evoke less stable results when using larger kernel width, especially with cluster size based permutation inference. Comparison with existing methods: While existing evaluation methods focus on validity and reliability, we show that data-analytical stability enables to further distinguish between performance of different methods. Conclusion: Data-analytical stability is an important additional criterion that can easily be incorporated in evaluation protocols.
Keywords
RELIABILITY, FRAMEWORK, REPRODUCIBILITY, SIZE INFERENCE, COMPONENT ANALYSIS, NPAIRS, PERMUTATION METHODS, STATISTICAL-ANALYSIS, Reproducibility, KeyWords Plus:RANDOM-FIELD THEORY, Stability, R PACKAGE, Author Keywords:fMRI, Reliability

Downloads

  • (...).pdf
    • full text
    • |
    • UGent only
    • |
    • PDF
    • |
    • 1.27 MB

Citation

Please use this url to cite or link to this publication:

MLA
Roels, Sanne, Han Bossier, Tom Loeys, et al. “Data-analytical Stability of Cluster-wise and Peak-wise Inference in fMRI Data Analysis.” JOURNAL OF NEUROSCIENCE METHODS 240 (2015): 37–47. Print.
APA
Roels, Sanne, Bossier, H., Loeys, T., & Moerkerke, B. (2015). Data-analytical stability of cluster-wise and peak-wise inference in fMRI data analysis. JOURNAL OF NEUROSCIENCE METHODS, 240, 37–47.
Chicago author-date
Roels, Sanne, Han Bossier, Tom Loeys, and Beatrijs Moerkerke. 2015. “Data-analytical Stability of Cluster-wise and Peak-wise Inference in fMRI Data Analysis.” Journal of Neuroscience Methods 240: 37–47.
Chicago author-date (all authors)
Roels, Sanne, Han Bossier, Tom Loeys, and Beatrijs Moerkerke. 2015. “Data-analytical Stability of Cluster-wise and Peak-wise Inference in fMRI Data Analysis.” Journal of Neuroscience Methods 240: 37–47.
Vancouver
1.
Roels S, Bossier H, Loeys T, Moerkerke B. Data-analytical stability of cluster-wise and peak-wise inference in fMRI data analysis. JOURNAL OF NEUROSCIENCE METHODS. 2015;240:37–47.
IEEE
[1]
S. Roels, H. Bossier, T. Loeys, and B. Moerkerke, “Data-analytical stability of cluster-wise and peak-wise inference in fMRI data analysis,” JOURNAL OF NEUROSCIENCE METHODS, vol. 240, pp. 37–47, 2015.
@article{5827408,
  abstract     = {Background: Carp (2012) demonstrated the large variability that is present in the method sections of fMRI studies. This methodological variability between studies limits reproducible research.
New method: Evaluation protocols for methods used in fMRI should include data-analytical stability measures quantifying the variability in results following choices in the methods. Data-analytical stability can be seen as a proxy for reproducibility. To illustrate how one can perform such evaluations, we study two competing approaches for topological feature based inference (random field theory and permutation based testing) and two competing methods for smoothing (Gaussian smoothing and adaptive smoothing).
We compare these approaches from the perspective of data-analytical stability in real data, and  additionally consider validity and reliability in simulations.
Results: There is clear evidence that choices in the methods impact the validity, reliability and stability of the results. For the particular comparison studied, we find that permutation based methods render the most valid results. For stability and reliability, the performance of different smoothing and inference types depends on the setting. However, while being more reliable, adaptive smoothing can evoke less
stable results when using larger kernel width, especially with cluster size based permutation inference.
Comparison with existing methods: While existing evaluation methods focus on validity and reliability, we show that data-analytical stability enables to further distinguish between performance of different methods.
Conclusion: Data-analytical stability is an important additional criterion that can easily be incorporated in evaluation protocols.},
  articleno    = {PMID: 25445059},
  author       = {Roels, Sanne and Bossier, Han and Loeys, Tom and Moerkerke, Beatrijs},
  issn         = {0165-0270},
  journal      = {JOURNAL OF NEUROSCIENCE METHODS},
  keywords     = {RELIABILITY,FRAMEWORK,REPRODUCIBILITY,SIZE INFERENCE,COMPONENT ANALYSIS,NPAIRS,PERMUTATION METHODS,STATISTICAL-ANALYSIS,Reproducibility,KeyWords Plus:RANDOM-FIELD THEORY,Stability,R PACKAGE,Author Keywords:fMRI,Reliability},
  language     = {eng},
  pages        = {PMID: 25445059:37--PMID: 25445059:47},
  title        = {Data-analytical stability of cluster-wise and peak-wise inference in fMRI data analysis},
  url          = {http://dx.doi.org/10.1016/j.jneumeth.2014.10.024},
  volume       = {240},
  year         = {2015},
}

Altmetric
View in Altmetric
Web of Science
Times cited: