Advanced search
1 file | 674.48 KB Add to list

Multiple testing in fMRI: an empirical case study on the balance between sensitivity, specificity, and stability

Joke Durnez (UGent) , Sanne Roels (UGent) and Beatrijs Moerkerke (UGent)
(2014) BIOMETRICAL JOURNAL. 56(4). p.649-661
Author
Organization
Abstract
Functional Magnetic Resonance Imaging is a widespread technique in cognitive psychology that allows visualizing brain activation. The data analysis encompasses an enormous number of simultaneous statistical tests. Procedures that either control the familywise error rate or the false discovery rate have been applied to these data. These methods are mostly validated in terms of average sensitivity and specificity. However, procedures are not comparable if requirements on their error rates differ. Moreover, less attention has been given to the instability or variability of results. In a simulation study in the context of imaging, we first compare the Bonferroni and Benjamini-Hochberg procedures. Considering Bonferroni as away to control the expected number of type I errors enables more lenient thresholding compared to familywise error rate control and a direct comparison between both procedures. We point out that while the same balance is obtained between average sensitivity and specificity, the Benjamini-Hochberg procedure appears less stable. Secondly, we have implemented the procedure of Gordon et al. (2009) (originally proposed for gene selection) that includes stability, measured through bootstrapping, in the decision criterion. Simulations indicate that the method attains the same balance between sensitivity and specificity. It improves the stability of Benjamini-Hochberg but does not outperform Bonferroni, making this computationally heavy bootstrap procedure less appealing. Third, we show how stability of thresholding procedures can be assessed using real data. In a dataset on face recognition, we again find that Bonferroni renders more stable results.
Keywords
Sensitivity, Stability, Neuroimaging, Multiple testing, fMRI, NUMBER, THRESHOLDING TECHNIQUES, FALSE DISCOVERY RATE

Downloads

  • (...).pdf
    • full text
    • |
    • UGent only
    • |
    • PDF
    • |
    • 674.48 KB

Citation

Please use this url to cite or link to this publication:

MLA
Durnez, Joke, Sanne Roels, and Beatrijs Moerkerke. “Multiple Testing in fMRI: An Empirical Case Study on the Balance Between Sensitivity, Specificity, and Stability.” BIOMETRICAL JOURNAL 56.4 (2014): 649–661. Print.
APA
Durnez, J., Roels, S., & Moerkerke, B. (2014). Multiple testing in fMRI: an empirical case study on the balance between sensitivity, specificity, and stability. BIOMETRICAL JOURNAL, 56(4), 649–661.
Chicago author-date
Durnez, Joke, Sanne Roels, and Beatrijs Moerkerke. 2014. “Multiple Testing in fMRI: An Empirical Case Study on the Balance Between Sensitivity, Specificity, and Stability.” Biometrical Journal 56 (4): 649–661.
Chicago author-date (all authors)
Durnez, Joke, Sanne Roels, and Beatrijs Moerkerke. 2014. “Multiple Testing in fMRI: An Empirical Case Study on the Balance Between Sensitivity, Specificity, and Stability.” Biometrical Journal 56 (4): 649–661.
Vancouver
1.
Durnez J, Roels S, Moerkerke B. Multiple testing in fMRI: an empirical case study on the balance between sensitivity, specificity, and stability. BIOMETRICAL JOURNAL. 2014;56(4):649–61.
IEEE
[1]
J. Durnez, S. Roels, and B. Moerkerke, “Multiple testing in fMRI: an empirical case study on the balance between sensitivity, specificity, and stability,” BIOMETRICAL JOURNAL, vol. 56, no. 4, pp. 649–661, 2014.
@article{5756134,
  abstract     = {Functional Magnetic Resonance Imaging is a widespread technique in cognitive psychology that allows visualizing brain activation. The data analysis encompasses an enormous number of simultaneous statistical tests. Procedures that either control the familywise error rate or the false discovery rate have been applied to these data. These methods are mostly validated in terms of average sensitivity and specificity. However, procedures are not comparable if requirements on their error rates differ. Moreover, less attention has been given to the instability or variability of results. In a simulation study in the context of imaging, we first compare the Bonferroni and Benjamini-Hochberg procedures. Considering Bonferroni as away to control the expected number of type I errors enables more lenient thresholding compared to familywise error rate control and a direct comparison between both procedures. We point out that while the same balance is obtained between average sensitivity and specificity, the Benjamini-Hochberg procedure appears less stable. Secondly, we have implemented the procedure of Gordon et al. (2009) (originally proposed for gene selection) that includes stability, measured through bootstrapping, in the decision criterion. Simulations indicate that the method attains the same balance between sensitivity and specificity. It improves the stability of Benjamini-Hochberg but does not outperform Bonferroni, making this computationally heavy bootstrap procedure less appealing. Third, we show how stability of thresholding procedures can be assessed using real data. In a dataset on face recognition, we again find that Bonferroni renders more stable results.},
  author       = {Durnez, Joke and Roels, Sanne and Moerkerke, Beatrijs},
  issn         = {0323-3847},
  journal      = {BIOMETRICAL JOURNAL},
  keywords     = {Sensitivity,Stability,Neuroimaging,Multiple testing,fMRI,NUMBER,THRESHOLDING TECHNIQUES,FALSE DISCOVERY RATE},
  language     = {eng},
  number       = {4},
  pages        = {649--661},
  title        = {Multiple testing in fMRI: an empirical case study on the balance between sensitivity, specificity, and stability},
  url          = {http://dx.doi.org/10.1002/bimj.201200056},
  volume       = {56},
  year         = {2014},
}

Altmetric
View in Altmetric
Web of Science
Times cited: