Advanced search
1 file | 865.99 KB Add to list

Unreliable yet still replicable: a comment on LeBel and Paunonen (2011)

Maarten De Schryver (UGent) , Sean Joseph Hughes (UGent) , Yves Rosseel (UGent) and Jan De Houwer (UGent)
Author
Organization
Abstract
Lebel and Paunonen (2011) highlight that despite their importance and popularity in both theoretical and applied research, many implicit measures continue to be plagued by a persistent and troublesome issue low reliability. In their paper, they offer a conceptual analysis of the relationship between reliability, power and replicability, and then provide a series of recommendations for researchers interested in using implicit measures in an experimental setting. At the core of their account is the idea that reliability can be equated with statistical power, such that "lower levels of reliability are associated with decreasing probabilities of detecting a statistically significant effect, given one exists in the population" (p. 573). They also take the additional step of equating reliability and replicability. In our commentary, we draw attention to the fact that there is no direct, fixed or one-to-one relation between reliability and power or replicability. More specifically, we argue that when adopting an experimental (rather than a correlational) approach, researchers strive to minimize inter-individual variation, which has a direct impact on sample based reliability estimates. We evaluate the strengths and weaknesses of the LeBel and Paunonen's recommendations and refine them where appropriate.
Keywords
MEASUREMENT ERROR, IMPLICIT ASSOCIATION TEST, power, reliability, INDIVIDUAL-DIFFERENCES, DIFFERENCE SCORES, POWER, RELIABILITY, COGNITION, PARADOX, ATTITUDES, VALIDITY, implicit measures, replication

Downloads

  • (...).pdf
    • full text
    • |
    • UGent only
    • |
    • PDF
    • |
    • 865.99 KB

Citation

Please use this url to cite or link to this publication:

MLA
De Schryver, Maarten et al. “Unreliable yet Still Replicable: a Comment on LeBel and Paunonen (2011).” FRONTIERS IN PSYCHOLOGY 6 (2016): n. pag. Print.
APA
De Schryver, M., Hughes, S. J., Rosseel, Y., & De Houwer, J. (2016). Unreliable yet still replicable: a comment on LeBel and Paunonen (2011). FRONTIERS IN PSYCHOLOGY, 6.
Chicago author-date
De Schryver, Maarten, Sean Joseph Hughes, Yves Rosseel, and Jan De Houwer. 2016. “Unreliable yet Still Replicable: a Comment on LeBel and Paunonen (2011).” Frontiers in Psychology 6.
Chicago author-date (all authors)
De Schryver, Maarten, Sean Joseph Hughes, Yves Rosseel, and Jan De Houwer. 2016. “Unreliable yet Still Replicable: a Comment on LeBel and Paunonen (2011).” Frontiers in Psychology 6.
Vancouver
1.
De Schryver M, Hughes SJ, Rosseel Y, De Houwer J. Unreliable yet still replicable: a comment on LeBel and Paunonen (2011). FRONTIERS IN PSYCHOLOGY. 2016;6.
IEEE
[1]
M. De Schryver, S. J. Hughes, Y. Rosseel, and J. De Houwer, “Unreliable yet still replicable: a comment on LeBel and Paunonen (2011),” FRONTIERS IN PSYCHOLOGY, vol. 6, 2016.
@article{7048356,
  abstract     = {Lebel and Paunonen (2011) highlight that despite their importance and popularity in both theoretical and applied research, many implicit measures continue to be plagued by a persistent and troublesome issue low reliability. In their paper, they offer a conceptual analysis of the relationship between reliability, power and replicability, and then provide a series of recommendations for researchers interested in using implicit measures in an experimental setting. At the core of their account is the idea that reliability can be equated with statistical power, such that "lower levels of reliability are associated with decreasing probabilities of detecting a statistically significant effect, given one exists in the population" (p. 573). They also take the additional step of equating reliability and replicability. In our commentary, we draw attention to the fact that there is no direct, fixed or one-to-one relation between reliability and power or replicability. More specifically, we argue that when adopting an experimental (rather than a correlational) approach, researchers strive to minimize inter-individual variation, which has a direct impact on sample based reliability estimates. We evaluate the strengths and weaknesses of the LeBel and Paunonen's recommendations and refine them where appropriate.},
  articleno    = {2039},
  author       = {De Schryver, Maarten and Hughes, Sean Joseph and Rosseel, Yves and De Houwer, Jan},
  issn         = {1664-1078},
  journal      = {FRONTIERS IN PSYCHOLOGY},
  keywords     = {MEASUREMENT ERROR,IMPLICIT ASSOCIATION TEST,power,reliability,INDIVIDUAL-DIFFERENCES,DIFFERENCE SCORES,POWER,RELIABILITY,COGNITION,PARADOX,ATTITUDES,VALIDITY,implicit measures,replication},
  language     = {eng},
  title        = {Unreliable yet still replicable: a comment on LeBel and Paunonen (2011)},
  url          = {http://dx.doi.org/10.3389/fpsyg.2015.02039},
  volume       = {6},
  year         = {2016},
}

Altmetric
View in Altmetric
Web of Science
Times cited: