Ghent University Academic Bibliography

Advanced

Unreliable yet still replicable: a comment on LeBel and Paunonen (2011)

Maarten De Schryver UGent, Sean Joseph Hughes UGent, Yves Rosseel UGent and Jan De Houwer UGent (2016) FRONTIERS IN PSYCHOLOGY. 6.
abstract
Lebel and Paunonen (2011) highlight that despite their importance and popularity in both theoretical and applied research, many implicit measures continue to be plagued by a persistent and troublesome issue low reliability. In their paper, they offer a conceptual analysis of the relationship between reliability, power and replicability, and then provide a series of recommendations for researchers interested in using implicit measures in an experimental setting. At the core of their account is the idea that reliability can be equated with statistical power, such that "lower levels of reliability are associated with decreasing probabilities of detecting a statistically significant effect, given one exists in the population" (p. 573). They also take the additional step of equating reliability and replicability. In our commentary, we draw attention to the fact that there is no direct, fixed or one-to-one relation between reliability and power or replicability. More specifically, we argue that when adopting an experimental (rather than a correlational) approach, researchers strive to minimize inter-individual variation, which has a direct impact on sample based reliability estimates. We evaluate the strengths and weaknesses of the LeBel and Paunonen's recommendations and refine them where appropriate.
Please use this url to cite or link to this publication:
author
organization
year
type
journalArticle (original)
publication status
published
keyword
MEASUREMENT ERROR, IMPLICIT ASSOCIATION TEST, power, reliability, INDIVIDUAL-DIFFERENCES, DIFFERENCE SCORES, POWER, RELIABILITY, COGNITION, PARADOX, ATTITUDES, VALIDITY, implicit measures, replication
journal title
FRONTIERS IN PSYCHOLOGY
volume
6
article number
2039
Web of Science type
Article
Web of Science id
000368008500001
JCR category
PSYCHOLOGY, MULTIDISCIPLINARY
JCR impact factor
2.323 (2016)
JCR rank
33/128 (2016)
JCR quartile
2 (2016)
ISSN
1664-1078
DOI
10.3389/fpsyg.2015.02039
language
English
UGent publication?
yes
classification
A1
copyright statement
I have transferred the copyright for this publication to the publisher
id
7048356
handle
http://hdl.handle.net/1854/LU-7048356
date created
2016-01-22 16:29:47
date last changed
2016-12-21 15:41:53
@article{7048356,
  abstract     = {Lebel and Paunonen (2011) highlight that despite their importance and popularity in both theoretical and applied research, many implicit measures continue to be plagued by a persistent and troublesome issue low reliability. In their paper, they offer a conceptual analysis of the relationship between reliability, power and replicability, and then provide a series of recommendations for researchers interested in using implicit measures in an experimental setting. At the core of their account is the idea that reliability can be equated with statistical power, such that {\textacutedbl}lower levels of reliability are associated with decreasing probabilities of detecting a statistically significant effect, given one exists in the population{\textacutedbl} (p. 573). They also take the additional step of equating reliability and replicability. In our commentary, we draw attention to the fact that there is no direct, fixed or one-to-one relation between reliability and power or replicability. More specifically, we argue that when adopting an experimental (rather than a correlational) approach, researchers strive to minimize inter-individual variation, which has a direct impact on sample based reliability estimates. We evaluate the strengths and weaknesses of the LeBel and Paunonen's recommendations and refine them where appropriate.},
  articleno    = {2039},
  author       = {De Schryver, Maarten and Hughes, Sean Joseph and Rosseel, Yves and De Houwer, Jan},
  issn         = {1664-1078},
  journal      = {FRONTIERS IN PSYCHOLOGY},
  keyword      = {MEASUREMENT ERROR,IMPLICIT ASSOCIATION TEST,power,reliability,INDIVIDUAL-DIFFERENCES,DIFFERENCE SCORES,POWER,RELIABILITY,COGNITION,PARADOX,ATTITUDES,VALIDITY,implicit measures,replication},
  language     = {eng},
  title        = {Unreliable yet still replicable: a comment on LeBel and Paunonen (2011)},
  url          = {http://dx.doi.org/10.3389/fpsyg.2015.02039},
  volume       = {6},
  year         = {2016},
}

Chicago
De Schryver, Maarten, Sean Joseph Hughes, Yves Rosseel, and Jan De Houwer. 2016. “Unreliable yet Still Replicable: a Comment on LeBel and Paunonen (2011).” Frontiers in Psychology 6.
APA
De Schryver, M., Hughes, S. J., Rosseel, Y., & De Houwer, J. (2016). Unreliable yet still replicable: a comment on LeBel and Paunonen (2011). FRONTIERS IN PSYCHOLOGY, 6.
Vancouver
1.
De Schryver M, Hughes SJ, Rosseel Y, De Houwer J. Unreliable yet still replicable: a comment on LeBel and Paunonen (2011). FRONTIERS IN PSYCHOLOGY. 2016;6.
MLA
De Schryver, Maarten, Sean Joseph Hughes, Yves Rosseel, et al. “Unreliable yet Still Replicable: a Comment on LeBel and Paunonen (2011).” FRONTIERS IN PSYCHOLOGY 6 (2016): n. pag. Print.