Advanced search
1 file | 911.60 KB Add to list

Evidence of validity does not rule out systematic bias : a commentary on nomological noise and cross-cultural invariance

(2023) SOCIOLOGICAL METHODS & RESEARCH. 52(3). p.1420-1437
Author
Organization
Abstract
We comment on the argument by Wetzel, Brunkert, Kruse and Inglehart (2021) that theoretically expected associations in nomological networks should take priority over invariance tests in cross-national research. We agree that narrow application of individual tools, such as multi-group confirmatory factor analysis with data that violates the assumptions of these techniques, can be misleading. However, findings that fit expectations of nomological networks may not be free of bias. We present supporting evidence of systematic bias affecting nomological network relationships from a) previous research on intelligence and response styles, b) two simulation studies, and c) data on the choice index from the World Value Survey (WVS). Our main point is that nomological network analysis by itself is insufficient to rule out systematic bias in data. We point out how a thoughtful exploration of sources of biases in cross-national data can contribute to stronger theory development.
Keywords
RESPONSE STYLES, STRUCTURAL EQUIVALENCE, FIT INDEXES, VALUES, Invariance, nomological networks, systematic bias, values, cross-cultural differences, multilevel models, simulation, choice index

Downloads

  • (...).pdf
    • full text (Published version)
    • |
    • UGent only
    • |
    • PDF
    • |
    • 911.60 KB

Citation

Please use this url to cite or link to this publication:

MLA
Fischer, Ronald, et al. “Evidence of Validity Does Not Rule out Systematic Bias : A Commentary on Nomological Noise and Cross-Cultural Invariance.” SOCIOLOGICAL METHODS & RESEARCH, vol. 52, no. 3, 2023, pp. 1420–37, doi:10.1177/00491241221091756.
APA
Fischer, R., Karl, J. A., Fontaine, J., & Poortinga, Y. H. (2023). Evidence of validity does not rule out systematic bias : a commentary on nomological noise and cross-cultural invariance. SOCIOLOGICAL METHODS & RESEARCH, 52(3), 1420–1437. https://doi.org/10.1177/00491241221091756
Chicago author-date
Fischer, Ronald, Johannes Alfons Karl, Johnny Fontaine, and Ype H. Poortinga. 2023. “Evidence of Validity Does Not Rule out Systematic Bias : A Commentary on Nomological Noise and Cross-Cultural Invariance.” SOCIOLOGICAL METHODS & RESEARCH 52 (3): 1420–37. https://doi.org/10.1177/00491241221091756.
Chicago author-date (all authors)
Fischer, Ronald, Johannes Alfons Karl, Johnny Fontaine, and Ype H. Poortinga. 2023. “Evidence of Validity Does Not Rule out Systematic Bias : A Commentary on Nomological Noise and Cross-Cultural Invariance.” SOCIOLOGICAL METHODS & RESEARCH 52 (3): 1420–1437. doi:10.1177/00491241221091756.
Vancouver
1.
Fischer R, Karl JA, Fontaine J, Poortinga YH. Evidence of validity does not rule out systematic bias : a commentary on nomological noise and cross-cultural invariance. SOCIOLOGICAL METHODS & RESEARCH. 2023;52(3):1420–37.
IEEE
[1]
R. Fischer, J. A. Karl, J. Fontaine, and Y. H. Poortinga, “Evidence of validity does not rule out systematic bias : a commentary on nomological noise and cross-cultural invariance,” SOCIOLOGICAL METHODS & RESEARCH, vol. 52, no. 3, pp. 1420–1437, 2023.
@article{01H48EVTX1WVBEQREDGA6CMKT9,
  abstract     = {{We comment on the argument by Wetzel, Brunkert, Kruse and Inglehart (2021) that theoretically expected associations in nomological networks should take priority over invariance tests in cross-national research. We agree that narrow application of individual tools, such as multi-group confirmatory factor analysis with data that violates the assumptions of these techniques, can be misleading. However, findings that fit expectations of nomological networks may not be free of bias. We present supporting evidence of systematic bias affecting nomological network relationships from a) previous research on intelligence and response styles, b) two simulation studies, and c) data on the choice index from the World Value Survey (WVS). Our main point is that nomological network analysis by itself is insufficient to rule out systematic bias in data. We point out how a thoughtful exploration of sources of biases in cross-national data can contribute to stronger theory development.}},
  author       = {{Fischer, Ronald and  Karl, Johannes Alfons and Fontaine, Johnny and  Poortinga, Ype H.}},
  issn         = {{0049-1241}},
  journal      = {{SOCIOLOGICAL METHODS & RESEARCH}},
  keywords     = {{RESPONSE STYLES,STRUCTURAL EQUIVALENCE,FIT INDEXES,VALUES,Invariance,nomological networks,systematic bias,values,cross-cultural differences,multilevel models,simulation,choice index}},
  language     = {{eng}},
  number       = {{3}},
  pages        = {{1420--1437}},
  title        = {{Evidence of validity does not rule out systematic bias : a commentary on nomological noise and cross-cultural invariance}},
  url          = {{http://doi.org/10.1177/00491241221091756}},
  volume       = {{52}},
  year         = {{2023}},
}

Altmetric
View in Altmetric
Web of Science
Times cited: