
Evidence of validity does not rule out systematic bias : a commentary on nomological noise and cross-cultural invariance
- Author
- Ronald Fischer, Johannes Alfons Karl, Johnny Fontaine (UGent) and Ype H. Poortinga
- Organization
- Abstract
- We comment on the argument by Wetzel, Brunkert, Kruse and Inglehart (2021) that theoretically expected associations in nomological networks should take priority over invariance tests in cross-national research. We agree that narrow application of individual tools, such as multi-group confirmatory factor analysis with data that violates the assumptions of these techniques, can be misleading. However, findings that fit expectations of nomological networks may not be free of bias. We present supporting evidence of systematic bias affecting nomological network relationships from a) previous research on intelligence and response styles, b) two simulation studies, and c) data on the choice index from the World Value Survey (WVS). Our main point is that nomological network analysis by itself is insufficient to rule out systematic bias in data. We point out how a thoughtful exploration of sources of biases in cross-national data can contribute to stronger theory development.
- Keywords
- RESPONSE STYLES, STRUCTURAL EQUIVALENCE, FIT INDEXES, VALUES, Invariance, nomological networks, systematic bias, values, cross-cultural differences, multilevel models, simulation, choice index
Downloads
-
(...).pdf
- full text (Published version)
- |
- UGent only
- |
- |
- 911.60 KB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-01H48EVTX1WVBEQREDGA6CMKT9
- MLA
- Fischer, Ronald, et al. “Evidence of Validity Does Not Rule out Systematic Bias : A Commentary on Nomological Noise and Cross-Cultural Invariance.” SOCIOLOGICAL METHODS & RESEARCH, vol. 52, no. 3, 2023, pp. 1420–37, doi:10.1177/00491241221091756.
- APA
- Fischer, R., Karl, J. A., Fontaine, J., & Poortinga, Y. H. (2023). Evidence of validity does not rule out systematic bias : a commentary on nomological noise and cross-cultural invariance. SOCIOLOGICAL METHODS & RESEARCH, 52(3), 1420–1437. https://doi.org/10.1177/00491241221091756
- Chicago author-date
- Fischer, Ronald, Johannes Alfons Karl, Johnny Fontaine, and Ype H. Poortinga. 2023. “Evidence of Validity Does Not Rule out Systematic Bias : A Commentary on Nomological Noise and Cross-Cultural Invariance.” SOCIOLOGICAL METHODS & RESEARCH 52 (3): 1420–37. https://doi.org/10.1177/00491241221091756.
- Chicago author-date (all authors)
- Fischer, Ronald, Johannes Alfons Karl, Johnny Fontaine, and Ype H. Poortinga. 2023. “Evidence of Validity Does Not Rule out Systematic Bias : A Commentary on Nomological Noise and Cross-Cultural Invariance.” SOCIOLOGICAL METHODS & RESEARCH 52 (3): 1420–1437. doi:10.1177/00491241221091756.
- Vancouver
- 1.Fischer R, Karl JA, Fontaine J, Poortinga YH. Evidence of validity does not rule out systematic bias : a commentary on nomological noise and cross-cultural invariance. SOCIOLOGICAL METHODS & RESEARCH. 2023;52(3):1420–37.
- IEEE
- [1]R. Fischer, J. A. Karl, J. Fontaine, and Y. H. Poortinga, “Evidence of validity does not rule out systematic bias : a commentary on nomological noise and cross-cultural invariance,” SOCIOLOGICAL METHODS & RESEARCH, vol. 52, no. 3, pp. 1420–1437, 2023.
@article{01H48EVTX1WVBEQREDGA6CMKT9, abstract = {{We comment on the argument by Wetzel, Brunkert, Kruse and Inglehart (2021) that theoretically expected associations in nomological networks should take priority over invariance tests in cross-national research. We agree that narrow application of individual tools, such as multi-group confirmatory factor analysis with data that violates the assumptions of these techniques, can be misleading. However, findings that fit expectations of nomological networks may not be free of bias. We present supporting evidence of systematic bias affecting nomological network relationships from a) previous research on intelligence and response styles, b) two simulation studies, and c) data on the choice index from the World Value Survey (WVS). Our main point is that nomological network analysis by itself is insufficient to rule out systematic bias in data. We point out how a thoughtful exploration of sources of biases in cross-national data can contribute to stronger theory development.}}, author = {{Fischer, Ronald and Karl, Johannes Alfons and Fontaine, Johnny and Poortinga, Ype H.}}, issn = {{0049-1241}}, journal = {{SOCIOLOGICAL METHODS & RESEARCH}}, keywords = {{RESPONSE STYLES,STRUCTURAL EQUIVALENCE,FIT INDEXES,VALUES,Invariance,nomological networks,systematic bias,values,cross-cultural differences,multilevel models,simulation,choice index}}, language = {{eng}}, number = {{3}}, pages = {{1420--1437}}, title = {{Evidence of validity does not rule out systematic bias : a commentary on nomological noise and cross-cultural invariance}}, url = {{http://doi.org/10.1177/00491241221091756}}, volume = {{52}}, year = {{2023}}, }
- Altmetric
- View in Altmetric
- Web of Science
- Times cited: