Ghent University Academic Bibliography

Advanced

All have won what? On the epistemic value of RCT based evidence in Evidence Based Treatment

Femke Truijens UGent and Mattias Desmet UGent (2015)
abstract
Aim “All have won, and all must have prizes,” - so the dodo bird verdicts the statistical indifference between ‘evidence based’ types of psychotherapy. Evidence is understood as the result of randomized controlled (RCT) research. This ‘golden standard’ design requires samples that are homogeneous with regard to symptom(s) - following from the statistical assumptions in central tendency statistics. Symptom specific measures are used to select patients scoring above cut-off, resulting in samples characterized by tight and simple symptom patterns. Whereas the external validity of this procedure is heavily critiqued, I study the internal validity of a priori methodological assumptions in RCT efficacy research. In this paper, I address the clinical and epistemic plausibility of sample selection by a quantified symptom measure for depression, the BDI-II, which was found to be a reliable screening measure for Major Depressive Disorder. Method In a pilot study on our mixed method psychotherapy data from depressed adults, we used the BDI-II to select patients eligible to participate in a strict RCT design. For both the eligible and non-eligible sample, we scrutinized individual symptom patterns. Results Analysis showed a 60-40 ratio of depressed patients to be included or excluded, respectively. 75% of the ‘eligible’ sample still showed comorbidity, which would have been reason to exclude them from a RCT. Discussion The extraordinary large exclusion percentage indeed inserts doubt on external and ecological validity of quantified eligibility screening in RCT efficacy research. More striking however is that a supposedly ‘eligible sample’ still harms the assumption of homogeneity. In this paper, I propose such invalidity of a priori methodological assumptions as an explanation for the dodo bird effect. I discuss the implications of these preliminary empirical results for the understanding of ‘evidence’ from RCTs and the epistemic value of Evidence Based Treatments.
Please use this url to cite or link to this publication:
author
organization
year
type
conference
publication status
unpublished
subject
keyword
Homogeneity, Dodo Bird Verdict, Randomized Controlled Trial, Evidence Based Treatment, Validity
conference name
8th European Conference of Psychotherapy Research
conference location
Klagenfurt, Austria
conference start
2015-09-24
conference end
2015-09-26
language
English
UGent publication?
yes
classification
C3
id
7080640
handle
http://hdl.handle.net/1854/LU-7080640
date created
2016-02-10 17:00:49
date last changed
2016-12-19 15:37:47
@inproceedings{7080640,
  abstract     = {Aim 
{\textquotedblleft}All have won, and all must have prizes,{\textquotedblright} - so the dodo bird verdicts the statistical indifference between {\textquoteleft}evidence based{\textquoteright} types of psychotherapy. Evidence is understood as the result of randomized controlled (RCT) research. This {\textquoteleft}golden standard{\textquoteright} design requires samples that are homogeneous with regard to symptom(s) - following from the statistical assumptions in central tendency statistics. Symptom specific measures are used to select patients scoring above cut-off, resulting in samples characterized by tight and simple symptom patterns. Whereas the external validity of this procedure is heavily critiqued, I study the internal validity of a priori methodological assumptions in RCT efficacy research. In this paper, I address the clinical and epistemic plausibility of sample selection by a quantified symptom measure for depression, the BDI-II, which was found to be a reliable screening measure for Major Depressive Disorder.
Method 
In a pilot study on our mixed method psychotherapy data from depressed adults, we used the BDI-II to select patients eligible to participate in a strict RCT design. For both the eligible and non-eligible sample, we scrutinized individual symptom patterns.
Results 
Analysis showed a 60-40 ratio of depressed patients to be included or excluded, respectively. 75\% of the {\textquoteleft}eligible{\textquoteright} sample still showed comorbidity, which would have been reason to exclude them from a RCT.
Discussion 
The extraordinary large exclusion percentage indeed inserts doubt on external and ecological validity of quantified eligibility screening in RCT efficacy research. More striking however is that a supposedly {\textquoteleft}eligible sample{\textquoteright} still harms the assumption of homogeneity. In this paper, I propose such invalidity of a priori methodological assumptions as an explanation for the dodo bird effect. I discuss the implications of these preliminary empirical results for the understanding of {\textquoteleft}evidence{\textquoteright} from RCTs and the epistemic value of Evidence Based Treatments.},
  author       = {Truijens, Femke and Desmet, Mattias},
  keyword      = {Homogeneity,Dodo Bird Verdict,Randomized Controlled Trial,Evidence Based Treatment,Validity},
  language     = {eng},
  location     = {Klagenfurt, Austria},
  title        = {All have won what? On the epistemic value of RCT based evidence in Evidence Based Treatment},
  year         = {2015},
}

Chicago
Truijens, Femke, and Mattias Desmet. 2015. “All Have Won What? On the Epistemic Value of RCT Based Evidence in Evidence Based Treatment.” In .
APA
Truijens, F., & Desmet, M. (2015). All have won what? On the epistemic value of RCT based evidence in Evidence Based Treatment. Presented at the 8th European Conference of Psychotherapy Research.
Vancouver
1.
Truijens F, Desmet M. All have won what? On the epistemic value of RCT based evidence in Evidence Based Treatment. 2015.
MLA
Truijens, Femke, and Mattias Desmet. “All Have Won What? On the Epistemic Value of RCT Based Evidence in Evidence Based Treatment.” 2015. Print.