Artifact in Pretest-Posttest Designs - How It Can Mistakenly Make Delinquency Programs Look Effective

Several studies (Unified Delinquency Intervention Service Program of the Illinois Department of Corrections, 1978; Empey and Erickson, 1972; Empey and Lubeck, 1971) published results demonstrating spectacular success rates of both institutional and community treatment efforts by using pretest and posttest comparisons. Their data exhibited intense criminal involvement just prior to intervention and a dramatic decrease in such behavior after treatment. A 'suppression effect' was evident since, regardless of the type of intervention, a 60 to 70 percent delinquency reduction rate occurred. A substantial part of this suppression effect is most likely due to a statistical artifact attributable to the process used in selecting juveniles for intervention. This selection-regression artifact arises because judges select individuals for intervention on the basis of prior records with high delinquent activity rates in the recent past. Statistically, this creates a misleadingly high aggregate offense rate for that period. Moreover, it is the judges who actually select the membership of both the experimental and control groups. Because the selection-regression artifact manifests itself before the assignment of delinquents to experimental or control groups, it exists in experimental as well as quasi-experimental studies. It is not possible for the evaluator to account for the selection-regression artifact in his data evaluation by estimating its mathematical effect, since actual decisions to intervene are based on complex multiple critera. It is advised that reduction of delinquent behavior from preintervention to postintervention not be considered as a measure of effectiveness. Data on past records can, however, be used for matching control and experimental groups to control their effect on outcome variables. They can also be used to characterize judges decisions and contribute to the understanding of decisionmaking in juvenile courts. Tabular data, notes, and references are included.