TOWARD A SYSTEMATIC APPROACH TO PROGRAM EVALUATION DESIGN

THE PAPER IDENTIFIES A DESIGN FRAMEWORK WHICH LINKS PROGRAM CHARACTERISTICS TO DESIGN ELEMENTS, DEFINES FIVE RELATED DESIGN COMPONENTS CONTAINING THE ESSENTIAL DESIGN ELEMENTS, AND PROPOSES AN EVALUATION TAXONOMY WHICH INCLUDES THE DIFFERENT TYPES OF EVALUATION. THE PAPER IS STRUCTURED AROUND A SERIES OF 12 FIGURES; THE TEXT OF THE PAPER SUPPLEMENTS THE INFORMATION CONTAINED IN THE FIGURES. THE DESIGN FRAMEWORK IS BASED ON A DYNAMIC ROLLBACK APPROACH WHICH CONSISTS OF THREE STEPS: (1) A PROJECTED LOOK AT THE RANGE OF PROGRAM CHARACTERISTICS, (2) A PROSPECTIVE CONSIDERATION OF THE THREATS TO THE VALIDITY OF THE FINAL EVALUATION, AND (3) A MORE IMMEDIATE IDENTIFICATION OF THE EVALUATION DESIGN ELEMENTS. THREATS TO VALIDITY WHICH CAN BE REGARDED AS PLAUSIBLE RIVAL HYPOTHESES OR EXPLANATIONS OF THE OBSERVED IMPACTS OF A PROGRAM ARE DISCUSSED. FIVE INTERRELATED DESIGN COMPONENTS OF TEST HYPOTHESES, SELECTION SCHEME, MEASURES FRAMEWORK, MEASUREMENT METHODS, AND ANALYTIC TECHNIQUES ARE DESCRIBED. DISCUSSION ENCOMPASSES PROBLEMS OF TESTING A HYPOTHESIS, DIFFERENCES BETWEEN NONEXPERIMENTAL AND QUASI-EXPERIMENTAL DESIGNS OR SELECTION SCHEMES, AND THE DEVELOPMENT OF A MEASURES FRAMEWORK. ALSO, TECHNIQUES AND PROBLEMS OF DATA SAMPLING ARE LISTED, AND AN EVALUATION TAXONOMY BASED ON EIGHT MEASURES-RELATED CLASSIFICATIONS IS PROPOSED. REFERENCES ARE PROVIDED. (AOP)