Date of Award

4-1980

Degree Name

Doctor of Education

Department

Educational Leadership

First Advisor

Dr. Daniel Stufflebeam

Second Advisor

Dr. Jim Sanders

Third Advisor

Dr. John Sandberg

Fourth Advisor

Dr. Mary Ann Bunda

Abstract

Educational evaluation theorists propose two methodologies that could be used to assess a project or program's achievements. One method is well established and is called goal-based evaluation. The other has been proposed recently and is called goal-free evaluation. Little information exists about goal-free evaluation in field settings. The problem to be addressed was: What would be the results of a field study that compared the relative utility of operationalized versions of goal-free and goal-based evaluation techniques? The perspective selected to investigate this problem was evaluator/evaluee interactions. This study had two objectives: (1) to develop materials and procedures for using the two techniques in an evaluation study, and (2) to investigate the relative utility of the two techniques through a field exploration of the evaluator/evaluee relationship.

The techniques were operationalized through handbooks that incorporated checklists. Subjects were randomly selected from recommendations by nationally recognized evaluators and were randomly assigned to training in one of the two techniques. Projects to be reviewed were randomly assigned to evaluators.

Three instruments were developed: two of a Likert type and one as a semantic differential. The two Likert instruments were used to assess the following elements of the on-site evaluation process: (1) evaluator/project director rapport; (2) evaluators' time utilization; (3) evaluator/project director expectations of each other; (4) evaluator/project director overall satisfaction; (5) evaluator confidence with the methodology. The third instrument went through a developmental process to establish reliability, and was used by the evaluators who were trained in one of the two techniques. Evaluator ratings of the on-site process were analyzed with a repeated measures ANOVA. Project director ratings of the on-site process and of the utility of the evaluation reports were analyzed through use of a completely randomized hierarchical design.

Findings supported some of the proposed differences between the goal-free and goal-based techniques: (1) the two groups reported different patterns of activities while on-site, (2) the goal-free evaluators rated themselves lower than the goal-based evaluators in elements of the on-site process, (3) evaluee ratings of the on-site process did not differ significantly, and (4) evaluee ratings of the reports produced from the two techniques did not differ significantly.

Access Setting

Dissertation-Open Access

Download

Share

COinS