Date of Award

12-2016

Degree Name

Doctor of Philosophy

Department

Evaluation

First Advisor

Dr. Chris L. S. Coryn

Second Advisor

Dr. Daniela C. Schröter

Third Advisor

Dr. Tarek Azzam

Keywords

Evaluation, logic of evaluation, valuing, content analysis

Abstract

This three-paper dissertation investigates the logic of evaluation proposed by Michael Scriven using two different methods of inquiry: content analysis and a survey method. Scriven’s logic of evaluation is one of the central concepts in evaluation, but has not been empirically investigated. Therefore, the purpose of this three-paper dissertation is to accumulate empirical knowledge on Scriven’s logic of evaluation to investigate the extent to which evaluators follow his logic and their perceptions of and familiarity with his logic. Furthermore, this dissertation aims to explore valuing practices in terms of Scriven’s logic of evaluation. Scriven’s logic of evaluation is important because it provides a fundamental reasoning process of conducting evaluation. Whether consciously or not, if one is to conduct evaluation, Scriven’s logic needs to be followed. Although there is no unanimous agreement on the extent to which Scriven’s logic should be followed, Scriven’s logic of evaluation is considered one of the most essential concepts in the field of evaluation.

This dissertation aims to empirically investigate different aspects of Scriven’s logic of evaluation using two methods of inquiry. In the first study, content analysis of evaluation reports was conducted to examine whether the utilization of Scriven’s logic of evaluation is clearly identifiable or not. Findings suggest that standards were not clearly established and synthesis methodology was not found in those evaluation reports. However, it remained unclear whether the evaluators in the sample of the evaluation reports did not follow Scriven’s logic of evaluation. The second study aimed to examine whether evaluators apply Scriven’s logic of evaluation into their evaluation practice as well as evaluators’ familiarity with and perceptions of Scriven’s logic of evaluation. Findings from the second study suggest that many of the evaluators were not familiar with his logic but evaluators typically, although not always, followed Scriven’s logic of evaluation. In addition, it was found that a large number of the respondents considered Scriven’s logic of evaluation useful and important in conducting evaluation regardless of their familiarity. The third study sought to explore details of performance standards and evaluative conclusions. The findings from the third study suggest that many of the evaluators were engaged in making evaluative conclusions, but did not always use performance standards to reach evaluative conclusions. Most of them considered the task of making evaluative conclusions difficult yet important in their evaluation practice.

Access Setting

Dissertation-Open Access

Download

Share

COinS