Date of Award

8-2019

Degree Name

Doctor of Philosophy

Department

Evaluation

First Advisor

Dr. Chris L. S. Coryn

Second Advisor

Dr. Jennifer Greene

Third Advisor

Dr. Bianca Montrosse-Moorhead

Keywords

Evaluation, evaluative practice, program evaluation, evaluation theory, theory in practice

Abstract

The field of evaluation generally agrees that evaluation theory is important. Evaluation theorists and scholars have written prolifically about the role of evaluation theory in practice. However, the empirical literature associated with how and why evaluation theory is important is still emerging. In particular, how evaluation theory fosters high quality evaluation practice through transparent logical reasoning, attention to values and valuing, and contextual responsiveness, and how these inform evaluative thinking are understudied phenomena in the field.

This study intends to provide insights into the relationship between evaluation theory and evaluation practice to answer two major research questions. The first research question asked to what extent American Evaluation Association (AEA) and Canadian Evaluation Society (CES) evaluators are trained in evaluation theory. The second asked what role evaluation theory plays in evaluative thinking. A sequential, two-phase, mixed-methods design was used to investigate these questions, including a cross-sectional survey of AEA and CES members and in-depth one-on-one interviews. The online survey asked a random sample of AEA and CES evaluators about their training in evaluation theory, including the modalities of training and their familiarity with specific approaches. The interviews were designed to unearth the role of evaluation theory in evaluators’ thinking and decision-making.

Findings from the survey reveal that 80% of AEA and CES evaluators had some type of training in evaluation theory. This training generally took the form of short presentations or webinars, graduate courses, and 1–4 day professional workshops. Overall, evaluators were most familiar with participatory evaluation, developmental evaluation, and utilization-focused evaluation, and least familiar with deliberative democratic evaluation, consumer-oriented evaluation, and constructivist or fourth-generation evaluation. Emergent themes from interviews showed that evaluators were responsive to a variety of contextual factors in their practice regardless of their level of training or familiarity with evaluation. Evaluators with all levels of training in and familiarity with evaluation theory endorsed the importance of use as an indicator of successful evaluation, and privileged stakeholder engagement as a factor in facilitating and increasing use. However, evaluators with more training in and familiarity with evaluation theory were more intentional in considering their options and more explicit in justifying the reasoning underlying their actions. Their decisions were generally backed by personal or professional values. Evaluators with less training in and familiarity with evaluation theory were more influenced by preferences or requirements of funders, their disciplinary area, or their own organization. Implications and avenues for future research of this study include identifying the best ways to teach evaluation theory, particularly in light of growing conversations around the professionalization of evaluation and the inclusion of evaluation theory in both the AEA and CES evaluator competencies.

Access Setting

Dissertation-Open Access

Download

Share

COinS