Date of Award

4-2014

Degree Name

Doctor of Philosophy

Department

Evaluation

First Advisor

Dr. Chris L. S. Coryn

Second Advisor

Dr. Daniela Schroeter

Third Advisor

Dr. Patricia Reeves

Fourth Advisor

Dr. Tarek Azzam

Keywords

Evaluation, ethics, crowd source, methodology, visual data, qualitative analysis

Abstract

Most evaluations in the nonprofit and international development sectors are conducted in contexts of complexity; the specific intervention being evaluated is but one of many interrelated factors influencing the desired outcome. Video data, especially when directly generated by program participants, can provide both exceptionally rich qualitative data as well as contextually-relevant feedback within complex systems. Despite these unique strengths and opportunities, video data is underutilized in the field of evaluation. This dissertation addresses specific barriers associated with video data through three inter-related papers: Papers one and two (Chapters II and III) present the findings from two interrelated studies of an analysis methodology utilizing crowd-sourcing, and paper three (Chapter IV) presents a practical ethical framework for the use of visual data.

The video data analyzed in papers one and two was user-generated video reviews of various products, and both studies explored the time, cost and quality implications of crowd-sourcing video analysis. The broad conclusions presented in Paper one indicate that while there are specific limitations, the crowdsourcing methodology offers considerable potential advantages for evaluators - not just saving time and money, but also enhancing the richness of analysis through the analytic perspective of a broad group of participants. Similarly, the findings of Paper two indicate that categorical coding of video data can be completed in a matter of hours at a reasonable cost, and that considerable accuracy is possible – though within specific parameters.

Paper three offers the conceptual framework of ‘relational integrity’ identified as a theme in the literature of ethical visual methodologies. Key themes from a comparative analysis of the image use procedures in the fields of entertainment, journalism, advertising, and social media are presented as contextual reference for guiding ethical decisions and informing relationships surrounding visual methodologies. Five categories of relationship are suggested based on this analysis to serve as a practical reference for evaluators.

While additional study is needed to extend and refine the findings from each of these studies, the findings combine to offer both new direction and practical guidance for the evaluation community working in complex contexts.

Access Setting

Dissertation-Open Access

Download

Share

COinS