Date of Award

12-2020

Degree Name

Doctor of Philosophy

Department

Evaluation

First Advisor

Dr. Daniela Schröter

Second Advisor

Dr. Kathryn E. Newcomer

Third Advisor

Dr. Michael A. Harnar

Keywords

Interdisciplinarity, research evaluation, Carol H. Weiss, academic influence, knowledge utilization, evaluation

Abstract

Research evaluation is the subfield of evaluation that examines the processes and products of scientific and technological research. This dissertation explores the case of interdisciplinary research evaluation by reviewing the publications of one of the most influential and highly cited scholars in evaluation, public administration, and knowledge utilization: Carol Hirschon Weiss. As one of the foundational theorists of evaluation, the evaluation perspective of Weiss's work typically pervades discussions. That approach ignores her influence in other fields, particularly her centrality to the knowledge utilization field. Prior research indicates that the influence of interdisciplinary research is challenging to ascertain because research evaluation tends to rely heavily on quantitative measures, which limit useful comparison among disciplines.

The present study examines the influence of Carol Hirschon Weiss's writings by creating a systematic map of Weiss's work and identifying disciplines in which citations to her scholarship appear. Rather than merely identifying which works are cited by which disciplines, the present study examines qualities presumed related to citation behavior, specifically the type of publication, the domain addressed, and whether the publication primarily focuses on methodologies, theories, research results, or reflecting on practice. These qualities are outside the purview of traditional bibliometrics.

A comprehensive catalog of Weiss's scholarship was necessary before attempting to determine the existence of relationships between these qualities and citations. Two catalogs (one each of published and unpublished work) identify Weiss's scholarly output. Categorizations of Weiss's work show most publications were either journal articles or book chapters, most of which espoused theory about knowledge utilization and, to a lesser degree, evaluation broadly and theory-based evaluation specifically. Through the development of a grounded typology of her publications, this study confirmed the earlier work of Mathison (2005) and Newcomer (2015), who independently determined Weiss's notable contributions were:

  1. The political nature of knowledge generation;
  2. The ways knowledge is used in decision-making;
  3. The nature and method of theory-driven evaluation.

Over 10,000 citations of Weiss's scholarship appear in Scopus as of July 19, 2020. These citations identify the influence of Weiss's scholarship in all twenty-eight subject categories. Somewhat expectedly, citations appear most frequently in the broad subject category of "Social Sciences," which includes education, criminology, sociology, political science, and the social aspects of health and medicine. Weiss's most extensive influence comes from her work identifying the varied uses of knowledge and the roles information plays in decision-making. Taken together, citations to her popular textbook on evaluation (1972, 1998) appear in all Scopus subject categories. Citations to her 1979 article, "The Many Meanings of Research Utilization," appear in every Scopus subject category except "Energy."

The breadth and persistence of Weiss's scholarly influence over the past fifty years distinguish Carol Hirschon Weiss as a notable scholar. Her centrality to the fields of evaluation, knowledge utilization, and policy studies point to her multi- and interdisciplinary relevance. These observations combine to form an overwhelming conclusion that recognizes Weiss as a foundational theorist to understanding the role information plays in society.

Access Setting

Dissertation-Open Access

Download

Share

COinS