Date of Award

8-2007

Degree Name

Doctor of Philosophy

Department

Interdisciplinary Studies

First Advisor

Dr. Michael Scriven

Second Advisor

Dr. John Hattie

Third Advisor

Dr. David Hartmann

Abstract

Due to its very nature, the evaluation of research permeates nearly every aspect of the work of researchers. They evaluate the work of others or have their own work evaluated. They evaluate hypotheses that come to mind, the previous literature, the quality of data, the explanatory power of theories, or the design of experiments or instruments. And this is not always casual evaluation. It is highly skilled evaluation, and becoming a first-rate or world-class researcher is a process of improving the quality of these evaluations. However, deciding when someone is or has become a first-rate or world-class researcher is an evaluation at a somewhat different level. It is a complex synthesis of judgments about how well the researcher does each of the constitutive types of evaluation, usually as evidenced in the work they are producing.

In the last few decades the evaluation of research has become a high-stakes enterprise. With increasing political governance and federal budgets often in the billions, the livelihood of individual researchers, research groups, departments, programs, and entire institutions often swing in the balance. Simultaneously, it has been recognized that many of the longstanding principles and practices often lead to poor decisions about the actual or prospective merits of researchers and their research.

The research in this dissertation describes, classifies, and comparatively evaluates the national models used to evaluate research and allocate research finding in sixteen countries. These models vary widely in terms of how research is evaluated and financed. However, nearly all share the common characteristic of relaxing funding to past performance. Each of these sixteen national models was rated on more than twenty-five quality indicators by independent, blinded panels of researchers and evaluators in two countries. The national models were than ranked in terms of their validity, credibility, utility, cost-effectiveness, and ethically. The results of the rankings show that the clear leaders are nations using large-scale research assessment exercises of various hues. Implications for research evaluation practice and policy are considered and discussed.

Access Setting

Dissertation-Open Access

Download

Share

COinS