The realist evaluation (RE) approach is a promising contribution to the literature on theory-driven evaluation.Thus far, RE has mainly been applied to the evaluation of human service programmes, but the aim of this article is to investigate if it can be extended to large-scale public sector reforms, in casu: the reform of the electricity market in Denmark. Five areas of regulation under the reform were selected and hypotheses on the context, mechanism and outcome (CMO) for each intervention were formulated. The evaluation of three of these interventions is summarized: opening markets to foreign trade, price regulation of monopolies and consumer protection. The article concludes that the CMO framework is applicable not just to interventions at the programme level, but also to large-scale public sector reform characterized by high technical complexity and strong policy coalitions. However, it only serves as a general framework, which has to be substantiated.This is done, first, by generating theories — i.e. programme theories — on each specific intervention. Second, theories are produced at the meso level to identify the institutional patterns that function as conditions for the effects to be obtained.

No abstract

The evidence movement and the idea of systematic reviews, defined as summaries of the results of already existing evaluation and research projects, have gained considerable support in recent years as many international as well as national evidence-producing organizations have been established. This article analyses how the idea is practised in the areas of health, social welfare and education and shows that evidence-producing organizations work differently. Some subscribe to the hierarchy of evidence, others to a typology of evidence. The consequences of these variations are discussed.

The article focuses on the relevance of Karl R. Popper to the evaluation community. Donald T. Campbell and Charles E. Lindblom are used as prisms through which Popper becomes visible to evaluators. The two authors hold different positions with regard to the theory of knowledge and to the use of evaluation. How can both be associated with Popper? The article presents references to Popper from these and other authors within evaluation. The relevance of Popper to evaluation is visible in relation to the issue of evaluative knowledge, the use of evaluation and unintended consequences of interventions.

Scund J Soc Welfure IY9X: 7: llX-I25 Evaluation and public service quality Rieper 0, Mayne J. Evaluation and public service quality. Scand J Soc Welfare 1998: 7: 118-125 0 Blackwell, 1998 Public service quality is a recent trend in the context of new public management. During the 1990s, public service quality initiatives have been launched in numerous countries at various levels of government. These initiatives have involved a variety of approaches: professional standard setting, service excellence based on users' perception of quality, government granted rights and choice for consumers, and the activity of consumer movements. Public service quality is considered on three levels: the micro level related to the output of the service delivery, the meso level related to the outcome of the service, and the macro level related t o public values. Various stakeholders of evaluations of service quality efforts are identified and their expected use of evaluation is discussed. Three roles of education are described: one in the implementation of public service quality, another in evaluation of the success of service quality initiatives, and a third role of evaluating higher lev1 outcomes. It is argued that even if evaluation techniques are necessary tools for improving service quality at the micro level during implementation, it is even more important to evaluate these initiatives against higher level policy goals and social values. 0. Rieper', J. Mayne2 ' Institute of Local Government Studies, Denmark Office of the Auditor General of Canada