Theory-Based Evaluation1

Abstract

A theory-based evaluation of a program is one in which the selection of program features to evaluate is detennined by an explicit conceptualization of the program in terms of a theory, a theory which attempts to explain how the program produces the desired effects. Fitz-GibbonThe theory might be psychological, such as a theory of child development (e.g., Piaget's) or a theory of learning (e.g., S-R theory), or social psychological (e.g., attitude change theories ; organization theories) or philosophical (e.g., the &dquo;Summerhill&dquo; philosophy). The essential characteristic is that the theory points out a causal relationship between a process A and an outcome B. A-~B; that is, A leads to, or causes B. (A, of course, may consist of many necessary components or stages-the whole process deemed necessary, by the theory, to produce B.) Thus, by a &dquo;theory-based&dquo; evaluation, we do not mean an evaluation based on a theory about evaluation. We mean rather, one based on a theory about how a program operates.Perhaps the inclusion of philosophies as theories needs some justification, especially as we shall subsume &dquo;models&dquo; (see Joyce & Weil, 1972) under the same rubric. We justify the inclusion by defining a philosophy as a set of attitudes. There has been extensive consideration of the nature of attitudes in the literature of social psychology and those models of which we are aware assert that attitudes have cognitive and affective components. The affect-cognition model (Rosenberg, 1960) is presented as an example since other models seem readily reducible to it. Rosenberg postulates that a person's attitude towards X is a function of his expectation (subjective probability) that X leads to Y and his emotional response to Y (Figure 1).In the cognitive component, there is a postulated causal relationship; the perceived probability that X~Y. Attitudes involve theories, therefore, since they postulate causal relationships. For this reason philosophies, like theories, suggest variables to study and imply that certain outcomes are likely to occur from certain processes.

WITHDRAWN: Understanding the complexity of population health interventions: assessing intervention system theory (ISyT)

“…A combination of them? In TDE, the theory explains how the program produces its effects (why and how the intervention works) by de ning a set of explicit or implicit assumptions on the part of stakeholders about what action is required to solve a problem and why the problem will respond to this action (9,18,19). In the line of our previous work on interventional systems, assuming this blurring between context and intervention components (5,20), the theory in TDE should integrate various elements coming from other theories, models, and frameworks because i) it is explanatory, considering what causal pathway is expected to meet the goal, similar to classical theories; ii) it hypothesizes which speci c actions and sequences of implementation contribute to this causal pathway, similar to a process model; and iii) it considers contextual elements and their in uence within a speci c setting.…”

Understanding the complexity of population health interventions: assessing intervention system theory (ISyT)

“…Or a combination of them? In TDE, the theory explains how a programme generates its effects (why and how the intervention works) by defining a set of explicit or implicit assumptions on the part of stakeholders about which action is required to solve a problem and why the problem will respond to this action [ 9 , 18 , 19 ]. Following on from our previous work on interventional systems assuming this blurring between context and intervention components [ 5 , 20 ], the theory in TDE should integrate various elements arising from other theories, models and frameworks because (i) it is explanatory, considering which causal pathway is expected to meet the goal, similar to classical theories; (ii) it hypothesizes which specific actions and sequences of implementation contribute to this causal pathway, similar to a process model; and (iii) it considers contextual elements and their influence within a specific setting.…”

El papel de la pedagogía en clases con computadoras uno a uno: un estudio observacional cuantitativo de las interacciones profesor-alumno

“…The justification for such a program is based on certain beliefs and assumptions that underlie an expected sequence of mechanisms, processes and contingencies through which the laptops are hoped to impact learning outcomes. Together, these make up the program theory (Fitz-Gibbon & Morris, 1996;Rogers & Weiss, 2007;Weiss, 1997). For example, if the program theory holds that learning will be enhanced because students will have much more access to accurate, up-to-date information, then that laptop program should be characterized by continuous Internet access, instruction on the use of browsers and browser-based tools for information search and retrieval, instruction in digital rights and copyrights and related matters, information search and retrieval activities, evidence of students evaluating the quality and veracity of sources, evidence of students incorporating retrieved information into their work, and other activities that link computers to information retrieval, information retrieval to learning activities, and those activities to actual learning.…”

“…Programs can then be assessed according to how successfully they fulfill the expectations of the given theory. This approach, known as theory-based evaluation (TBE) allows the researcher to evaluate the mechanisms -the participants' responses to the program -through which change, in our example actual learning, takes place (Fitz-Gibbon & Morris, 1996;Rogers & Weiss, 2007;Weiss, 1997).…”