Reconceptualizing ‘effectiveness’ in environmental projects: Can we measure values-related achievements?

Abstract

There have been recent calls for a shift to an evidence-based paradigm in environmental management, grounded in systematic monitoring and evaluation, but achieving this will be complex and difficult. Evaluating the educational components of environmental initiatives presents particular challenges, because these programs often have multiple concurrent goals and may value 'human outcomes', such as value change, which are intangible and difficult to quantify. This paper describes a fresh approach based on co-creating an entirely new values-based assessment framework with expert practitioners worldwide. We first discuss the development of a generic framework of 'Proto-Indicators' (reference criteria constituting prototypes for measurable indicators), and then demonstrate its application within a reforestation project in Mexico where indicators and assessment tools were localized to enhance context-relevance. Rigorously derived using unitary validity, with an emphasis on relevance, practicability and logical consistency from user perspectives, this framework represents a step-wise advance in the evaluation of non-formal EE/ESD programs. This article also highlights three important principles with broader implications for evaluation, valuation and assessment processes within environmental management: namely peer-elicitation, localizability, and an explicit focus on ethical values. We discuss these principles in relation to the development of sustainability indicators at local and global levels, especially in relation to post-2015 Sustainable Development Goals.

An Unexpected Means of Embedding Ethics in Organizations: Preliminary Findings from Values-Based Evaluations

“…The project used an emancipatory action research approach, working with organizations and businesses engaged in providing non-formal education for sustainable development (ESD). The latter considered themselves unable to articulate "intangible" aspects of their work which they nonetheless considered key, and the work focused on co-developing an approach that could be highly localized (but also generalized and transferable to some extent) [5,[32][33][34][35] which later became known as WeValue.…”

“…Considering the potential importance of effectively bringing values into organizational practices, the authors identified the need for a systematic follow-up study to explore the impacts of the WeValue intervention within a broader organizational context (rather than focusing specifically on project evaluation in relation to non-formal ESD, which has been discussed in depth by Harder et al [33]). It is the findings of this follow-up study, based on a re-analysis of the original dataset, which we report in this paper.…”

“…The action research approach used in its development, which particularly highlighted the importance of localizable proto-indicators, has been reported [34], as have several examples of its use in different organizations as an evaluation approach [5,32,33]. None of these report on the impacts of the use of the intervention, or insights of any links with business ethics and current related research gaps.…”

Can We Improve Indicator Design for Complex Sustainable Development Goals? A Comparison of a Values-Based and Conventional Approach

“…We thus analyse it for relevance and coverage of Target 4.7, denoting it the ESDinds set, and the approach as 'values-based'. This process produced not rigid indicators, but 'proto-indicators'-reflecting the concept of a 'prototype' in design literature, to refer to concise statements that can serve as templates or triggers for the local development of immediately relevant measurable indicators [33].…”

Enhancing the Understanding of Urban Systems for Sustainability Transition: A Study of Urban Environmental Management in the Natural Resource-Based Industrial City of Jinchang, China

“…By investigating different groups of local residents from multiple levels of the society, the study builds up an improved urban environmental management system, combining top-down and bottom-up approaches for better urban development directed at sustainability. The synthesis of the top-down and bottom-up approaches is conducive to the effectiveness of environmental management as the two approaches complement each other (Böhringer and Rutherford, 2008;Harder et al, 2014;Zhang et al, 2011). …”