PESTOL - Framework for «Project Evaluation on Strategic, Tactical and Operational Levels»

Abstract

The paper focuses on the development of the ex-post conceptual holistic framework for Project Evaluation on Strategic, Tactical and Operational Levels, the PESTOL model, by reviewing different definitions of project success and/or failure and combining the findings with the logic framework. The model reflects the project life cycle by considering all project phases, such as identification and conception. To demonstrate the relevance of the developed model, the authors applied it to a project case, the Algerian East–West Highway megaproject. The project has attracted media attention and a number of media discussions of the project have been limited to the completion of the project in a short-term perspective. In this regard, the discussions have been notably associated with delays and expenditures coming in over budget, referring to project efficiency. One reason for the media focus on these aspects alone is that they can easily be measured. The relevance of the project and its effects - whether it attains its goals and objectives measured in terms of effectiveness, including impact and sustainability - can only be verified at a later stage, after the project has delivered its results. These are much broader aspects and are therefore difficult to measure.

Developing a Multidimensional Conception of Project Evaluation to Improve Projects

“…The multidimensional view of project evaluation supplements earlier project evaluation frameworks in several ways. For instance, it suggests adding multiproject comparisons to logical framework and log frame approaches (Baccarini, 1999; Couillard et al, 2009; Crawford & Bryce, 2003), considerations of time to some frameworks (Crawford & Bryce, 2003; Zidane et al, 2016), and learning and outcome criteria to frameworks focusing exclusively on project efficiency (Xu & Yeh, 2014). Although some studies suggest extensive lists of general evaluation criteria (Ngacho & Das, 2014; Zidane et al, 2016; Zidane & Olsson, 2017) and shared perceptions of project success across different stakeholders (Davis, 2014), our findings emphasized the use of different criteria across different projects and firms, and the need to acknowledge multiple evaluators with different backgrounds and priorities.…”

“…For instance, it suggests adding multiproject comparisons to logical framework and log frame approaches (Baccarini, 1999; Couillard et al, 2009; Crawford & Bryce, 2003), considerations of time to some frameworks (Crawford & Bryce, 2003; Zidane et al, 2016), and learning and outcome criteria to frameworks focusing exclusively on project efficiency (Xu & Yeh, 2014). Although some studies suggest extensive lists of general evaluation criteria (Ngacho & Das, 2014; Zidane et al, 2016; Zidane & Olsson, 2017) and shared perceptions of project success across different stakeholders (Davis, 2014), our findings emphasized the use of different criteria across different projects and firms, and the need to acknowledge multiple evaluators with different backgrounds and priorities. While project evaluation research displays much heterogeneity regarding informative criteria (Haass & Guzman, 2020), in reality, project managers and end users agree more about success criteria in successful projects compared to unsuccessful projects (Wateridge, 1998, in Jugdev & Müller, 2005).…”

“…When choosing among different alternatives, the circumstances of the project evaluation are decisive. Project evaluation conditions involve mandate and power (Loo, 1985), as well as political stances, ideologies, assumptions, and norms (Zidane et al, 2016), not to mention institutional forces, organizational commitments, sectional interests, professional affiliations, and individual agendas (Haass & Guzman, 2020). Such rules can be conscious and official, or unofficial and subconscious, but should be explicated (Linzalone & Schiuma, 2015) as much as possible, as they can be quite powerful in guiding the evaluation.…”

“…Effectiveness criteria include the value generated by the project, the relevance and usefulness of the project's results, meeting of project goals, strategies, and organizational objectives, and (direct) organizational benefits, (indirect) community benefits, side benefits, and future potential. Effectiveness can be measured in stakeholder satisfaction, sales, income, profit, and market share as well as sustainability, innovation and new ideas, skills, technologies, capabilities, and core competences (Atkinson, 1999;Baccarini, 1999;Barclay & Osei-Bryson, 2009;Haass & Guzman, 2020;Laursen & Svejvig, 2016;Martinsuo, 2019;Martinsuo & Killen, 2014;Nelson, 2005;Pinto & Slevin, 1988;Samset & Volden, 2016;Serrador & Turner, 2015;Shenhar et al, 2001;Svejvig et al, 2019;Williams & Samset, 2010;Zidane et al, 2016). Project evaluation literature lacks consensus regarding which criteria to use (Haass & Guzman, 2020), and central concepts are considered ambiguous and overlapping (Zidane & Olsson, 2017).…”

“…Moreover, project evaluators need to recognize the complexity of project evaluation and take into account the evolving, comparative, subjective, and relative nature of evaluations (McLeod et al, 2012). The current study is an attempt to bridge the gaps in extant evaluation models (Zidane et al, 2016) and address recent calls for holistic conceptual approaches for project evaluation.…”