Introduction
On Tuesday, August 02, 2022, the webinar "Evaluation Guide on Transformational Change: Evaluating the Transformational Impacts of Policies and Actions" of the Climate Action Transparency Initiative (ICAT). This is the seventh event in a series of nine virtual sessions, co-organized between ICAT and the LEDS LAC platform, which aim to present three of the evaluation guides and methodologies that ICAT has developed and that have recently been translated into Spanish.
The webinar was opened with welcoming remarks from Stefania D'Annibali, Climate Change Specialist at ICAT and featured presentations by Mirko Dal MasoImpact Evaluation Specialist and former UNEP CCC partner, who showed what the Transformational Change Evaluation Guide is and what it is useful for, while Ana Lucia MoyaThe UN Environment Program's Project Manager (CBIT-Costa Rica) gave a presentation on Costa Rica's experience using the guide. Also in attendance were 58 participants from 20 countries.
A recording of the event can be found at here while the Spanish translation of the Transformational Change Assessment Guide and its infographic can be downloaded here.
Key messages
About ICAT and the translation of the impact assessment guidelines (Stefania D'Annibali)
ICAT provides support to developing countries to implement their strengthened climate transparency frameworks as well as their domestic transparency systems. Within the framework of this support, a set of guidelines has been developed to assess climate policies in terms of greenhouse gases (GHG), Sustainable Development or Transformational Change. It is a robust methodological framework aimed at preparing countries to report to the Paris Agreement.
Due to demand from Latin America and the Caribbean, 3 of their cross-cutting impact assessment guides have been translated into Spanish. These documents are important for the development of robust NDCs and robust MRV systems that can raise climate ambition. In this context, to date, 6 webinars have been held with more than 300 attendees from 20 countries.
Transformational Change can be defined as a sustained and fundamental system change that disrupts established carbon-intensive practices and contributes to a zero-carbon society in line with the Paris Agreement and the United Nations Sustainable Development Goals (SDGs). These types of transformations are essential in the context of the current climate emergency, so from ICAT it is very relevant to increase access to guidelines on the subject. Its objective is to provide decision makers with the necessary information to implement evidence-based policies and bet on policies with transformative potential.
What is the Transformational Change assessment guide? What are its benefits? (Mirko Dal Maso)
Transparency is related to analyzing the impacts of various policies or actions since this way a better understanding among the different actors is possible in order to make more objective decisions. For this reason, the ICAT impact assessment guidelines were developed.
Climate change is fundamental to society because nature is the basis for a sustainable society and economy. Thus, climate change and biodiversity represent planetary boundaries that if exceeded could destabilize the earth system. To avoid this, rapid change is needed to reduce GHG emissions and achieve carbon neutrality while achieving the SDGs. Actions must not only be low-carbon but also equitable, resilient, welfare and fair; in line with the global goals of the Paris Agreement and the SDGs. There are synergies between climate actions and Sustainable Development (SD); however, at the same time there are trade-offs. Thus, we should aim to enhance synergies and minimize trade-offs.
The guide helps to develop and implement effective actions, to integrate stakeholders in an objective manner, and to show results at national, international and financial levels. In addition, it allows us to identify and describe what vision of change we want to achieve in the short, medium and long term as well as what stage of transformation we are in and what are the existing barriers to achieving our objectives. Then, it is possible to analyze the impacts that can be achieved or have been achieved (ex-ante and ex-post) on results (GHG and SDG impacts) and processes. To measure the results according to their scale and the time factor, it is useful to use the Sustainable Development Guide. In the case of processes, this includes changes in agents, incentives, behavioral norms and technologies that will enable transformational change.
To carry out the evaluation, it is necessary to choose which characteristics of Transformational Change will be measured (both for processes and results), which correspond to those that are relevant in the context in which the initiative is developed. After assessing the characteristics, the initial situation or baseline is described according to the characteristics. Indicators are then chosen and measured to subsequently evaluate the expected or achieved change. The guide performs a qualitative evaluation and the results are obtained by results and processes that can be visualized in an aggregate way in a heat map in which the scale of Transformational Change in which an action is located is visualized. It is important to evaluate the aggregated or disaggregated results according to the user's needs.
Costa Rica has been working for more than 7 years on the National Climate Change Metrics System (SINAMECC), which integrates MRV schemes and unifies information for data-driven decision making. It is a digital platform that facilitates the capture, processing and reporting of information to share it with the general public. Among the information available is a record of the impacts being achieved and climate finance flows.
SINAMECC supports Costa Rica's transition to the Enhanced Transparency Framework and includes adaptation, mitigation, climate finance and SD in a cross-cutting manner. In the country, there are scattered actions of a variety of actors, so the value of SINAMECC is to register them and follow up on climate mitigation and adaptation actions in a unified manner. The registry is made up of three chapters: the first is related to GHG mitigation based on the WRI guidelines (adapted to the national context). In the second chapter, the registry of mitigation actions was strengthened and the ICAT Sustainable Development guide was added. In the third chapter, the Transformational Change guide was added, since the plans analyzed not only seek GHG reduction but also a transformation in the development model. In the end, the three guides were integrated into one to objectively record not only their mitigation impact but also their SD and transformational potential sustained over time.
The guide was applied in three pilots: REDD+, the Montes Oca Curridabat bike path and NAMA waste. It proved useful to apply the guide at different scales, sectors and levels of progress. Among the lessons learned is that certain concepts of the guide are sometimes only known within the climate public; however, the actions are applied with people from different sectors. For this reason, the capacities of stakeholders were strengthened through web modules on SINAMECC and the two ICAT guides in the Costa Rican context.
Currently, with UNDP support, the guide is being adapted to be used not only for mitigation measures, but also for adaptation. It is planned to measure actions in relation to how they can reduce vulnerability and increase resilience in the long term.
Q&A
- How long does it take (on average) to complete an evaluation using the guide? First, it is important to mention that the application time includes understanding the guidance. In this sense, you can have a very good level of analysis by integrating stakeholders in 3 months. Or, you can have a screening in one month.
- Is it possible to apply the guidance when the policy / initiative implemented has neither been designed nor implemented from a transformational approach? It can be applied and can function as a screening. However, the score is likely to be low.
- How do you think about the involvement of civil society in these instruments in a sustained manner and with technical training criteria for their application? This is a very important point. In Costa Rica, when the guidelines were adapted, a part of the stakeholder involvement guide was included so that the analysis would include a variety of stakeholders.
- What variables or criteria are of greatest weight or priority for implementation and what type of stakeholders do you consider key in the use of the guidelines? The quality of the evaluation will depend on the impact categories chosen. These should be the most important according to the context and are chosen in conjunction with the key stakeholders (those who have an interest or who will be positively or negatively impacted by the action).
- Is it possible to apply it to actions implemented in local contexts, where different actors that may affect or be affected by the actions, such as indigenous peoples, converge? It seems that it can only be applied in national contexts. From Costa Rica, it seemed the same and that is why we conducted pilots at all levels of development and sectoral, national or local scales. The result was that for all actions the guide worked very well since it is very flexible and allows us to see which criteria work for each action.
- Different reports reveal pessimism regarding the fulfillment of the Paris Agreement and the 2030 Agenda: how can a community accelerate the building of resilience at the local level in terms of mitigation and adaptation to climate change? An impact analysis can help us reach this goal. Making sure that the steps we are taking now are the right steps and fair to stakeholders. However, while it is important to analyze actions, the most important thing is to execute these actions.
