Trialling the Theory of Change approach to impact evaluation in conservation using the Saiga Conservation Alliance’s work in Uzbekistan as a case study

There is a need for high quality impact evaluation in conservation biology. In response to this need there is a growing body of work on how best to evaluate conservation interventions. However, often these methods are too time-consuming or resource-heavy for small NGOs to carry out. The Theory of Change approach has the potential to be a useful and feasible approach for small NGOs.

Kirsty Simkin, kirsten.simkin@some.ox.ac.uk

University of Oxford

There is a need for high quality impact evaluation in conservation biology. In response to this need there is a growing body of work on how best to evaluate conservation interventions. However, often these methods are too time-consuming or resource-heavy for small NGOs to carry out. The Theory of Change approach has the potential to be a useful and feasible approach for small NGOs. As part of my undergraduate degree I used the Saiga Conservation Alliance's work in Uzbekistan to trial the Theory of Change approach to impact evaluation.

The simplest way to define a Theory of Change (ToC) is as "a theory of how and why an initiative works". More fully articulated, this can be understood as a way to describe the set of assumptions that explain both the mini-steps that lead to a long-term goal and the connections between these activities and the outcomes of an intervention or programme. The ToC process is a theory-based approach to planning, implementing or evaluating change at an individual, organisational or community level. If seen as an on-going process of discussion-based analysis and learning, the ToC approach has the potential to provide powerful insights to support programme design, strategy, implementation, evaluation and impact assessment. The approach is communicated through diagrams and narratives which are updated at regular intervals. ToCs may be developed and used at various points in the lifecycle of an initiative or programme, from planning an idea through to implementation, delivery and review.

A ToC typically functions according to a sequential logic that runs from activities through to impacts (see Figure 1). This pathway is underpinned by a series of assumptions which need to be articulated.

Figure 1. Diagram of the sequential components of a ToC:

Key ToC definitions:

Activities	Specific actions undertaken to mitigate a threat or support an opportunity.
Outputs	The immediate and measurable products of the activities of an intervention.
Outcomes	the intermediate result that is brought about by producing preceding project outputs.
Impacts	The wider and longer term effects of an intervention and the contribution the intervention makes to broader goals.
Assumptions	Statements that explain both the connection between preconditions for long-term change that occur in the early and intermediate stages of the change process, and the expectations about how and why proposed interventions will bring them about.

I developed a series of ToCs based on the SCA's work in Uzbekistan, by collecting perspectives and feedback from experts in interviews and reviewing the SCA's strategic plans and other documents and reports. Figure 2 shows one of the pathways of a ToC, which focuses on reducing the pressure on saigas from poaching through activities involving law enforcement. It shows the hypothesised pathway from inputs to our desired impact.

Fig 2 Figure 2. One pathway of the Theory of Change developed for decreasing pressure on saigas from poaching via support for law enforcement

I also identified assumptions for each step of the framework. Assumptions explain the underlying logic behind our expectations of the connections between different components of the pathway of change. Ideally assumptions should be supported by scientific research, best practice or expert knowledge. It is also possible to test assumptions with field research, depending on funding and time constraints.

Assumptions for pathway 1:

Code	Assumption
A1.1	People care about the opinions of their communities
A1.2	Interagency cooperation is helpful to law enforcement
A1.3	Better trained and equipped guards do not use their advanced equipment for poaching or other purposes
A1.4	Training increases knowledge of participants
A1.5	Being celebrated in communities increases pride in job and subsequently a desire to work harder
A1.6	Motivation of law enforcement increasing is coupled with sufficient capacity of law enforcement
A1.7	Poachers have not similarly strengthened their capacity and equipment, negating any gain through an ongoing 'arms race'.
A1.8	Relative value of poaching is not so high as to make increased risk of poaching worth it
A1.9	Disincentive for poaching are larger than incentives

Using the ToC approach to look at the SCA's work in Uzbekistan showed the potential this approach has as a framework for communication, planning and learning, and in particular for setting priorities for future data collection and evaluation. The ToC approach is intended to be an evolving tool that is tested and improved over time and there is potential for the approach to be utilised by the SCA in the future.

To read my full report please go to http://www.iccs.org.uk/sites/www.iccs.org.uk/files/inline-files/K%20Simkin%20UG%20project_0.pdf