Many studies have been conducted on the impact of EITI and reforms in extractives governance over the past decade, with mixed results. There is widespread interest across the EITI community in establishing better ways of measuring impact using a range of methods to develop a more convincing and balanced narrative. This will be central to better communicate the tangible impact the EITI is having in 52 countries.
The session reviewed lessons from past efforts to study the impact of EITI on natural resource governance and development outcomes, highlighted some encouraging innovations and pointed to the need for more clearly articulated and politically savvy approaches to measuring impact going forward.
Moderator: Mr Michael Jarvis, Executive Director, Transparency & Accountability Initiative
Mr Mario Picon, Program Director, Results for Development
Ms Leila Kazemi, Project Lead, Columbia Center for Sustainable Development
Mr Oleksiy Orlovsky, International Renaissance Foundation, Ukraine
Mr Ian Mwiinga, Zambia EITI
How has EITI had impact? Mario Picon drew upon the existing literature to illustrate that the answer strongly depends on how impact is defined. EITI was created to generate transparency of information; stakeholders have since layered different ultimate objectives on to the initiative, but the systems were established to demonstrate compliance with a standard, not to measure broader societal impacts. At the global level, existing studies show a more positive correlation with curbing corruption, whereas the evidence of EITI effects on accountability and good governance is less clear and even contradictory. The panel strongly agreed that in order to evaluate the EITI’s impact it is crucial to first define the objectives to be measured against.
Leila Kazemi highlighted the need to better understand the mechanisms used to achieve defined objectives. Implementation gaps still exist in many countries. Why the mixed track record? In part it reflects an emphasis on technical approaches without sufficient consideration of the underlying political dynamics influencing EITI data generation and use. Clearer stories of impact will only emerge if we recognize and anticipate political economy factors, and account for country context more effectively – and not assume these are static.
An important starting point is to create tools that facilitate impact measurement. Oleksiy Orlovsky introduced the experience of Ukraine, where they have created a monitoring and evaluation assessment framework for EITI for application at subnational level. He highlighted how such tools need to be developed in consultation with communities to track progress aligned to their priorities. Such investment holds promise in terms of aligning EITI processes to concerns of local populations. Resulting discussion with session participants revealed the variety of different ways in which different countries (and subregions even) interpret “impact” associated to EITI. One challenge going forward is enabling different multistakeholder groups (MSGs) to track impacts that resonate for their context, while still having a sense of the cumulative effects of EITI globally.
Building in flexibility for measurement may become easier as EITI mechanics shift. Ian Mwiinga focused on the need to go from reports to reporting, noting the potential of the EITI push to “mainstreaming” of data generation to enable a smarter approach to measuring progress (not least through freeing up resources currently used for report generation and validation). Drawing on the experience in Zambia, Ian also flagged the importance not only of measuring impact but of communicating it effectively. Effective narratives – and deploying EITI information in support of those narratives – are critical to influencing change. There are some positive outliers on the communication front within the EITI community, but more to be done.
In sum, the panel and participants were excited at the increasingly sophisticated conversation on results and impact within EITI. The global secretariat and national MSGs increasingly recognize the importance of articulating theories of change and aligning impact measurement to better assess validity of those theories and adapt in line with what the evidence tells us. There is a steady shift from prioritizing the data itself to what is done with the data for what motivating purpose. Ten years from now we anticipate a much clearer set of compelling impact stories.
Many studies have been conducted on the impact of EITI and reforms in extractives governance over the past decade. These include quantitative studies of impacts across countries and themes to single country studies that are more qualitative in nature. The focus ranges from efforts to quantify development impacts through to governance outcomes. Most studies point to mixed or uncertain results, claiming that there is little evidence to demonstrate improvements in governance or development outcomes resulting from EITI activities.
These studies have been criticised for failing to capture the more immediate results or intermediate impacts resulting from improvements in systematic disclosure, beneficial ownership or contract transparency. There is also methodological critique of the excessive emphasis on quantitative techniques and statistical methods which seek to investigate outcomes at the end of a results chain over which EITI would have little direct influence. Another line of questioning highlights the failure to address social and environmental impacts arising from EITI, in terms of the differential benefits accruing to men and women and social groups at the local level, improved revenue collection at the local level, or on positive or negative environmental impacts resulting from improved extractives transparency.
Finally, there is also an appreciation that EITI implementing countries could benefit from clearer narratives to communicate stories of impact to decision makers and the donor community. While the periodic Validation process for EITI implementing countries presents an assessment of progress against the Standard there is an opportunity to capture insights from this process that have a direct bearing on impact at the country level.
There is widespread interest across the EITI community in establishing better ways of measuring impact using a range of methods and approaches to develop a more convincing and balanced narrative on impact.