Measuring the impact of something like the EITI is riddled with challenges.
EITI stakeholders measure and understand impact in different ways. For some, it is about improved trust and less conflict, for others it is about growth. For others, it is about enacting stronger laws at home and sanctioning non-reforming regimes. For others again, it is often about incremental opening of democratic space and improved accountability of their governments.
In all cases, the EITI should create a public good. However, how it relates to those impacts is difficult to measure because there are usually a number of forces at play and also because the causality is difficult to establish. Given the differences of country experience, this is not an area given to usual multiple regression analysis. Are Nigerians any less poor because of the EITI? Yes, probably, but not to a measureable extent, of course. And you cannot measure it for at least two obvious reasons: a) you cannot tie impact to the EITI and b) the impact is part of a much wider set of actions.
Below I explore some of the factor that contribute to the complexity of measurement of the impact of the EITI.
Voluntary vs mandatory
We know very little about the interplay between mandatory and voluntary. To start with, there is confusion about what these terms mean. Some criticise the EITI as being ‘merely’ voluntary, and many companies like to emphasize its voluntary nature. For sure, the EITI is voluntary in that governments are sovereign and the EITI is not backed by treaty, but the EITI is mandatory in that all companies have to report in an implementing country, and this reporting is often mandated by law.
The EITI is increasingly complemented by other mandatory actions, such as Dodd Frank 1504 in the United States and similar disclosure requirements in Europe which cover some of the same issues from the perspective of where companies are listed rather than where they operate . However, we know little about whether disclosure rules have come about as a result of the EITI or despite the EITI.
Getting the right kind of information
We tend to suffer sometimes from a naïve belief in figures. While it is important to have quantitative information, it is even more important to have the right kind of information and to have it used. We need more serious analysis, in words and numbers.
The EITI has evolved overtime. It has moved from a simple reconciliation of payments exercise to an international standard that covers the whole extractives value chain from extraction to social impact, seeks to strengthen government systems and requires the disclosure of the beneficial owners of companies in EITI Reports. The time it takes to have academic work compiled and published results in that it is often out of date in analysis the impact of the EITI.
For these reasons we continue to rely on case studies to provide empirical evidence of impact. The following examples do not seek to show the extent of EITI impact, rather the range of impacts that make measurement challenging.
Examples of documented impact
EITI Reports contain information about opportunities for impact and reform. The EITI Reports often make recommendations aimed at addressing weaknesses in government systems and improving sector management. The 2016 EITI Progress Report showcases some examples.
The Government of Nigeria claims that information in EITI Reports has led to the recovery of USD2.4bn of unpaid revenue and has identified a further USD 9bn which it expects to recover. In addition, the last Nigeria EITI Report highlighted recommendations to split up the national oil company, to review oil contracts and fuel subsidies, and scrap the controversial crude oil swaps (opaque barter agreements in the sector). President Buhari is now doing all of these.
Following EITI recommendations, the Government of Kazakhstan made amendments to a decree that now ensures that companies transfer social payments to the central budget. Both local and central governments can now easily track the payments.
EITI Reports in Ghana revealed that extractive sector revenues earmarked for sharing between various national regulatory and oversight bodies, local government authorities, traditional land-owning authorities and communities affected by mining activities did not reach the intended beneficiaries. This was due to a misapplication of these funds, lack of proper accounting for and reporting of the use of these resources, and irregular transfers. Implementing recommendations on timely, regular and full transfers of the funds has resulted in increased accountability at subnational levels, dedicated accounts set up for local governments and guidelines developed by the Minerals Commission for the use of mineral revenues at subnational levels.
Measuring the impact of the EITI was never going to be easy. It means different things to different people. It is often hard to establish causality between the EITI and any impact. We must embrace quantitative assessments but also be mindful of the pitfalls with such methodologies. At the EITI, our role is to make ourselves, our data and information available. We can facilitate contacts and draw attention to good case studies and solid research, whether it demonstrates impact or not.
Head, EITI International Secretariat