Lessons from the 2016 Validations

Digging deeper into the EITI Validation data

When the EITI Standard was adopted in 2016, one of the most important changes was a new Validation system. Validation is the EITI’s independent quality assurance mechanism (see the bottom of this blog for more details on the validation process). Results from the first 13 Validations are now available. In this blog post I’m going to dig a little deeper into the Validation data. We’ve prepared this spreadsheet, which collates the results from the 13 scorecards.

Opportunities to make the EITI more relevant

The right place to start is with impact. While Validation assesses whether a country is meeting the EITI Requirements, it also explores the critical question of whether the EITI is having an impact. We have written about this challenge before, and our  progress report presents some compelling case studies. The Validations completed to date have highlighted an interesting tension: there are countries that are making good progress in meeting the EITI requirements, but where the impact of the EITI is in question. There are also cases where the EITI is technically deficient in many respects, but where the EITI is having a profound impact. Lesson: Too much focus on compliance often misses the bigger picture!

Looking at the data, we can unpack the question of impact further. It’s telling that 7 of 13 countries (54%) were scored at less than satisfactory progress on requirement 1.5 regarding the EITI work plan. There is a requirement that MSGs “set EITI implementation objectives that are linked to the EITI Principles and reflect national priorities for the extractive industries”. In several cases, Validation found that these objectives were absent or vague. Put simply: the MSGs did not have a clear vision for what they were hoping to achieve with EITI implementation. Lesson: MSGs should invest more time in discussing opportunities for the EITI to add value and more clearly setting out their objectives and expected outcomes.

That said, stakeholders will often have different (and unstated) expectations for the EITI process. And impact isn’t always planned for. For example, in several cases, the EITI has highlighted that governance mechanisms are not working as they should. For example, an implementing country may have clear laws and regulations regarding audit and assurance procedures both for companies and government agencies. But EITI reporting has demonstrated that these rules are not being followed, or that implementation and public disclosures are absent or behind schedule. Accordingly, EITI reporting often leads to important recommendations for improving extractive industry governance that were not anticipated at the outset. That said, in 5 out of 13 (38%) cases there was insufficient follow up on recommendations from EITI reporting (requirement 7.3). And in 11 out of 13 cases (85%), there was insufficient work by MSGs to discuss and document the outcomes and impact of implementation (requirement 7.4). To some extent, Validation has been an opportunity to address this gap. Lesson: Most MSGs need to take more time to consider the findings from EITI Reports, and ensure that these recommendations are followed up in the next round of reporting.

Opportunities to make the EITI more effective

Why is this cycle of work planning, implementation and review breaking down? Validation has shown that, in some parts of the world, the EITI needs to improve its own internal governance. 10 of 13 countries (77%) were scored at less than satisfactory progress on requirement 1.4 regarding MSG governance. There are myriad aspects to this requirement – from nomination procedures to transparency regarding per diem payments. But in several cases, it was clear that the MSG was not functioning effectively and that the relationship between national secretariats and MSGs needed to be clarified. The lesson: MSG governance matters! But there’s a flip side to this challenge. Validation has shown that some MSGs are almost completely preoccupied with internal governance issues, and that these tensions have crowded out meaningful debate about improving transparency and accountability. Validation has brought these tensions to the surface.

Improving the quality of the EITI data

Turning to the quality of EITI reporting and EITI data – four key results jump out: 

First, only 5 of 13 (38%) countries achieved a score of “satisfactory progress” or above on requirement 4.1 on comprehensive disclosure of taxes and revenues and only 2 of 13 (15%) achieved a score of “satisfactory progress” or above on requirement 4.9 data quality and assurance. Ostensibly, this looks disastrous! But a closer look at individual cases is important. In some cases, the overall approach is reasonably solid, but key aspects of the Standard Terms of Reference for Independent Administrator services have been neglected. One common weakness is that MSGs have struggled to develop a clear and consistent definition of materiality (see this guidance note). In other cases, EITI Reports have not provided a clear assessment from the Independent Administrator “on the comprehensiveness and reliability of the (financial) data presented”. The terms of reference for independent administrators also require “an assessment of whether all companies and government entities within the agreed scope of the EITI reporting process provided the requested information … including naming any entities that failed to comply with the agreed procedures”. Lesson: Comprehensive tax and revenue information is critical if transparency is to lead to accountability. This is an area that needs more attention from the International Secretariat, MSGs and Independent Administrators.

Second, only 3 of 12 (25%) of countries achieved a score of “satisfactory progress” or above on requirement 2.2 on license allocations and only 3 of 13 (23%) achieved a score of “satisfactory progress” or above on requirement 2.3 on the register of licenses. These disclosures are rarely (if ever) the responsibility of national EITI bodies. Rather, the expectation is that the EITI Report will refer to other publicly available sources of information. Lesson: EITI Reporting and Validation has shown that that licensing systems are not comprehensive and that further reforms are needed. This is not necessarily a weakness of EITI implementation, but rather an illustration of the EITI’s diagnostic value.

Third, the 2016 Validations show that state owned enterprises play a key role in many EITI implementing countries, and that disclosures regarding these firms are often in inadequate. Requirement 2.6 regarding state participation was considered applicable in 10 of the 13 countries. In 9 of these 10 cases, progress was less than satisfactory. In the eight cases where requirement 6.2 on SOE quasi-fiscal expenditures was considered applicable, no country achieved satisfactory progress. Lesson: more work is needed to engage SOEs in the EITI Reporting process.

Finally, the Validations show that the coverage of social expenditures by extractive companies is often problematic. Requirement 6.1 on social expenditures was considered applicable in 10 out of 13 countries. In 9 of these cases, progress was less than satisfactory.[MOU2]  In most cases these expenditures are not large, especially compared to other revenue streams. However, there is often significant public interest in whether companies are delivering on mandated social payments, and how voluntary social payments are managed. As one stakeholder put it: “while these payments are not financially material; they are a politically essential aspect of the EITI”.

Validation is a heavy process but it is improving implementation

These results highlight the value of Validation. The process has been demanding for all involved. It is important that progress is celebrated, and that weaknesses are addressed openly. The countries that have completed Validation are now working through the corrective actions, and considering other recommendations that have emerged from the Validation process. We will explore these opportunities in the next blog in this series.

Any questions of on the Validation methodology and the Validation findings can be directed to the EITI’s Validation Manger Alex Gordy and the EITI Technical Director Sam Bartlett.

Validation explained

Validation has always been a key feature of EITI implementation, and part of what makes the EITI unique. When a country joins the EITI, deadlines are established for regular reviews. This provides an opportunity for all stakeholders to step back and take stock of their work. It also provides a mechanism for the EITI Board to recognise progress and, where progress has stalled, to consider suspension or delisting. Previously, implementing countries were classified as “candidate” or “compliant”. This was easy to understand, but did not do justice to the complexity of most cases. Under the new Validation system, the results are presented in a more nuanced (and useful) way, with various levels of progress that can be tracked over time. The supporting documentation includes a detailed initial assessment, the Validation report prepared by the independent Validator, comments from multi-stakeholder groups (MSGs), and the decision of the EITI Board. The Board decision includes a scorecard addressing each of the EITI requirements and corrective actions addressing all the requirements assessed as less than “satisfactory progress”.