Consultation on Validation
This consultation is concluded.
In 2015, the EITI Board conducted a review of the Validation process, including an extensive consultation process. The consultation has now closed. An overview of changes to the EITI Standard is available here, including a summary of changes to EITI Validation. An overview of the current Validation system is available here.
2015 Consultation on Validation
In 2015, the EITI undertook a review of the Validation system. EITI stakeholders were invited to submit comments and proposals. Additional details on the review are provided below.
Validation is an essential feature of the EITI process. It serves to assess performance and promote dialogue and learning at the country level. It also safeguards the integrity of the EITI by holding all EITI implementing countries to the same global Standard. Validation is intended to provide all stakeholders with an impartial assessment of whether EITI implementation in a country is consistent with the Requirements of the EITI Standard. Validation also addresses the impact of the EITI and provides recommendations for strengthening the future implementation of the EITI.
Although the adoption of the EITI Standard in May 2013 was a major step forward in terms of strengthening the EITI’s relevance and potential to contribute to improved governance in the extractive industries, certain issues were inevitably left unaddressed or have emerged as countries have begun implementing the EITI Standard. As countries have made improvements in their reporting under the Standard, it has become clear that the Validation system may need refinement. The Board recently agreed new Terms of Reference for validators. In 2015, seven countries are scheduled to undertake Validation in accordance with the EITI Standard. There are concerns by many stakeholders that Validation is unlikely to provide fair assessments. While the bar for achieving compliance should not be changed, there are concerns that the current Validation system does not adequately consider the diversity of implementing countries or take into account progress over time. There are also concerns about nomenclature and other terminology.
Thus, the EITI Board agreed at its 29th meeting in Brazzaville to consider developing proposals for a stronger Validation system and alternative nomenclature. It also agreed to consider whether the current Validation firms provide an adequate assessment of implementation efforts and value for money, or whether alternative means of conducting Validation should be explored. In June 2015, the EITI Board’s Implementation Committee met in Abuja to further consider the challenges with the current Validation model and considered how Validation may potentially evolve. While there is broad agreement that Validation should continue to assess compliance with the EITI Requirements as at present, i.e. no changes to the current bar for achieving compliance a new system could, if agreed by the EITI Board, potentially contain a more disaggregated assessment of each requirement, recognise progress towards meeting the requirements and the direction of travel, possibly include more frequent assessments, and consider alternatives to the current use of validators. Before elaborating these proposals further, the EITI Board first requested the International Secretariat to seek input from stakeholders through a public consultation.
The EITI is currently analysing the Validation system and assessing a number of different proposals with a view to improve the system. While the current Validation system has served the EITI relatively well in the past, some stakeholders are concerned that the Validation process is no longer fit for purpose as the EITI continues to expand both in membership and in scope. Some note that the current Validation system does not recognize the diversity across implementing countries, local circumstances, and the progress that countries make towards meeting the EITI Requirements even if these efforts are not always sufficient to reach compliance on all requirements. Others have expressed concern that the current nomenclature sends misleading signals to implementing countries by suggesting that the job is done, or that progress is made when there may in fact be significant shortcomings or little impact. Some stakeholders have also highlighted that the current use of Validation firms is expensive, inefficient and risks not providing a true reflection of implementation efforts and the EITI’s impact.
A refined Validation model could resolve some of these challenges by giving greater recognition to progress and diversity, whilst maintaining the current bar for what is required of countries to become compliant with the EITI Standard and the consequences for non-compliance. EITI stakeholders are invited to submit comments and proposals for improving EITI Validation. Below are some questions that the EITI is seeking to address through this consultation. We welcome comments by 7 September 2015 to Dyveke Rogan (firstname.lastname@example.org). All submissions will be published unless otherwise indicated.
Stakeholders may wish to address the questions and statements below, but should not consider themselves confined to them. Stakeholders are welcome to provide alternative proposals.
(i) To ensure the EITI was addressing new and developing areas of transparency in the extractive industries, the standard was strengthened in 2013 to include a number of new disclosure requirements. There is no desire to lower the bar, or change what is required of countries to achieve compliance. However, combined with the current pass/fail assessment approach and tough timelines for achieving compliance, this higher bar could represent three risks: (1) that implementing countries will become demoralised because Validation does not take into account progress and considerable advances in reporting under the new Standard; (2) that implementing countries will not be incentivised to take on additional issues beyond the EITI Requirements addressing the particular needs of their country, but rather seek to find ways to reduce the scope of EITI reporting to heighten chances of passing validation; and (3) that the EITI will be under pressure to take positive decisions regarding a country’s compliance even if implementation does not quite measure up because of the risk of losing members. There is a real risk that without considering refinements to the validation system that would recognise the progress and improvements in implementation over time, the EITI will find itself in a difficult situation if the number of countries with good and improving but not fully compliant implementation grows but the validation system fails to distinguish these countries from those where implementation is clearly inadequate.
- Should effort and progress in meeting the requirements over time be taken into account in validation?
- Should validation encourage reporting that go beyond the EITI requirements?
(ii) The EITI Standard requires implementing countries to work on a wider range of issues related to natural resource governance than before. While this greatly enhances the potential of the EITI to contribute to improve governance along the whole value chain, it means uneven workloads for implementing countries. As the EITI Standard has become broader it has become unevenly applicable but compliance is still expected within equal timeframes. Some countries will have considerable challenges in nearly all areas covered by the EITI Standard. Other countries will have less difficulty in meeting the requirements but will seek to use the EITI to tackle additional issues that are important to their national development.And while the EITI Standard itself continues to evolve, the validation methodology could do a better job at taking into account this diversity. It was designed at a time when the EITI requirements only related to revenue transparency and were applicable to all member countries. As a result, it is quite likely that a country with a small and simple extractive sector could become compliant despite having made relatively few efforts with implementation, whereas countries with larger and more complex sectors would need to put in considerable efforts over many years to be recognised (compliant).
- Should the timeframe for countries to achieve compliance be the same for all implementing countries? If so, should the timeframe be a fixed number of years as at present?
- Should progress and direction of travel matter for how much time countries are given to achieve compliance?
- To what extent should the local context in which the EITI is being implemented be taken into account during validation?
(iii) The current binary model of assessment where requirements are assessed as either met or unmet is simplistic and does not give recognition to progress made by implementing countries, nor to the broader scope of the EITI Standard. There are numerous examples of how implementing countries have gradually improved their EITI reporting, but where efforts have not been acknowledged because the results might not quite measure up to full compliance. Just as it would be incorrect to deem a country compliant if there are still shortcomings, it seems contrary to the objectives of the EITI to punish a country that faces major governance challenges with suspension or delisting if it is has been documented that the country is making progress in the right direction. The concern with the binary model has been flagged by implementing countries and other stakeholders since 2011, when the EITI was still a relatively simple reporting tool focused on revenue transparency. This issue has become even more pressing with the more complex Standard, where it is no longer only one issue – revenue transparency – that is being assessed, but governance across the extractive industry value chain. As the current system provides limited opportunity to reflect nuances in compliance across the requirements, a country may well be compliant with EITI’s licensing, production, revenue collection and revenue management requirements, without this being recognised because it fails on e.g. workplanning and state-owned enterprises which means that the overall assessment is “not met”. Whilst there is agreement that compliance should only be awarded when all the requirements are met, can the validation make more nuanced assessments of meaningful progress, or can assessment of requirements themselves be more nuanced?
- Should EITI requirements continue to be assessed as met or unmet?
- Should there be more disaggregated assessments, showing which requirements are met and which requirements are unmet, including the level of progress in meeting each requirement?
- Should the consequences of not reaching compliance status be removed? I.e. countries are allowed to stay members of the EITI as long as they make progress towards meeting requirements?
- How can validation measure progress or direction of travel towards meeting a requirement?
- Should validation take place more frequently to measure progress, for example at the end of each EITI reporting cycle, or is the current frequency of every 3 years adequate?
(iv) For many countries, and for many champions and stakeholders within those countries, the ‘prize’ of compliance has been important to drive the EITI process. The flipside is that complacency may follow compliance. The concept of compliance can be interpreted as problematic in that it signals that the job is done and reduces incentives for continuous improvement.There is often little motivation once compliance has been achieved to set goals that will continue to drive reform in the management of extractives. Similarly, some of the terminology provided for in the EITI Standard aimed at indicating validation outcomes, e.g. limited progress, meaningful progress, etc., is potentially misleading because compliance with the individual requirements is not broken down by requirement. Thus, a country may be deemed to have made meaningful progress overall in circumstances where there are significant breaches of individual requirements.
- Should the concept of “Candidate” and “Compliant” be replaced, and if so with what?
- What terminology could be suitable to indicate the various degrees of progress in meeting EITI Requirements? Are the current concepts of “limited progress” and “meaningful progress” appropriate?
- How can Validation incentivize countries to continue to progress and innovate both before and after reaching compliance status?
(v) The EITI is presently outsourcing the assessment of implementation to accredited consultancy firms. There have on several occasions been concerns about the quality, cost and efficiency of the Validators. Some reforms were made with the EITI Standard and Validators are now contracted and paid by the International Secretariat. While this might address some of the earlier concerns related to poor quality Validation reports and conflicts of interest, accredited Validators would still need in-depth knowledge of the EITI as well as global and local extractive sector governance issues in order for Validation reports to be useful. While all Validators are accredited, experience with Validation shows that there is very different understanding of the EITI across Validation teams, which again impacts the quality and consistency of the way implementation is being assessed.
- Should multi-stakeholder groups and/or local and international experts on extractive sector governance have a greater role in Validation?
- Should the International Secretariat have a greater role in carrying out Validation assessments? What are the risks and benefits of this approach? What should be done to mitigate conflicts of interest?
The following submissions have been received by the International Secretariat:
 Azerbaijan, Ghana, Kyrgyz Republic, Mongolia, Sao Tome and Principe, Solomon Islands and Timor Leste.
 Ref. 2011 Scanteam evaluation