Invitation to tender: Proposal for EITI rating scheme

The EITI International Secretariat seeks applications from suitably qualified service providers to develop a proposal for a rating scheme for the EITI standard. The EITI sets a global standard for transparency in the extractive industries.  It supports improved governance in resource-dependent countries through the verification and full publication of company payments and government revenues from oil, gas and mining. Validation is the EITI's quality assurance mechanism to ensure that the countries implementing the initiative are fully complying with the international standard. The methodology for the EITI is set out in the EITI Rules.

Additional information is presented below regarding:

1. Introduction
2. Purpose of the assigment
3. Background
4. Main tasks
5. Timeline and deliverables
6. Skills and competencies required
7. Submission of applications
8. Reference materials 

Annex A- Evaluator's findings on a Flexible Rating Scheme

Annex B-  Scoring and Peer Review: Lessons for the EITI from UNCAC, APRM, and PEFA 

1 Introduction

The Extractive Industries Transparency Initiative (EITI) is a global standard for revenue transparency in the extractive industries.  The EITI Criteria and requirements establish the scope of the EITI standard. Validation is an independent assessment that verifies whether implementing countries have met the requirements in accordance with the EITI Rules[1]. Following the recent evaluation[2] of the EITI, the EITI Board has requested that an alternative system for assessing compliance with the standard and EITI performance is explored.   

2 Purpose of the assignment

The overall purpose of the assignment is to develop a proposal for a rating scheme for the EITI standard. The rating scheme should maintain the EITI core requirements as the minimum EITI standard, but should be designed to reward high quality EITI processes and reports, and complementary actions agreed by implementing countries to realise the broader EITI Principles.

It is expected that the proposal for a rating scheme includes:

1.  A brief overview of strengths and weaknesses in similar rating schemes;
2. The basis for assessing implementation and rating EITI implementing countries;
3. Scoring methods/weighting of the core elements that form the basis for assessment;
4. The process for how scoring and rating will be conducted.

The proposal will contribute towards the wider EITI Board discussion on the strategic direction for the EITI in the next 3-5 years.

3 Background

Three and a half billion people live in countries rich in oil, gas and minerals. With good governance the exploitation of these resources can generate large revenues to foster economic growth and reduce poverty. However when governance is weak, such resources endowments may result in poverty, corruption, and conflict. The Extractive Industries Transparency Initiative (EITI) was launched in 2002 to strengthen governance by improving transparency and accountability in the extractives sector.

The EITI is a global standard for revenue transparency in the extractive industries implemented by 35 countries. It is a coalition of governments, companies, civil society, investors and international organisations. In 2006 it developed a robust yet flexible methodology for assessing the progress of implementing countries in adhering to the EITI principles and criteria, called validation. The EITI Board (established in October 2006) and the International Secretariat (established in September 2007) are the guardians of the EITI process and oversee the Validation in each country.

The EITI Criteria and requirements currently establish the scope of the EITI standard. In order to be recognised as an EITI Candidate, a country must meet five sign up requirements. Candidate countries have 18 months to produce their first EITI report disclosing payments and revenues from the extractive sector, and two and a half years to complete Validation and submit a final Validation report to the Board. There are currently 12 Compliant countries and 23 Candidate countries. Compliant countries must maintain adherence to the EITI requirements, and must be revalidated within five years in order to retain Compliant status.

Validation is the quality assurance mechanism of the EITI standard. It provides an independent assessment of the progress achieved and identifies what measures are needed to strengthen the EITI process. Validation is carried out by an independent Validator selected by the multi-stakeholder group in-country, using the methodology set out in the EITI Validation Guide. If the EITI Board considers a country to have met all the EITI requirements, the country will be recognised as EITI Compliant. If a country has made good progress, but does not meet all of EITI requirements, the Board will renew the country's Candidate status for another twelve months. Where Validation shows that no meaningful progress has been achieved, the Board will revoke the country's Candidate status. 

EITI recently underwent an independent evaluation[3]. The evaluation included a detailed assessment of the EITI standard and the Validation model. The evaluators concluded that 'EITI's validation scheme appears too rigid along a series of important verification dimensions and may wish to consider a more dynamic and development-oriented ratings scheme. The EITI Validation approach encourages "just in time, just sufficient" performance by Candidate countries rather than incentives for constant improvement and encouragement towards extended value-chain monitoring. It also sets some standards, such as for public accounts audits that many countries for years will not formally be able to fulfil. The EITI may wish to consider a more flexible rating approach that provides a more objective assessment of degree of fulfilment of "good practice" or international standards on key indicators'. The evaluators' recommendations for a flexible ratings scheme are set out in annex A.

4 Main tasks

The final report with the proposal for an EITI rating scheme is expected to include:

1.  A brief overview of strengths and weaknesses of similar rating schemes, including for example African Peer Review Mechanism[4], Bribe Payers Index[5], Commitment to Development Index[6], Corruption Perception Index[7], Doing Business[8], FATF[9], PEFA assessments[10] and United Nations Convention Against Corruption Review Group[11].  The consultant should consider the note prepared on the review mechanisms of UNCAC, APRM, and PEFA (Annex B).

2.  The basis for assessing implementation and rating implementing countries. The EITI Strategy Discussion Paper[12] suggests that the rating scheme has three core elements. The consultant might wish to propose other alternatives, including possibilities for nuanced scoring as suggested in the evaluation (Annex A).

• Assessment of whether the EITI requirements 1-18 are met or unmet, with a clear indication of whether the country has achieved Compliance.
• Scoring of EITI performance: For each of the requirements 1-18, countries would be awarded points for achievements that exceed the minimum requirements. For example, for requirement 9, countries would obtain points for meeting the minimum requirements. They would also receive points for setting ambitious materiality thresholds, for covering social payments, barter agreements, subnational payments, and disaggregation. At requirement 18, extra points could be awarded for providing relevant context in EITI reports, such as data on overall contribution to public revenue, and for widespread and innovative efforts to disseminate findings to local communities.
• Scoring on wider governance reform: A country would be able to obtain points for other related governance efforts. Issues might include licensing, contracting, production transparency, profit transparency, coverage of other sectors, beneficial shareholder transparency, regulation, citizen participation, public oversight, revenue collection, revenue management, linking revenue spending to development planning, budget openness and public financial management, social and environmental issues, local content, etc. A concrete example would be where countries go beyond the EITI requirements to use the EITI Reports to verify and audit that the payments and revenues received are consistent with the regulatory / contractual obligations.

3.  Scoring methods: The consultant should develop a proposal for scoring and weighting methods. The assessment should give primacy to the core requirements either through weighting or some sort of league arrangement separating Candidate countries from Compliant countries. 4.  Scoring and rating process: The consultant should explore the process for how scoring and rating will be conducted. The process for scoring and rating will need to consider, but not be limited to:

• Frequency (annually, biannually, biennial, ad hoc)
• The stage of implementation at which a country will join the rating scheme (Candidate, after publication of the first EITI report, Compliance etc.);
• How the assessment is conducted (self-assessment, assessment by the Board, Board accredited validators or the International Secretariat, peer review or a combination).
• The consultant should pay particular attention to the need for multi-stakeholder group participation, assessment consistency, conflicts of interest, quality assurance and cost.

5 Timeline and deliverables

The assignment is expected to commence on 8 December 2011.  It is foreseen that a total of 15 days will be required to complete the assignment.  The consultant will submit deliverables as specified below:

-          Draft report with the proposal for an EITI rating scheme as specified in section 4 to be submitted no later than three weeks from commencement of the assignment.

-          Based on feedback from the Secretariat, a final report to be submitted no later than 9 January 2012.

6 Skills and competencies required

The application will need to state how the tasks set out in section 4 will be fulfilled, and need to demonstrate:

• Technical and financial skills, including knowledge and work on development including transparency and good governance, public finance and financial accountability, multi-stakeholder dialogue, working with civil society and poverty reduction and economic management.
• A demonstrable track record in similar work.
• Knowledge of the oil, gas and mining sectors or other natural resources sectors.

Tenders for this contract will be assessed in accordance with good commercial practice, taking into account the consultant's relevant experience for the assignment and the qualifications of the key staff proposed.

7 Submission of applications

Suitably qualified service providers should submit an application outlining how they meet the above requirements. The application should also include:

• A summary (no longer than 500 words) outlining the applicant's experience and expertise.
• A list of reference projects demonstrating the technical capacity of the candidate;
• A financial proposal.

Applications must be submitted in English to the EITI International Secretariat by email or postal service. The electronic application must be submitted by 2 December 2011, 17:00 Central European Time. The Secretariat will confirm receipt of all applications. Applications lodged after this date will not be accepted.

By email to Dyveke Rogan (drogan@eiti.org)  

By postal service:

EITI International Secretariat
Att. Dyveke Rogan
Ruseløkkveien 26
0251 Oslo
Norway

8 Reference Materials

1. EITI Rules, 2011 edition: http://eiti.org/document/rules
2. EITI Strategy discussion paper: http://eiti.org/about/strategy-working-group
3. EITI website: www.eiti.org  
4. Achievements and strategic options: Evaluation of the Extractive Industries Transparency Initiative, Final report by Scanteam: http://eiti.org/document/2011-evaluation-report
5. Report on the Criteria and Methodology for Determining the Eligibility of Candidate Countries for Millennium Challenge Account Assistance in Fiscal Year 2012, Millennium Challenge Corporation:http://www.mcc.gov/documents/reports/report-2011001066202-fy12-selection-criteria.pdf
6. Ghana's Oil Boom: A readiness report card: http://www.oxfamamerica.org/files/ghana-oil-readiness-report-card.pdf
7. Bribe Payers index: http://bpi.transparency.org/
8. Corruption Perception index: http://www.transparency.org/policy_research/surveys_indices/cpi/2010
9. Doing Business: http://www.doingbusiness.org/
10. PEFA assessments:http://web.worldbank.org/WBSITE/EXTERNAL/PEFA/0,,contentMDK:22677273~hlPK:7334624~menuPK:7313083~pagePK:7313176~piPK:7327442~theSitePK:7327438,00.html
11. Natural Resource Charter: http://www.naturalresourcecharter.org/  
12. United Nations Convention Against Corruption, Review group:http://www.unodc.org/unodc/en/treaties/CAC/IRG.html
13. African Peer Review Mechanism: http://aprm-au.org/
14. Commitment to Development Index: http://www.cgdev.org/section/initiatives/_active/cdi/
15. Financial Action Task Force: http://www.fatf-gafi.org/pages/0,2987,en_32250379_32235720_1_1_1_1_1,00.html

Annex A – Evaluator's findings on a Flexible Rating Scheme

Box 5.10: Flexible Rating Scheme may Address Weaknesses in EITI Validation

Instead of an absolute list of criteria that must be fulfilled – today no more than such a list of criteria need to be fulfilled! – an EITI ratings scheme could encompass "the desired universe" and let each country decide how many of these dimensions it wishes to be rated on. Following PEFA's budget cycle logic, this "desired universe" could be the value chain from concession to export, but where revenue payments could be given an important weighting in the overall scheme and should be based on the EITI Principles. If a country does not wish to be rated on its concession/contracting performance, it will score a "0" for all the world to see.

Rather than binary values ("Yes/No") on each dimension, there could be a defined list of ratings, such as with PEFA. For the current requirement of public accounts audits, a maximum value of "5" could be given to countries that adhere to IPSAS accrual accounting standards, a "4" for IPSAS cash accounting, a "2" for a statement by the national audit body stating the accounts are in accordance with Generally Accepted Accounting Practices (GAAP) etc. The decomposed ratings would show where the country is performing well and where it needs to improve.

Such a ratings scheme allows for flexible expansions if EITI agrees that its standards should be modified by including new dimensions[13]. Such a ratings scheme also makes it simple for EITI to create "virtual strategic partnerships" in complementary fields. It can point to ratings schemes that track the value-chain downstream through public finance management assessments; it can report such ratings alongside its own to show how petroleum sector performance is compared with how public finance management is seen; or it can in fact aggregate several such indexes into its own system as long as the ratings systems are methodologically compatible. This opens up the EITI certification scheme to external linkages, both showing how EITI contributes to and perhaps can be seen as part of other systems, but also helps EITI define the boundaries for its own activities and thus helps it clarify where it does not need to engage[14].

Such a ratings scheme can be based on the questionnaire approach used in other systems[15]. A full validation/audit can be carried out for example every three years, and in the intervening two years a partial audit of for example the five poorest indicators or the sub-set of indicators that the government prefers can be done. This allows for a constant update of the scorings, and in particular gives a country an annual chance to upgrade in those areas where additional effort will yield the highest payoff in terms of ratings improvements.

Such a ratings scheme would move EITI away from a one-level certificate to a system of perhaps five classes of performance, each one defined by upper and lower values on the ratings system. No country would presumably ever score a perfect 100, so all countries would have incentives year-onyear to improve performance. The system would also be providing capital and risk assessment markets useful data on where performance has improved, why, and where remaining weaknesses are, and what needs to be done to address them.

Source: Scanteam (2011) Achievements and Strategic Options: Evaluation of the Extractive Industries Transparency Initiative. Final Report. Oslo. May 2011. Source: http://eiti.org/files/2011-EITI-evaluation-report.pdf. Pages 51-52

Annex B- Scoring and Peer Review: Lessons for the EITI from UNCAC, APRM, and PEFA

Draft for Discussion

Jurgen Reitmaier

15 November 2011

In the context of its 18^th meeting on 25-26 October 2011, the EITI Board discussed the strategic directions for the EITI. It held, inter alia, that "encouraging linkages and developing synergies with other initiatives and instruments such as the UN Convention against Corruption, the APRM mechanism, PEFA, etc., … is important" (Board Minutes). The ground for this view was prepared by the Evaluation of the EITI (Scanteam, May 2011) and the World Bank's input of August 2011 to the EITI Board's Strategy Working Group. While the exploration of linkages and synergies is not limited to scoring methods, the EITI's interest in the APRM and PEFA is mainly motivated by considerations around scoring and peer review. As UNCAC now also features a peer review mechanism, it may be useful to compare the review methods of all three instruments and discuss their relevance for the EITI. This note attempts to do this.

1.       United Nations Convention Against Corruption (UNCAC)

In November 2009, the States Parties to UNCAC adopted terms of reference of a Mechanism for the Review of Implementation of the Convention (UNCAC Resolution 3/1). Guided by an Implementation Review Group (IRG) and a secretariat, the mechanism started up in July 2010, with 27 States parties selected to be reviewed in Year 1 of the first five-year round.

The UNCAC review is a peer review comparable to the reviews practiced by the OECD: "Each State party shall be reviewed by two other States parties" (Resolution 3/1, Para. 18), drawn by lot and with one reviewing state from the same region. Each state party appoints a focal point and up to 15 governmental (!) experts for the purpose of the review process, which shall be an intergovernmental process and not produce any form of ranking (Paras. 3, 4, and 21). State parties to be reviewed may request twice that the drawing of lots be repeated. Each country review consists of the following steps: (1) completion of a self-assessment by the state party under review, involving broad consultation of stakeholders at the national level; (2) desk review of the self-assessment and other relevant material by the reviewing states; (3) if agreed by the state party under review, a country visit or joint meeting elsewhere; (4) preparation of a country review report, to be finalized "upon agreement between the reviewing States parties and the State party under review" (Para. 34). It is further stipulated that the "country review reports shall remain confidential" (Para. 37), although executive summaries of such reports have been published.

The early experience with the review process and specific country review reports were discussed by the IRG during the recent UNCAC Conference of States Parties in Marrakech (October 2011). An earlier decision to exclude civil society representatives (organized in an UNCAC Coalition of civil society) from IRG meetings was reaffirmed, despite lobbying in favor of inclusion by several states parties.

Possible model character for EITI?

A peer review à la UNCAC by two other implementing countries selected by lot could be taken into consideration (as it already is, following the DAC example) for a qualitative assessment of implementing countries' efforts beyond basic compliance with EITI requirements. It would, of course, have to be stripped of the political permeating the UNCAC review mechanism. The EITI review could be conducted almost entirely amongst the MSGs of the reviewed and reviewing countries.

2.      African Peer Review Mechanism (APRM)

Despite its name, the APRM is primarily a country review through an international expert team led by one member of the APR Panel of Eminent Persons. The element of peer review comes in only at the very end of the review process, when the APR Forum of participating Heads of State and Government discuss the country review of a given country in the face of that country's leader. The eminent person in charge of a scheduled country review is selected from within the APR Panel, and the expert team is assembled by the APRM Secretariat.

Like the UNCAC review, the APRM review starts out with a self-assessment by the country under review, following a detailed questionnaire in four chapters: political governance, economic governance, corporate governance, and socio-economic development. Following a submission by the EITI International Secretariat, the recently revised APRM self-assessment questionnaire has specific questions on EITI status in both the economic and corporate governance sections.

A country's self-assessment needs to show evidence of a broad participatory approach. The review team is invited to an initial support mission, followed by the review mission proper, and is held to consult broadly with stakeholders in-country. The review team prepares a review report that is vetted by the APRM Secretariat and the APR Panel of Eminent Persons. The government of the country under review may submit a response to the review, which becomes an integral part of the review report. The APR rules foresee publication of the country review report within six months of the peer review by the APR Forum.

Possible model character for EITI?

Like the UNCAC review mechanism, the APRM could serve as model for assessing activities beyond basic EITI compliance. It is indeed preferable in that the review is done by a team of experts, and only the final sign-off is by peers, in the case of the EITI perhaps by the Board. The detailed self-assessment questionnaire of the APRM is a good practice, which could be adapted to the EITI. It would require developing a catalogue of possible extensions of the EITI and illustrating it progressively with good practices, against which performance could be assessed.

3.       Public Expenditure and Financial Accountability (PEFA) Framework

A PEFA assessment measures the extent to which a country's public financial management (PFM) systems, processes, and institutions meet the core requirements of an open and orderly PFM system as laid down in the PEFA framework. These core requirements are articulated in 28 high-level indicators across six groups of PFM activities. Each of these indicators is sub-divided into between two and four lower-level dimensions, each of which is in turn equipped with a catalogue of stylized practices ranging from neglect or poor practice to model behavior for that particular dimension. A country's actual PFM practices can now be scored in detail and objectively against this catalogue of dimensions and indicators. As explained by the Head of the PEFA Secretariat in September 2011, more than [100] PEFA assessments have so far been conducted across all types of countries, including numerous repeat assessments, providing a large body of experience with a nuanced scoring system.

Possible model character for EITI?

The PEFA framework seems particularly well suited as a model of nuanced scoring for certain base requirements of the EITI, less well for the voluntary extensions beyond the minimum standard. It is thus a complement, not an alternative, to the UNCAC or APRM assessment tools; they each may help address a different issue of the EITI's reform agenda.

As noted, not all requirements of the EITI lend themselves to nuanced scoring, particularly if the concept of compliance and the bar for attaining it must not be modified. The following requirements could be accompanied by a catalogue of stylized practices that could be used in scoring actual practices: 5 (work plan), 6 (engagement of civil society), 7 (engagement of companies), 8 (addressing obstacles), 11 (ensuring full participation), 12/13 (auditing to international standards), 17 (performance of reconciler), 18 (dissemination), 20 (commitment to follow-up).

4.      Conclusion

In the wake of the recent Board discussion on EITI strategy, this note has considered the review mechanisms of UNCAC, APRM, and PEFA. It argues that elements of the UNCAC and APRM mechanisms could be useful in the design of a monitoring framework for voluntary extensions of the EITI beyond the core requirements, in particular up and down the value chain. Both, UNCAC and APRM, are peer review mechanisms, but they differ in the degree of reliance on stakeholder participation and expert assessments; APRM is more appealing on both fronts. Meanwhile, PEFA could help in designing nuanced scores of possible shortfalls below minimum requirements of EITI. Such scores could accompany validation findings of "meaningful progress" or "close to compliant" should they survive in the reform.

 

 

---

End Notes

[1] http://eiti.org/document/rules

[2] http://eiti.org/document/2011-evaluation-report

[3] http://eiti.org/document/2011-evaluation-report

[4] http://aprm-au.org/

[5] http://bpi.transparency.org/

[6] http://www.cgdev.org/section/initiatives/_active/cdi/

[7] http://www.transparency.org/policy_research/surveys_indices/cpi/2010

[8] http://www.doingbusiness.org/

[9] http://www.fatf-gafi.org/pages/0,2987,en_32250379_32235720_1_1_1_1_1,00.html

[10] http://web.worldbank.org/WBSITE/EXTERNAL/PEFA/0,,contentMDK:22677273~hlPK:7334624~menuPK:7313083~pagePK:7313176~piPK:7327442~theSitePK:7327438,00.html

[11] http://www.unodc.org/unodc/en/treaties/CAC/IRG.html

[12] This paper was submitted to the EITI Board in October 2011 and provides a starting point for considering an EITI rating scheme. It is expected that the consultant carefully considers the issues and options outlined in this paper. The paper is available at http://eiti.org/about/strategy-working-group .

[13] If a ratings scheme for the "value chain" today sums to 100 and EITI later on, in line with ISEAL Alliance standards, wishes to add in environmental and social standards (in the mining/extraction operations), these could be given a weight of 10 each. The old rating scheme would thus be reduced to a maximum of 80points. If a country scored 80 out of 100 in the old scheme, in the new one these points would now count as 64 (80 * 80%). Added to this would be whatever the country scored in the environment dimension and the social dimension.

[14] Nigeria's NEITI has tracking of revenue allocations and expenditures as part of its mandate. It would make NEITI's task a lot simpler if this can be monitored for example through a PEFA or Open Budget Index instead of NEITI itself having to establish and monitor a public finance management system – a near-impossible task.

[15] The Revenue Watch Institute's handbook "Drilling Down: The Civil Society Guide to Extractive Industry Revenues and the EITI" contains sets of very good questions for EITI's validation requirements, for example.