Invitation to Apply for Evaluation of the EITI

UPDATE, 13 September 2010: Please take note of the following clarifications.

On behalf of the Board of the Extractive Industries Transparency Initiative (EITI), the International EITI Secretariat seeks applications from suitably qualified service providers to undertake an evaluation of the EITI. The EITI sets a global standard for transparency in the extractive industries.  It supports improved governance in resource-dependent countries through the verification and full publication of company payments and government revenues from oil, gas and mining. Validation is the EITI’s quality assurance mechanism to ensure that the countries implementing the initiative are fully complying with the international standard. The methodology for the EITI is set out in the EITI Rules.

Additional information is presented below regarding:

  1. Introduction
  2. Purpose of the Evaluation
  3. Background
  4. The Evaluation
  5. Main tasks and Deliverables
  6. Reference Materials
  7. Skills and competencies required of Evaluators
  8. Submission of applications
  9. Process and Outcome Indicators
  10. UPDATE, 13 September 2010

1 Introduction

The Extractive Industries Transparency Initiative (EITI) is an international standard for transparency in natural resource revenue management.   The initiative was launched in 2002 but the indicators for meeting the standard were not established until 2006.  An evaluation was prepared for the 4th Global Conference in February 2009 that assessed EITI’s contribution to good governance of natural resources.  EITI implementation has gathered considerable pace since 2009: 31 countries are now implementing, 20-plus validation processes are completed or underway and 46 EITI reports have been published.  There is now richer evidence and data to evaluate the performance of the EITI globally and its structures and policy framework, and how and to what extent the latter has contributed to  impact on key development outcomes globally and in individual EITI implementing countries. The EITI Secretariat in its work plan 2010 committed to “commission an independent evaluation of the EITI in time for the 2011 Conference.” Additionally, following the Secretariat Expenditure Review presented to the Board in Baku in October 2009, the Finance Committee concluded by noting that the Secretariat provided a high level of service but more tools were needed to assess if the Secretariat provides value for money. The Board also ratified in Baku the establishment of a working group to develop EITI outcome indicators. 

The working group for EITI outcome indicators, created on 12 May 2010, has prepared a list of outcome indicators (see section Process and Outcome Indicators below) and have endorsed the present terms of reference for the evaluation to be conducted by an independent party and be presented to the next Global Conference.

2 Purpose of the Evaluation

The overall aim of this evaluation is to document, analyse and assess the relevance and effectiveness of the EITI and its contribution, through improved governance and accountability of the extractive sector, to sustainable development and poverty reduction. The relevance and effectiveness relate to the extent to which the EITI is achieving its main objective of increasing transparency over payments and revenues in the extractives sector. 

It is expected that this evaluation answers the following questions:

1) What are the results of the EITI and what impact the EITI is having?

The evaluation should provide a better comprehension of the precise benefits of the EITI through a combination of its contribution to improving the understanding of the sector, identifying actions and wider reforms required to improve the management of the sector, especially revenue and expenditure management. Recognising that given the importance and complexity of the development outcomes involved in ensuring sustainable development and in reducing poverty levels, the evaluation is not expected to establish causation but rather to 1) provide context, establish benchmarks and indicate directional change of key development outcomes such as fight against corruption, governance and accountability of the extractive sector, protection of civil society whilst engaged in legitimate activities, management of resources obtained from natural resources and 2) provide evidence of the results the EITI is achieving in implementing countries.

2) Is the EITI “fit for purpose” and does the EITI provide “value for money”?

The evaluation should assess whether the current institutional and managerial framework of the EITI (i.e. principles, criteria and policies, the International Secretariat and Board) is delivering results in accordance with the resources allocated and its mandate as derived from its Principles and Criteria. Additionally the evaluation should provide inputs for discussing future direction of the EITI Board and Secretariat, especially in terms of size of secretariat, level or source of support, working method, strength of supporting network, monitoring, policies, and scope and boundaries.

3 Background

Three and a half billion people live in countries rich in oil, gas and minerals. With good governance, the exploitation of these resources can generate large revenues to foster economic growth and reduce poverty. However when governance is weak, such resource endowments may result in poverty, corruption, and conflict. The Extractive Industries Transparency Initiative (EITI) was launched in 2002 to strengthen governance by improving transparency and accountability in the extractives sector. 

The EITI is a coalition of governments, companies, civil society, investors and international organisations. In 2006 it developed a robust yet flexible set of indicators for monitoring and reconciling company payments and government revenues. Implementation takes place at the country level, in a process that emphasises multi-stakeholder participation. The EITI Board (established in October 2006) and the International Secretariat (established in September 2007) are the guardians of the EITI process and oversee the validation (the quality assurance mechanism for the standard) in each country.

The EITI is a globally developed standard that promotes revenue transparency at the local level.  

To become an EITI Candidate, a country must meet four sign up indicators, including the development of a work plan documenting how the country intends to achieve EITI Compliance. The plan must be discussed with and agreed by key stakeholders.  To achieve EITI Compliant status – or to extend Candidate status beyond 2 years – a country must complete an EITI validation process.

Validation is therefore an essential element of the EITI global standard. It provides an independent assessment of the progress achieved and identifies what measures are needed to strengthen the EITI process. The validation is carried out by an independent validator selected by the national Multi-stakeholder Group, using the methodology set out in the EITI Validation Guide. If the EITI International Board considers a country to have met all the indicators in the validation grid, the country will be recognised as EITI Compliant. If a country has made good progress, but does not meet all of the EITI requirements, the country may apply to retain its Candidate status for a limited period. Where validation shows that no meaningful progress has been achieved, the Board will revoke the country’s Candidate status.  Twenty-two candidate countries had a validation deadline in March 2010. Only Azerbaijan, Liberia and Timor-Leste have achieved the status of Compliant. Equatorial Guinea and Sao Tome and Principe have been de-listed and are no longer candidate countries.  The rest of countries have received an extension to complete validation.  Ten other countries are due to be validated in the coming year.

EITI reconciliation reports are the heart of the EITI process. To date 46 reports have been produced (including 12 from Azerbaijan, which publishes two reports per year). Half of these reports have been published in the last year (July 2009-June 2010). The table below summarizes this output from EITI countries:

Country

Validation Deadline

Reports published 

Last report published in

Covering years 

Covering sectors: 

Cameroon 

Sept 2010

2

2007

2005

Oil & gas  (* considering Mining for 3rd report)

R of Congo 

Sept 2010

1

2010

2004 to 2006

Oil & gas

DR Congo 

Sept 2010

1

2009

2007

Oil & gas, Mining

Gabon 

Sept 2010

3

2008

2006

Oil & gas, Mining

Ghana 

Sept 2010

3

2008

2005

Mining (* considering Oil & gas for future reports)

Kazakhstan 

Sept 2010

4

2010

2005 to 2008

Oil & gas, Mining

Kyrgyzstan 

Sept 2010

2

2009

2004 & 2008 Data for intervening years has also been published.

Mining

Mali 

Sept 2010

1

2009

2006

Mining

Mauritania 

Sept 2010

2

2007

2006

Oil & gas, Mining

Niger 

Sept 2010

1

2009

2005 to 2006

Mining

Nigeria 

Sept 2010

2

2009

2005

Oil & gas (* considering a separate report for mining)

Peru 

Sept 2010

1

2009

2004 to 2007

Oil & gas, Mining

Sierra Leone 

Sept 2010

1

2010

2006 to 2007

Mining (* considering oil exploration for future reports)

Timor-Leste 

Compliant

1

2009

2008

Oil & gas

Mongolia 

Oct 2010

2

2009

2007 to 2008

Mining

Cote d'Ivoire 

May 2010

1

2010

2006 to 2007

Oil & gas

Central African Republic 

Nov 2010

1

2009

2006

Mining

Norway 

Feb 2011

1

2009

2008

Oil & gas

Tanzania 

Feb  2011

 

   

Yemen 

Mar 2011

 

   

Madagascar 

Mar 2011

0*

*Pilot report- 2010

2007-09

Mining

Albania 

May 2011

 

   

Burkina Faso 

May 2011

 

   

Mozambique 

May 2011

 

   

Zambia 

May 2011

 

   

Afghanistan 

Feb 2012

 

   

Iraq 

Feb 2012

 

   

Chad 

Apr 2012

 

   

Guinea 

Suspended

1

2007

2005

Mining

Liberia 

Compliant

2

2010

Fiscal year ending on June 2009

Oil & gas, Mining, Forestry, Agriculture

Azerbaijan 

Compliant

12

2009

2nd Semester 2009

Oil & gas

Equatorial Guinea 

Delisted

1

2010

2008

Oil & gas

Sao Tome & Principe 

Delisted

 

  

Oil & gas

Three countries have achieved EITI Compliant status (Azerbaijan, Liberia and Timor-Leste) and there are 28 Candidate status (Cameroon, Democratic Republic of Congo, Republic of Congo, Côte d’Ivoire,  Gabon, Ghana, Guinea, Kazakhstan, Kyrgyzstan, Mali, Mauritania, Madagascar, Mongolia, Niger, Nigeria, Peru,  Sierra Leone, Timor-Leste ,Yemen, Cote d’Ivoire, Central African Republic, Norway, Tanzania, Albania, Burkina-Faso, Mozambique, Zambia, Afghanistan, Iraq, and Chad) . Equatorial Guinea and Sao Tome and Principe were candidate countries until April 2010. Several other countries, including Indonesia, Bulgaria, Ukraine, Guatemala, Guyana, Togo and Rwanda have signalled their intent to implement the EITI, and are working towards meeting the sign up indicator requirements.

Fifty of the world’s largest oil, gas and mining companies support and actively participate in the EITI process – through their country operations in implementing countries, through international-level commitments, and through industry associations. Also, the EITI has won the support of over 80 global investment institutions that collectively manage assets worth over 16 trillion USD.

Civil Society Organisations participate in the EITI directly and through the Publish What You Pay campaign, which is supported by over 300 NGOs worldwide.  International Organisations supporting the EITI include the World Bank, IMF, African Development Bank, Asian Development Bank, the European Bank for Reconstruction and Development, Inter-American Development Bank and the European Investment Bank. These organisations provide technical and financial support to implementing countries, and support EITI outreach.

A number of governments including  Australia, Belgium, Canada, Denmark, Finland, France, Germany, Italy, Japan, the Netherlands, Norway, Spain, Sweden, Switzerland,  the United Kingdom and the United States support the EITI.  These governments provide political leadership and technical support in promoting the Initiative. Many also contribute financially to the international management of the EITI, and support implementation through direct bilateral support to EITI implementing countries or through a multi donor trust fund managed by the World Bank. 

The EITI has also been endorsed by the UN General Assembly, G8, G20, AU and EU. The EITI is overseen by the EITI International Board, chaired by Dr Peter Eigen, founder and former chairman of Transparency International. The Board consists of representatives from EITI implementing country governments, extractive companies, civil society groups, investors, and supporting country governments.  The highest governing body is the biennial EITI International Conference. The next EITI Conference will take place in early 2011.

4 The Evaluation

The evaluation will provide an independent assessment of the results of the global EITI initiative, policy framework and structures, and its impact as discussed in section 2 of this document.  It will be an independent platform for discussion of EITI strategy, workplan and policy in the run up to the EITI 2011 Conference. It will be an important background for the 2009-11 Progress Report. The evaluation report will doubtless be heavily quoted by both advocates and critics of the initiative and will thus have to be academically rigorous and robust. 

4.1 Target audience

The target audience will include:

  • The EITI Global Conference (members)
  • The EITI Board (20 members plus 19 alternates) drawn from Implementing and supporting Governments, companies and institutional investors, and civil society;
  • The EITI International Secretariat;
  • EITI Financial Supporters;
  • Key stakeholders in implementing countries (including their own multi-stakeholder groups and secretariats, key Government officials, media, etc.);
  • International development community – civil society, aid agencies, consultants, academics, etc.;
  • International business community especially, but not exclusively, in the oil, gas and mining sectors;
  • International media;
  • Parliamentarians;
  • Other multi-stakeholder initiatives. 

4.2 Main challenges

1) Diverse Audiences.

 The different EITI constituencies have a shared commitment to the EITI principles, but different expectations for EITI and different understanding of EITI effectiveness.  Consequently the agreed key performance indicators are limited in terms of comprehensiveness and appropriateness.  The evaluation will have to be informed by interviews with stakeholders, case studies, assessment of available qualitative and – ideally – quantitative data, and other anecdotal evidence. 

2) Attribution and causation difficulties.

The EITI is often part of a package of governance and economic reforms within implementing countries which collectively lead to measurable outcomes. The evaluation will need to show main trends in key development outcomes that help to understand where countries are and the directional change, if any, in terms of those outcomes.  Even if direct causation cannot be attributed to the EITI, the evaluation needs to compare with other resource-dependent countries not implementing the EITI, providing a wider context to understand if EITI implementing countries show distinct trends.  

3) Timing.

Although the EITI was launched 8 years ago, it has only been a standard and had an active Secretariat for less than 3 years and an agreed methodology being tested for the first time. Only a number of limited countries have completed validation. 23 countries have published an EITI report, but experience suggests that several reporting cycles are needed before the report reach a sufficient level of quality and comprehensiveness. The sample size is thus smaller than initially apparent and the time to demonstrate impact extremely short. 

4.3        Evaluation methods

In achieving the purpose of this evaluation and in answering the questions posed in section 2 of this document in particular, the evaluation will draw from all relevant sources including evidentiary from stakeholders and documentary from all relevant output produced by, with help from or as a consequence of the EITI.

The evaluator(s) will decide how best to undertake this evaluation and will specify its methods in its workplan.  It is expected that the evaluator will use a combination of quantitative and qualitative methods.

On the quantitative side, the proposed set of Process and Outcome Indicators is intended to be assessed by quantitative methods as much as possible. In particular, big picture indicators will be examined through the data available for all EITI implementing (both compliant and candidate) countries and, ideally, contrast with as many resource-rich countries (as defined by the IMF, see Annex No. 2) not implementing the EITI. To assess attributable outcome indicators the evaluator will examine all validation reports available, reference material written on the EITI both at national and international levels and extensive exchange with stakeholders at national levels, TA providers and other interlocutors. In particular, the evaluator will examine all validation reports for extracting evidence of how discussion of reports and follow-up actions triggered by EITI implementation are impacting the way the extractive sector is governed.

On the qualitative side, the evaluator should closely examine 4 countries (two from compliant countries and two from candidate countries). This examination should focus on answering the question “How the EITI has contributed to this country’s management of its natural resources?”.   To get comparable information, the most effective approach to the case studies would be to standardise the questions used and the focus areas.  In addition the case studies need to specify the "contextual variables" (which are termed ‘big picture indicators’ above) in which the EITI operates to see the influences these have on the how and why of EITI outcomes and impacts.  If possible, questions of subnational resource flows might also be examined.  The evaluator will select the best cost-effective method to conduct this examination. The evaluator will also need to seek interviews with policy-makers and analysts at the international level to assess the general policy impact of the EITI.  It might also be helpful for the evaluator to speak with the International Financial Institutions (including the regional development banks), the credit rating agencies and sovereign debt lenders and major project financers, to assess whether the EITI has had any impact on their policies and approaches. 

In assessing agency effectiveness indicators the evaluator will draw from EITI key managerial documents including work plan, budget, calendar, and internal documents such as “Back to office” and Implementation reports (as relevant), overview of the EITI reports and EITI publications. The Secretariat will also facilitate access to metric-software such as Google analytics and Factiva. To supplement the assessment based on this documentation the Secretariat will be available for interviews and will facilitate interviews with stakeholders, partners organisations and service providers (accounting, auditing, legal counselling) at the international level as required.

Adding to these quantitative methods the evaluator is expected to include other data collection methods such as:

  1. Document review of relevant documentation furnished by the Secretariat (publicly available and if not, provided, in confidence, for review);
  2. In-depth, semi-structured interviews with stakeholders and/or group interviews;
  3. Questionnaires / surveys;
  4. Field visits;
  5. Observation.

The evaluation is combining both an assessment of the EITI results and whether the EITI is fit for purpose (which includes looking at the Board and the Secretariat performance).  The evaluation team will need to work closely with the Secretariat to be able to evaluate the EITI results while at the same time assess its performance. The Evaluation Team would want to consider the issues of impartiality when approaching these 2 distinct areas of the assessment and put in place measures to mitigate any potential risks to impartiality such as having separate members of the team responsible/working on the 2 areas.

5 Main tasks, deliverables and timetable

5.1 Main tasks

The evaluation is envisaged as a process that will consist of a number of clearly defined tasks.

Task 1 – Further understanding of the Terms of Reference

The evaluator will meet with the EITI Secretariat to understand further the ToRs for the evaluation. More specifically, this will include:

  1. To develop a common understanding of the TORs;
  2. To identify and agree upon the sampling method;
  3. To fine-tune the timetable for carrying out the evaluation;
  4. To address any logistical or administrative issues that might need to be resolved during the initial planning phase of the evaluation;
  5. To outline jointly the work plan with key milestones and deliverables.

Task 2 – Workplan

The evaluator will prepare a detailed workplan closely based on this TOR and the proposed set of Process and Outcome Indicators. The workplan is to provide information about the proposed methodology beyond the material presented in these TOR. If the evaluation is to include any surveys, questionnaires, case studies, etc. these tools need to be fully described and annexed to the work plan. It might provide details on the following:

  1. Brief context of EITI. The logic or theory behind the EITI. A description of how the EITI is supposed to work: its objectives, activities, outputs and expected outcomes and interrelationships.
  2. Evaluation purpose and scope. A clear statement of the objective of the evaluation and the main aspects or elements to be examined.
  3. Evaluation methodology. The data collection methods proposed to be employed during the evaluation.
  4. Evaluation criteria. The criteria the evaluation will use to assess performance, and an explanation of where the criteria came from.
  5. Key milestones and deliverables and limitations of the evaluation.

This workplan will be reviewed and approved by the working group for outcome indicators.

Task 3 – Initial review

The evaluator will review all of the relevant documentation prepared by the Secretariat and to conduct a series of interviews with relevant EITI stakeholders and to collect the publicly available information for assessing the set of process and outcome indicators.  At this point, the evaluator will meet by teleconference with the working group to discuss initial process.

Task 4 – Supplementary data collection

The evaluator, in coordination with the Secretariat, will wish to collect and analyse the remaining necessary data to enable them to conclude upon the evaluation questions outlined in the work plan.

Task 5 – “Initial findings” report

The evaluator will first prepare a draft report with its “initial findings” for the consideration of the working group for outcome indicators. The working group will provide comments (focus on correcting errors in data and editorial matters) that the evaluator will process to produce an “Initial findings” report to be presented to the Global Conference in Paris on the 2nd of March 2011.

Task 6 – Paris Conference

The evaluator will present the initial findings of the evaluation in a plenary session (expected to last no longer than 30 mins). Additionally, the evaluator will conduct a special session (expected to be one of the breakout sessions scheduled on day 2 of the conference) to receive feedback from stakeholders. The evaluator will compile this feedback and incorporate it, as applicable, in a final report.

Task 7 – Final Report

The evaluator will produce a final report to the EITI International Board.

5.2 Deliverables

Workplan as described above to be completed prior to implementation. It is important that the plan include the proposed methodology including a) proposed methods, b) proposed sources of data, c) data collection procedures, and include a proposed calendar of activities within the proposed timetable.

Draft evaluation report to allow stakeholder discussion (via the working group for outcome indicators) of the findings and formulation of recommendations. Secretariat comments back to the evaluation team will be submitted as one consolidated response.

Final evaluation report. The final report should include but not necessarily be limited to, the elements outlined below.

  1. Executive summary (maximum 4 pages)
  2. EITI description
  3. Evaluation purpose
  4. Evaluation methodology
  5. Major findings at macro level.
  6. Impact of the EITI.
  7. Results achieved and agency effectiveness.
  8. Lessons learnt and recommendations.
  9. Annexes to include interview list and key documents consulted.

5.3 Timetable

TASK 

TIME (after signing contract) 

Task 1- Further understanding of the Terms of Reference

Weeks 1-2

Task 2 – Workplan

Weeks 3-4

Task 3 – Initial review

Weeks 5-6

Task 4 – Supplementary data collection and field visits (if necessary)

Weeks 6-14

Task 5 – “Initial findings” report

Week 15-Submission drat to WG

Weeks 16-17 Revision of Draft by WG

Week 18 – Finalizing “Initial finding” report

Task 6 –  Paris Conference

Week 19

Task 7 -  Final evaluation report

Weeks 20-21

6 Reference Materials

  • EITI Workplans for 2007-10.
  • EITI Rules, including Validation Guide and Articles of Association.
  • EITI validation reports.
  • EITI reconciliation reports.
  • EITI Business Guide.
  • World Bank Guide on Implementing the EITI, lessons from the field.
  • EITI Newsletters, papers and minutes of Board meetings, reports of Conferences and other key meetings, other policy and update documents produced by Secretariat.
  • Reports and assessments by international and national EITI stakeholders (civil society and others) e.g., Eye on the EITI, and the 2009 EITI Evaluation.
  • Relevant research work and media articles. 
  • EITI website.
  • EITI implementing country work plans and reports.
  • Evaluation’s Terms of Reference.

7 Skills and competencies required

The Evaluator (or members of the evaluation team) will need to be able to demonstrate that they have:

  • Expertise, knowledge and experience of the EITI or similar programs.
  • Technical and financial skills, including knowledge and work on development including transparency and good governance, public finance and financial accountability, multi-stakeholder dialogue, working with civil society and poverty reduction and economic management.
  • Knowledge of the oil, gas and mining sectors or other natural resources sectors.
  • Regional and country knowledge: a demonstrable track record in similar work in regions and countries where the EITI is implemented.

7.1 Credibility and independence

The evaluator needs to be credible in the eyes of the target audience. The Evaluator needs to divulge any prior involvement with the EITI, directly or indirectly, so that potential conflicts of interest may be assessed and ways to mitigate these devised. At least one member of the evaluation team — generally the team leader — should be selected who is judged to be able to provide objective, unbiased evaluation.

7.2 Proposal

Suitably qualified service providers should submit an expression of interest outlining how they meet the above requirements. The expression of interest should also demonstrate:

  • Economic and financial capacity, stating the average annual turnover of the organisation for the last three years (2007 to 2009).
  • Professional capacity in terms of in-house staff. To this end, the candidate has to submit:
    • A list of permanent in-house experts
    • A list of temporary experts the candidate can provide
    • CVs for the above experts
  • Technical capacity of candidate by including a list of at least 6 reference projects. For each reference, the list must indicate the link or links with the fields covered by the EITI. Ideally, at least 50 % of the references must be for projects carried out in the current EITI implementing countries or other resource-rich countries.
  • The application should include a summary (no longer than 500 words) summarising the applicant’s experience and expertise.

Applicants should specify a contact person for the application, including email, phone and postal address. 

8 Submission of applications

Applications must be submitted in English exclusively to the EITI International Secretariat by email and official postal service. The electronic application must be submitted by 17:00 CET 29 September 2010. The Secretariat will confirm receipt of all applications. Applications lodged after this date will not be accepted.

By email to:

fparis@eiti.org

By official postal service:

Evaluator Applications
Attention: Francisco Paris
EITI International Secretariat
Ruseløkkveien 26
0251 Oslo
Norway

Selection Criteria

Tenders for this contract will be assessed in accordance with good commercial practice, taking into account the consultant’s relevant experience for the assignment and the qualifications of the key staff proposed.

9 Process and Outcome Indicators

The Extractive Industries Transparency Initiative (EITI) is an international standard for transparency in natural resource revenue management.   The initiative was launched in 2002 but the indicators for meeting the standard were not established until 2006.  An evaluation was prepared for the 4th Global Conference in February 2009 that assessed EITI’s contribution to good governance of natural resources.  EITI implementation has gathered considerable pace since 2009. 31 countries are now implementing, 20-plus validation processes are completed or underway and 47 EITI reports have been published.  There is now richer evidence and data to evaluate the performance of the EITI and its impact on key development outcomes.

The Board has established a working group to develop a set of process and outcome indicators (set of indicators, henceforth). The purpose of this set of indicators will be to provide the EITI with better means for learning from experience, improving delivery, planning, governance structure, and allocation of resources, and demonstrating results. This set of indicators, which once endorsed by the Board will provide the basis for on-going monitoring of the EITI and for an independent evaluation on its impact and effectiveness.

9.1 Methodology

The working group suggests using a multi-tier framework for this evaluation. This framework has been adapted from the one used by the Results Unit of the Operations Policy and Country Services within the World Bank. 

Figure 1 summarizes this approach.

”Big picture” indicators provide context, establish benchmarks and indicate directional change. These indicators are not directly attributable to any single project or organization since many such activities and efforts have to come together to achieve these development outcomes. For better understanding the wider benefits of the EITI, the evaluation will look at the proposed set of the “Big picture” indicators with the purpose of:

  • Providing a general outlook of EITI countries in terms of key development outcomes.
  • Establishing a general context and directional change.
  • Establishing benchmarks to allow monitoring and comparing of these outcomes against other resource-rich countries not implementing the EITI.

For assessing the performance of the EITI and its results, the evaluation will look at the proposed set of ”Attributable outcome” indicators that answer the question ”Because of this activity, project or initiative these outcomes have been achieved” and ”Agency Effectiveness” indicators that measure inputs and outputs to assess the organizational effectiveness and efficiency, with the purpose of:

  • Measuring input to the EITI.
  • Measuring output of the EITI.
  • Assessing the quality of the output of the EITI in terms of specific yardsticks such a level of coverage in the reports, dissemination, multi-stakeholder dialogue, impact on governance reforms, improved understanding of the sector, etc.
  • Assessing the effectiveness and appropriateness of EITI’s policies (e.g. Rule Book) and guidance documents.
  • Assessing the effectiveness of EITI management (management tools, e.g. work plan, and stakeholder relations)
  • Assessing the effectiveness of EITI communication tools (Website, newsletters, materials, etc)
  • Assessing if the EITI structure, especially if the International Secretariat is fit for purpose and provides value for money.

9.2 Proposed set of indicators

Following the proposed methodology the working group examined:

  • A long list of available “Big picture” indicators produced by a wide range of internationally renowned organizations that address issues and development outcomes directly related to the EITI principles and goals;
  • The EITI International Secretariat key performance indicators approved by the Board as part of the Workplan submitted each year;
  • EITI reconciliation and validation reports and the EITI validation grid; and
  • Abundant reference material including the EITI Evaluation presented in Doha in February 2009.

After careful examination and deliberation the working group selected indicators or sources from which indicators can be extracted for each tier.

These are:

9.2.1  Big picture indicators

(Some of these indicators might not be available for each country -both EITI and other resource-rich countries-)

  1. Disclosure index measure from the World Bank’s Doing Business Report.
  2. Credit ratings (available from leading credit rating agencies)
  3. From the World Bank’s Country Policy and Institutional assessment  (CPIA):
    1. Macroeconomic management rating.
    2. Equity of public resource use rating.
    3. Transparency, accountability and corruption in the public sector rating.
  4. Transparency International’s Corruption Perception Index.
  5. UNDP Human Development Index (http://hdr.undp.org/en/statistics/indices/hdi/)
  6. UN GINI coefficient
  7. GDP growth (from World Bank national accounts data, and OECD National Accounts data files).
  8. From the Global Integrity Indexes (http://www.globalintegrity.org/):
    1. Civil society organizations.
    2. Public access to information.
    3. Government accountability.
    4. Overall country score.
  9. Open Budget Index (http://www.openbudgetindex.org/)
  10. Freedom in the World Report by Freedom House http://www.freedomhouse.org/template.cfm?page=15

Note:  There are other sources of information from which qualitative assessments can be made such as debt relief(see IMF/WB Heavily Indebted Poor Countries (HPIC) at http://www.imf.org/external/np/hipc/index.asp), human rights (see reports from the US State Department or organisations such as Amnesty International or Human Right Watch), freedom of the press (see Press Freedom Index by Reporters without borders -http://en.rsf.org/press-freedom-index-2009,1001.html-) and governability (The Failed States index published by Foreign Policy/Fund for peace - http://www.fundforpeace.org/web/index.php-)

9.2.2  Attributable outcome indicators

  1. No. of compliant countries
  2. No. of candidate countries
  3. No. of supporting companies
  4. No. of supporting investors
  5. No. of supporting countries
  6. No. of completed validations
  7. Communication and awareness raising
    1. Users of EITI website
    2. Articles published about the EITI
    3. References of EITI in articles, news items and blogs.
    4. Subscribers to EITI ’s newsletter
  8. Reporting
    1. No of reports (including disaggregated reports)
    2. Sector coverage (percentage)
    3. Regularity in EITI disclosure
    4. Companies participation

9.2.3   Agency effectiveness indicators

These indicators relate to activities developed by the Secretariat, the Board (with support from partner organisations) and the input (monetary, time, quantity of publications) put into these activities. Each indicator is crossed-referenced with the 2010 Secretariat workplan action list (WP).

Inputs:

  1. Resources allocated for missions and support to implementing countries (ref. WP 1).
  2. Staff resources allocated to validation (ref. WP 4, 5, 6).
  3. Resources allocated for missions to outreach countries (ref. WP 7,8)
  4. No. of Board meetings and resources allocated to Board meetings and Chairman’s support (ref. WP 28, 29).
  5. Resources allocated to relations with stakeholders including: conference, supporters’ roundtables and National Coordinators meeting (ref. WP 2, 9, 10, 13, 31).
  6. Resources allocated to relations with supporting companies and investors (ref. WP 11, 12, 32).
  7. Resources allocated to communication (ref. WP 14, 15, 16, 17, 20, 21, 22, 23).
  8. Resources for training, including InWent seminars (ref. WP 3).
  9. Resources allocated to governance, management and administration (ref. WP 30, 33, 34).

Outputs:

  1. Publications, including website, notes and reports (ref. WP 14, 18, 19, 24, 25, 26, 27).
  2. Validation reports reviewed (ref. WP 4, 5, 6).
  3. Reconciliation reports reviewed
  4. Meetings organised (including roundtable, Board and side meetings, national coordinators meeting and other conferences) (ref. WP 2, 9, 10, 13, 28, 29,31).
  5. Number of people trained (ref. WP 3).
  6. Number of countries visited (ref. WP 1, 7, 8).

9.3 Sources

Big picture 1 indicators are available mostly through the websites and publications of each of the organisations responsible for producing data and rankings about various development and institutional outcomes. Attributable outcome indicators and Agency effectiveness indicators can be assessed based on the EITI Secretariat key managerial documents and tools, notably the work plans, budget, financial and auditing reviews and calendar; publications including its website and the EITI reconciliation and validation reports.

 

If you have any questions relating to this invitation to apply for evaluation, please contact Eddie Rich (erich@eiti.org) or Francisco Paris (fparis@eiti.org) at the EITI International Secretariat.


UPDATE, 13 September 2010

Please take note of the following clarifications:

  1. The EITI is expecting interested bidders to submit a proposal before the 29 of September. The reference to “expression of interest” in the announcement published in the website on 25 August 2010 should be disregarded.
  2. The mentioned proposal should contain:
    1. Economic and financial capacity, stating the average annual turnover of the organisation for the last three years (2007 to 2009).
    2. Professional capacity in terms of in-house staff. To this end, the candidate has to submit:
      1. A list of permanent in-house experts (in the entire organization)
      2. A list of temporary experts the candidate can provide to work on this assignment.
      3. CVs for the above experts (available for this assignment)
    3. Technical capacity of the proposed team by including a list of at least 6 reference projects in which members of the team have worked. For each reference, the list must indicate the link or links with the fields covered by the EITI. Ideally, at least 50 % of the references must be for projects carried out in the current EITI implementing countries or other resource-rich countries.
    4. A summary (no longer than 500 words) summarizing the applicant’s experience and expertise.
    5. A financial proposal clearly stating your estimate on budget/cost.
  3. Note: The electronic application must be submitted by 17:00 CET 29 September 2010 (If applicants wish to submit copies in print this could reach the International Secretariat on a later date)