International Evaluation Expert (Consultant)

Background and context

The United Nations Development Programme (UNDP) Afghanistan Country Office’s Peace Pillar with financial contributions from the Royal Danish Embassy in Afghanistan commenced the Afghanistan Anti-Corruption, Transparency, Integrity and Openness (ACTION) Project on 07 August 2019 with the current end date on 31 December 2024 to allow more time to complete the planned activities.

ACTION project was a continued joint effort to support both institutional (supply) and civilian engagement (demand) sides to increase public trust in and transparency of Afghan security and justice institutions. On the government institutions’ side, under the overall guidance from the Afghanistan Attorney General’s Office (AGO), the project focused its support to the flagship initiative of the government to combat corruption, namely the Anti-Corruption Justice Center (ACJC) and newly established Anti-Corruption Commission (ACC) by enhancing their logistical and technical capacity to address high-level corruption cases. On the demand side, the project is working with civil society organizations and media to improve public awareness on anti-corruption and how to hold service provider accountable for better service delivery. The project started with three outputs and in July 2021, two additional outputs were added as mentioned below:

Output 1: The Anti-Corruption Justice Center Improved Technical and Operational Capacities to Effectively Adjudicate Corruption Cases

Output 2: Non-State Actors and Community Groups, including women and youth, can better promote Transparency, Accountability and Integrity in the Security and Justice Sectors.

Output 3: COVID-19 response’s Transparency and Accountability is strengthened through citizen monitoring

New Output 4: The AGO has improved internal integrity, more efficient processes, and is better able to serve the public

New Output 5: The ACC is strengthened to provide more effective oversight of justice and security institutions for increased prevention of corruption and enhanced service provision.

The Royal Danish Embassy supported the initial three outputs, and the rest were funded by UNDP’s TRAC and other resources.

Under Output 1, ACTION project implemented the capacity building activities for ACJC by delivering 28 training modules identified through a comprehensive training needs assessment. During 2020 and 2021, 110 ACJC judges, prosecutors and technical staff including 15 females were trained. For enabling better working environment for ACJC, the ACTION project also provided resource support to the ACJC. The items of logistics and equipment support to ACJC have been delivered which included new office space, security cameras, kindergarten with equipment, printers, clinic items, sim cards for improved their efficiency and work productivity.

Under output 2, supporting the demand side (civic engagement) ACTION project successfully implemented 27 small grants projects (22 projects Denmark fund and 5 projects UNDP TRAC fund) to build trust between the Afghan community and the state through mobilizing local community on how to fight corruption, conducting community based monitoring of health facilities, social audit of service delivery, reporting corruption cases by media and establishing community based volunteer network to represent community in identifying and resolving corruption cases through active coordination with local and provincial authorities. Through these grants, communities have been mobilized and awareness have been generated on anti-corruption activities by providing information on access to information law. Under investitive journalism 22 local journalists were trained and they produced 55 investigative reports on different corruption cases.

Under Output 3, The project is working in 34 Provinces across 1500 (Primary Health Center) PHCs to monitor the health services through community members related to the support provided by government and other donors to address COVID-19. ACTION M&E COVID-19 app has been developed to create a dashboard by feeding all these data live collected through the partners.

No activity under output 4&5 have been initiated yet after the approval of the revised project document in August 2021.

Due to the recent political changes in the country (the government takeover by the Taliban) and because there are no government partners available anymore to support like the ACJC, the Attorney General’s Office (AGO), as per the instructions by United Nations, the ACTION project will not continue work on activities under output 1, 4 and 5 which are relevant to the government institutions. Out of the above three outputs, the Royal Danish Embassy in Afghanistan supported output 1.

ACTION project will continue the activities under output 2 and output 3 on the demand side engaging with civil society organizations, media, academia, and the private sector.

The Anti-Corruption, Transparency, Integrity and Openness (ACTION) project has been implemented since July 2019. It was originally envisaged to end on 31 December 2020 but was extended through a project revision until December 2021. The project was subsequently revised and extended until 31 December 2024.

Basic Project information can also be included in table format as follows:

PROJECT/OUTCOME INFORMATION

Project/outcome title

The Anti-Corruption, Transparency, Integrity and Openness (ACTION)

Atlas ID

00120040

Corporate outcome and output

Outcome 2: Accelerate structural transformation for sustainable development

Country

Afghanistan

Region

Asia

Date project document signed

01 July 2019

Project dates

Start

Planned end

01 July 2019

31 December 2024

Project budget

US$ 7,025,000

Project expenditure at the time of evaluation

US$ 6,075,000

Funding source

Denmark

Implementing party

UNDP/ACTION team (Direct Implementation Modality)

Duties and Responsibilities

The Purpose and Objectives of the Midterm Evaluation

The Midterm Evaluation (MTE) aims to inform UNDP Afghanistan and its partners of lessons learned, results achieved and areas for improvements. The MTR will draw out progress toward project deliverables, identify gaps in programming, and any course correction required for the future programming. Furthermore, the findings of MTR will inform the future designing of UDNP’s work. The MTR will be able to produce valuable lessons and experiences, providing useful findings to the other relevant projects and various initiatives organized by UNDP Afghanistan as well as Country Offices (COs) globally. Also, MTR will be able to produce a report on the achievements of the ACTION project plus success stories on the project, which will be published on UNDP website.

Responding to the Theory of Change (ToC) as described in the project document, the agreed results, and resources framework (RRF) and the approved workplans, the MTR should look at the relevance of the project, quality of the project design, effectiveness, and efficiency of the implementation to date, sustainability of the overall project results, impact of intervention made to date, and forward-looking directions for future. To meet these ends, MTR will serve to:

assess project performance and progress against the expected outputs, targets including indicators presented in the RRF and contribution to expected outcome.
review and document the success and draw out lessons for deepening impact
assess the effectiveness of the project’s engagement with UNDP ACTION project implementing partners including civil society organizations (CSOs).
identify challenges and the effectiveness of the strategic approaches that project adopted for addressing those challenges.
ascertain the relevance, effectiveness, efficiency, and sustainability of the project interventions.
outline recommendations, including potential realignments in scope and approach in line with the project’s desired outcome
provide forward looking recommendations to inform the future designing of UNDP’s work on Anti-corruption area.
report on the achievements of the ACTION project plus success stories on the project, which will be published on UNDP website.
highlight project’s overall accountability within the corporate framework, e.g., contribution to higher level results, such as Strategic Plan, CPD etc.

The Scope of the MTR

The MTR is expected to assess the ACTION project progress against the Project Document, targets stipulated in the RRF and the achieved results from 1 July 2019 to 31 December 2022 and propose recommendation which will inform and help improving the coming implementation of the project and designing any future projects. The MTR will be based on a desk review of project related documents and in-depth virtual interviews and surveys as outlined in the methodology section. The MTR will also intend to document achievements, good practices, success stories, lessons learned or transferable examples. Based on the achievements to the date, the MTR will provide forward looking programmatic recommendations for the project’s next months, using the OECD/DAC evaluation criteria on (a) relevance; (b) effectiveness; (c) efficiency; and (d) sustainability

and cross-cutting issues including gender equality.

The MTR’s geographical coverage includes the project’s targeted provinces of Afghanistan. Will be provided to the consultants during the implementation stage.

Evaluation criteria and key guiding questions

In responding to the MTR purpose and objectives, the MTR criteria and guiding questions can be outline below:

Table 2 - Criteria and Guiding Questions

Criteria

Guiding Questions

Relevance

Relevance of the project: review the progress against project outputs and contribution to outcome level results as defined in the project’s ToC whether assumptions and risks remain valid. Identify any other intended or unintended, positive, or negative, results using following guiding questions.

To what extent was the project in line with the regional development priorities and UNDP strategic Plan.
To what extent does the project contribute to the ToC for the relevant regional programme outcomes?
To what extent were the project activities in line with the national development priorities and country development programme’s outputs and outcomes?
To what extent the overall design and approaches of the project were relevant?
To what extent, the inputs and strategies identified were realistic, appropriate, and adequate to achieve the results?
To what extent did the project achieve its overall outputs? Are the project’s contributions to outcomes clear?
To what extent does the project contribute to gender equality and women’s empowerment?
To assess whether the results achieved had a differentiated impact on women and other vulnerable groups?
To what extend has the project been appropriately responsive to COVID-19 pandemic as well as other political, legal, economic, institutional changes in the country?

Effectiveness

Effectiveness of implementation approaches: review project’s technical as well as operational approaches, the regionality and deliverables, quality of results and their impact, alignment with national priorities and responding to the needs of the CSOs; covering the results achieved, the partnerships established, as well as issues of capacity using following guiding questions.

To what extent the project activities were delivered effectively in terms of quality, quantity, and timing?
How effective were the strategies used in the implementation of the project?
To what extent the project was effective in enhancing the capacity of CSOs?
What are the key internal and external factors (success & failure factors) that have contributed, affected, or impeded the achievements, and how UNDP and the partners have managed these factors?
In which areas does the project have the greatest achievements? Why and what have been the supporting factors? How can the project build on or expand these achievements?
In which areas does the project have the fewest achievements? What have been the constraining factors and why? How can or could they be overcome?
To what extent have UNDP partners been involved in project implementation? To what extend are project management and implementation participatory?
To what extent the project is align to the corporate context such as contribution to SDGs, UNDP Strategic Plan and Country Programme Document etc.
How will the evaluation fit within the context of other ongoing and previous evaluations and the evaluation cycle? What synergies have been created?

Efficiency

Efficiency of the project management structure and the added value of the project’s regional approach: review planning, management, monitoring and quality assurance mechanisms for the delivery of the project interventions and the added value of the regionality of the project set up in the context of fiscal reform at national and subnational level using following questions.

To what extent is the existing project management structure appropriate and efficient in generating the expected results?
Have resources (funds, human resources, time, expertise, etc.) been allocated strategically to achieve outcomes?
Was the process of achieving results efficient? Were the resources effectively utilized?
Did the project activities overlap, and duplicate other similar interventions funded nationally, and/or by other donors?
To what extent did the project produce synergies within UNDP and with other development partners and play complementary roles each other?
How does the project align with other regional and national level initiatives/activities on Anti-corruption? How efficiently are national and regional activities connected and complement each other?

Sustainability

Sustainability of the project results and risks along with opportunities related to future interventions: review and assess if the current project setup has plans for future resource mobilization, synergy, long term partnership and / or considering institutionalization of the project impact for continued support after the project end using following questions.

To what extent did the project bring momentum in the country for implementation of the Anti-corruption?
What is the likelihood of the continuation and sustainability of national level dialogues engaging various UNDP implementing partners and strengthening national and regional partnership architectures, made up of UN system, CSOs, and private sector actors working on Anti-corruption?
How were capacities of a various set of ACTION implementing partners strengthened at the national level through regional peer-learning and other training programs?
Describe key factors that will require attention to improve the prospects of sustainability of Project outcomes and the potential for replication of the approach?
To what extent do UNDP implementing partners support the project’s long-term objectives?
To what extent will financial and economic resources as well as political wills be available to sustain the benefits achieved by the project?
Are there any social or political risks that may jeopardize sustainability of project outputs and the project’s contributions to country programme outputs and outcomes?

Diversity and inclusion

Sustainability and effectiveness of diversity and inclusion approach: review the project’s approaches and strategies in integrating gender and social inclusion (GSI) in the countries, using following questions.

To what extent has the project promoted positive changes in gender equality and the empowerment of women? Were there any unintended effects? How can the project further broaden in a future phase its contribution to enhancing diversity and inclusion?
To what extent have local communities, women, youth, people with disabilities and other disadvantaged groups benefited from the project, anti-corruption?

Principled

Has the project applied Social and Environmental Screening checklist of UNDP?

Management and Monitoring

Has the project monitored the results of progress against indicators?
Has the project established a strong M&E standard?

Methodology

The TOR may suggest an overall approach and method for conducting the evaluation, as well as data sources and tools that will likely yield the most reliable and valid answers to the evaluation questions within the limits of resources. However, final decisions about the specific design and methods for the evaluation should emerge from consultations among the UNDP ACTION project team the evaluators about what is appropriate and feasible to meet the evaluation purpose and objectives and answer the evaluation questions, given limitations of budget, time and data.

Methodological approaches may include some or all of the following:

Evaluation should employ a combination of both qualitative and quantitative evaluation methods and instruments.
Document review of all relevant documentation. This would include a review of inter alia

Project document (contribution agreement).
Theory of change and results framework.
Programme and project quality assurance reports.
Annual workplans.
Activity designs.
Consolidated quarterly and annual reports.
Results-oriented monitoring report.
Highlights of project board meetings.
Technical/financial monitoring reports.

Semi-structured interviews with key representatives of key civil society organizations, and implementing partners:

Development of evaluation questions around relevance, effectiveness, efficiency, and sustainability and designed for different UNDP ACTION project implementing partners to be interviewed.
Key informant and focus group discussions with men and women, beneficiaries of UNDP ACTION project implementing partners.
All interviews should be undertaken in full confidence and anonymity. The final evaluation report should not assign specific comments to individuals.

Surveys and questionnaires including participants in development programmes, surveys and questionnaires involving UNDP ACTION project implementing partners at strategic and programmatic levels.
Field visits and on-site validation of key tangible outputs and interventions.
The evaluator is expected to follow a participatory and consultative approach that ensures close engagement with the evaluation managers, implementing partners and direct beneficiaries.
Other methods such as outcome mapping, observational visits, group discussions, etc.
Data review and analysis of monitoring and other data sources and methods.
Ensure maximum validity, reliability of data (quality) and promote use; the evaluation team will ensure triangulation of the various data sources.

As part of the requirement, evaluation must include an assessment of the extent to which the design, implementation, and results of the project have incorporated gender equality perspective and rights-based approach. The evaluators will make sure Integrating Human Rights and Gender Equality in Evaluation during the inception phase. In addition, the methodology used in the evaluation, including data collection and analysis methods should be human rights and gender-sensitive to the greatest extent possible, with evaluation data and findings disaggregated by sex, ethnicity, age, etc. – with a focus on people with disabilities. Detailed analysis on disaggregated data will be undertaken as part of final evaluation from which findings are consolidated to make recommendations and identify lessons learned for enhanced gender responsive and rights-based approach of the project. These evaluation approach and methodology should consider different types of groups in the project intervention – women, youth, vulnerable groups etc.

The final methodological approach including interview schedule, field visits and data to be used in the evaluation should be clearly outlined in the inception report and be fully discussed and agreed between UNDP and the evaluators.

Evaluation products (deliverables)

The terms of reference should clearly outline the outputs UNDP expects from the evaluation team as well as a detailed timeline and schedule for completion evaluation products. Where relevant, the TOR should also detail the length of specific products (pages). These products could include:

Evaluation inception report (10-15 pages). The inception report should be carried out following and based on preliminary discussions with UNDP after the desk review and should be produced before the evaluation starts (before any formal evaluation interviews, survey distribution or field visits) and prior to the country visit in the case of international evaluators.
Evaluation debriefings. Immediately following an evaluation, UNDP may ask for a preliminary debriefing and findings.
Draft evaluation report (within an agreed length). The ACTION project team should review the draft evaluation report and provide an amalgamated set of comments to the evaluator within an agreed period, addressing the content required (as agreed in the TOR and inception report) and quality criteria as outlined in these guidelines.
Evaluation report audit trail. Comments and changes by the evaluator in response to the draft report should be retained by the evaluator to show how they have addressed comments.
Final evaluation report, reflecting the achievements and success stories of the project.
Presentations to NDP ACTION project team

Competencies

Competencies:

Functional Competencies:

Ability to work independently.
Ability to perform tasks in a timely manner and produce quality final product.
Strong interpersonal, communication and diplomacy skills.
Openness to change and ability to receive and integrate feedback.

Corporate Competencies:

Demonstrates integrity by modelling the UN’s values and ethical standards.
Promotes the vision, mission, and strategic goals of UNDP.
Displays cultural, gender, religion, race, nationality and age sensitivity and adaptability.
Treats all people fairly without favouritism.
Fulfils all obligations to gender sensitivity.

Responsibilities of the MTR consultant/s:

Review the relevant documents.
Develop and submit a draft and final inception report
Conduct evaluation.
Maintain ethical considerations.
Develop and submit a draft evaluation report
Organize meeting/consultation to discuss the draft report
Incorporate inputs and feedback in draft report
Submit final report with due consideration of quality and effectiveness
Organize sharing of final evaluation report
Evaluator is expected to work within Afghanistan- Kabul working hours, particularly for the interviews.

Required Skills and Experience

I. Academic Qualifications:

A minimum of a master’s degree or equivalent in law, political science, development studies, history, or other relevant social science.

II. Years of experience:

At least 7 years of professional experience in the provision of policy, analytical, and technical advisory support for international development organization.
At least 5 years of proven experience in development, risk assessment, and/or evaluation of programmes or projects in Anti-Corruption, democratic governance, rule of law, and/or development.
Experience in the result-based management, evaluation methodologies and programme/project monitoring approaches with development partners
The project mid-term review/evaluation experience with UNDP is highly desired.
Sound understanding of the UN system and of UNDP’s mandate and role.

III. Language:

Excellent knowledge, both oral and written, of English with presentational capacities is required