48450-001 - Selected Evaluation Studies for 2015: Technical Assistance

Description

The research and development technical assistance (TA) supports the 2015-2017 program cycle, particularly for products due from 2016. It will fund a set of evaluations from the operational (e.g., country program evaluations or validations); and knowledge (i.e., thematic and corporate evaluations, impact evaluations, topical working papers) programs of the Independent Evaluation Department (IED). The TA will also support stepped up knowledge sharing and outreach to maximize usefulness and impact of such evaluations. 1. Country assistance program evaluations and validations of country partnership strategy final reviews. This TA will fund at least one CAPE (Sri Lanka) and five validations (e.g., Kazakhstan, Mongolia, Philippines, Solomon Islands, Uzbekistan). These will evaluate ADB country support and cover ADB strategy, policy dialogue, completed and ongoing projects and TA, and ADB performance. They will provide ADB and its DMCs, with an evaluative basis for future CPSs. Sector assessments will serve as inputs to assessing ADB performance in a given sector, focusing on strategy, policy dialogue, ongoing and completed projects, and TA performance. The TA will also support validations of completed project/program completion reports to inform the larger country and sector assessments. 2. Effectiveness of Safeguards Implementation Phase 2. The next phase of IED's safeguards assessment is to draw further insights (results and costs) from an in-depth field research of a purposive sample of ADB supported projects. IED will also examine the on the ground effectiveness of ADB safeguards. The evaluation will examine the extent to which: (i) any potentially adverse impacts of ADB projects have been (or will likely be) avoided where possible or appropriately mitigated/minimized where avoidance has not been possible; and (ii) ADB safeguard efforts have brought about (or are likely to bring about) positive development outcomes. The study will examine the cost and benefit aspects of safeguards, look at category A and B projects, and some financial intermediary projects. Projects will be purposively selected in East Asia, Central and West Asia, South, and Southeast Asia (possibly, P.R. China, Kazakhstan, Cambodia, Sri Lanka which have more projects approved under the 2009 Safeguard Policy Statement (SPS) that are relatively advanced in implementation) regions. The study will adopt a mix-methods framework and will be informed by documentation analysis, key informant interviews, pilot economic analysis, and site inspections. 3. Review of Sustainable Transport Initiatives (STI). The thematic evaluation study (TES) will examine the relevance, responsiveness, and effectiveness of ADB support to DMCs under its STI Operational Plan. It will assess ADB support in sustainable transport infrastructure and services. The TES will assess the implementation of the Operational Plan and identify good practices and draw lessons for future operations. It will cover sustainable road transport (urban and non-urban), traffic management, transport management and policies, and rail transport. The study will cover at least two regions and will focus on the (i) extent to which CPSs aimed to help the sustainability of transport infrastructure and services; (ii) extent to which countries implemented policies and regulatory framework, financial mechanisms, and capacity building activities for sustainable transport initiatives, (iii) extent to which these measures have been effective, and (iv) determinants of their success. The TES will conduct a desk review of operational documents for lending and non lending support to the sector from 2009 to 2014. Perception surveys, country case studies of selected sustainable transport projects, and key informant interviews of DMC stakeholders and development partners undertaking related work in the sector will also be conducted. 4. Additionality of Private Sector Operations. The corporate evaluation study (CES) will concentrate on the additionality provided by ADB in conjunction with its non-sovereign loans and equity investments approved since 2000. It will focus in particular on: (i) how ADB defines and encourages value addition in its PSO based on a review of pertinent corporate strategies, institutional incentives and project approval documents; (ii) to what extent it has achieved value addition based on a review of completed transactions, available evaluation reports, and feedback from project sponsors, financiers and other relevant stakeholders; and (iii) whether ADB could do more to optimize its financial and non-financial value addition in non-sovereign operations (NSO) based on feedback from private sector sponsors and financiers and a review of value addition achieved by similar multilateral financial institutions. 5. Impact evaluation. The TA will support at least one impact evaluation study (IES). The IES will look at the development outcomes of select urban water supply projects in a country (possibly Viet Nam or India), depending on the availability of a well-structured and well-conducted household survey, and provide evidence of the poverty impacts of water supply interventions. The IES will use impact evaluation methods based on statistical and econometric analyses of quantitative data from household and community level surveys, both existing and to be conducted by the study. The counterfactuals will be drawn from comparable non-participants areas. For additional insights on non-quantifiable variables and study robustness, it will collect qualitative information and integrate it into the quantitative analyses. 6. Evaluation research and topical working papers. The TA will support sector and/or thematic assessments that feed into or help disseminate higher-level evaluations in IED s work program. These include selective ad hoc evaluative researches (or quick turnaround topical working papers) on high impact topics with strong potential implications for ADB s institutional policies and practices or of specific interest and relevance to DMCs. 7. Evaluation outreach, knowledge sharing, and partnerships. IED continues to work to make its evaluation results more accessible and influential. It will disclose, disseminate, and share evaluation feedback through: (i) a soundly managed publicly available lessons database in its Evaluation Information System, (ii) outreach activities like briefings, presentations, and interactive learning events (i.e., training, seminars, and conferences) on completed major evaluations; (iii) preparation and publishing of syntheses, briefs, and occasional papers (designed to widen existing readership); and (iv) new mediums for knowledge sharing like online readers and tablets (to extend the reach of IED products).
Project Rationale and Linkage to Country/Regional Strategy

The Board of Directors of the Asian Development Bank (ADB) approved the 2015 -2017 work program of IED, with effect from 11 December 2014. This yearly work program focuses on high-level evaluations that complement project- level evaluations, and gives a strategic perspective on ADB operations and direction. High -level evaluations give feedback on the effectiveness of ADB strategies, policies, processes and operations to its Management and Board of Directors. This feedback needs to be more accessible and influential to sustain sharing and use of evaluation knowledge generated. 1. Country assistance evaluations. The TA will fund CAPEs and validations of CPS final reviews. These will provide an objective assessment and rating of ADB support, and identify lessons and recommendations of use to new CPSs. 2. Effective implementation of safeguards. The CES on Safeguards Operational Review: ADB Processes, Portfolio, Country Systems, and Financial Intermediaries was discussed by the Development Effectiveness Committee (DEC) on 7 November 2014. It was limited in scope but drew wide attention given the recently started consultation process regarding the update of the World Bank safeguards policy, and perhaps the entry of new 2 new development banks on the scene, the Asian Infrastructure Investment Bank and the Brazil-Russia-India-China-South Africa (BRICS) bank. Independent Evaluation will build on this operational review to conduct a more results oriented assessment of the effectiveness of ADB s safeguard implementation, with special emphasis on the effectiveness of the changes introduced by the SPS. The CES will do extensive field work to check results of safeguard implementation on the ground. The evaluation is mandated by the SPS and is to take place 5 years upon the start of SPS. Due to the fact that not many projects will have been completed that were started under SPS, the evaluation will take a broader view of safeguard implementation and include case studies of projects started before SPS, but implemented at a time of much higher numbers of safeguard specialists in ADB and greater safeguard experience. 3. Promoting sustainable transport. The STI Operational Plan (2010) aims to steer ADB transport sector operations and align these with Strategy 2020 by adopting sustainable transport as a guiding principle. The TES will look at the design of the STI Operational Plan and the extent to which it has been operationalized and implemented. It will attempt to answer the questions how effectively ADB has integrated transport sector road maps in country partnerships and strategies (CPS) and how effectively has it used these road maps in designing sustainable transport operations? The evaluation will also assess the extent to which the transport lending pipeline shifted to accommodate the major changes proposed by the STI Operational Plan. Initial results will be checked, notably the extent to which the new transport projects have achieved their intended outputs, outcomes, and impacts. Factors hindering the plan or facilitating it will also be assessed notably good governance and legal and regulatory frameworks. How can the STI be strengthened to enhance development effectiveness? 4. Ensuring additionality of private sector operations. Strategy 2020 has been promoting a significant expansion of private sector operations (PSO) to generate greater economic growth in the Asia and Pacific region. ADB is to assume greater risks and act as a catalyst for investments that the private sector might not be willing to make. The CES will assess the value addition provided by ADB in its PSO in terms of providing funding or catalyzing other funding, and improving the risk profile, design, functioning, and development impact of supported investments. 5. Impact evaluation. IED is committed to do at least one impact evaluation per year to better understand and assess development effectiveness in select priority areas of ADB operations. This TA will support an evaluation of the impact of urban water supply and sanitation. After more than a decade in pursuit of the Millennium Development Goals, there have been gains in reducing poverty as well as in improving health and access to clean water. The achievement of the global target in terms of access to clean water was 89% in 2012, and 70% in terms of the access to sanitation. Yet these figures mask significant differences in the access within countries and areas and also in the quality of services provided. The impact evaluation will provide an indepth assessment of the various impacts of projects on the access of different groups to the services provided and on the health and well-being of the populations served. 6. Strengthening evaluation usage. IED s knowledge sharing, outreach, and partnerships are anchored on a (i) a good outreach program in place, (ii) use of participatory communication channels and new communication approaches for learning, such as interactive knowledge-sharing events to communicate evaluation findings); and (iii) a responsive database for evaluation information.
Impact

Improved development effectiveness of ADB operations, policies, strategies, and business processes
Project Outcome
Description of Outcome

Increased use of evaluation findings, lessons, and recommendations by the ADB Board of Directors and its Development Effectiveness Committee, by ADB Management, and operations staff, and also by DMC agencies and practitioners, and other interested parties in the general public
Implementation Progress
Description of Project Outputs

Practical lessons and recommendations (country, sector/theme, corporate) provided through completed evaluations Knowledge sharing and learning events Derivative products from the learning events Dissemination of evaluation products Management of evaluation information
Stakeholder Communication, Participation and Consultation
During Project Design

Selected Evaluation Studies. The evaluation process will be guided by an approved evaluation approach paper (EAP) (or in some specific cases a concept paper) which defines the intended study, i.e., key evaluation issues and questions, methods and approach, inputs and activities. The formulation of the EAP will involve interactions between IED staff and ADB operations (including interdepartmental review or commenting on the draft paper) and non-operations staff, representatives from the international evaluation community, non-government organizations, and, where appropriate, consultations with relevant study countries. Evaluation Outreach, Knowledge Sharing, and Partnership Initiatives. With assistance from ADB's in-country Events Management Coordinator (national consultant), the host or collaborating partner in the DMC will help organize the planned knowledge sharing and learning event for policymakers, decision makers, and associated stakeholders.
During Project Implementation

These will include a mixed set of (i) country visits to meet beneficiaries and affectees (including vulnerable groups), (ii) in-country meetings with DMC government staff, non-government organizations, and development partners, (iii) consultations with operations and non-operations staff at ADB headquarters and resident missions, (iv) targeted field interviews/surveys, and (v) video conferences with relevant evaluation units, international think tanks etc. Evaluation Outreach, Knowledge Sharing, and Partnership Initiatives. With help from ADB's in-country Events Management Coordinator (national consultant), the host or collaborating partner in the DMC will facilitate the planned knowledge sharing and learning event for policymakers, decision makers, and associated stakeholders.

Country     Regional
Project Number     48450-001
Type or Modality of Assistance     Technical Assistance
Financing     Public sector (Sovereign)
Status     Approved
Approval Number     TA No. -
Approval Date     13 Jul 2015
ADB Financing ($ thousand)     2,500
Sector     Agriculture, natural resources and rural development
Education
Energy
Finance
Health and social protection
Public sector management
Transport
Water supply and other municipal infrastructure and services
Subsector     Agricultural policy, institutional and capacity development
Agro-industry, marketing, and trade
Irrigation
Land-based natural resources management
Rural water policy, institutional and capacity development
Water-based natural resources management
Education sector development
Pre-primary and primary
Secondary
Technical and vocational education and training
Tertiary
Conventional energy generation
Electricity transmission and distribution
Energy efficiency and conservation
Energy sector development and institutional reform
Energy utility services
Large hydropower generation
Renewable energy generation - solar
Renewable energy generation - wind
Banking systems and nonbank financial institutions
Finance sector development
Housing finance
Inclusive finance
Infrastructure finance and investment funds
Insurance and contractual savings
Money and capital markets
Small and medium enterprise finance and leasing
Trade finance
Health care finance
Health insurance and subsidized health programs
Health sector development and reform
Health system development
Mother and child health care
Decentralization
Economic affairs management
Law and judiciary
Public administration
Public expenditure and fiscal management
Reforms of state owned enterprises
Social protection initiatives
Multimodal logistics
Rail transport (non-urban)
Road transport (non-urban)
Transport policies and institutional development
Urban public transport
Urban roads and traffic management
Water transport (non-urban)
Other urban services
Urban policy, institutional and capacity development
Urban sanitation
Urban sewerage
Urban solid waste management
Urban water supply
Strategic Agendas     Environmentally sustainable growth (ESG)
Inclusive economic growth (IEG)
Regional integration (RCI)
Drivers of Change     Gender equity and mainstreaming (GEM)
Governance and capacity development (GCD)
Knowledge solutions (KNS)
Partnerships (PAR)
Private sector development (PSD)
Gender Equity and Mainstreaming Categories     Category 3: Some gender elements (SGE)

Project Data Sheets (PDS) contain summary information on the project or program. Because the PDS is a work in progress, some information may not be included in its initial version but will be added as it becomes available. Information about proposed projects is tentative and indicative.

Responsible ADB Officer: Hans Van Rijn (Request for information)   
Responsible ADB Department: Independent Evaluation Department
Responsible ADB Divisions :Office of the Director General, IED
Last PDS Update: 14 July 2015