> home > Reports

Open on Impact? Slow progress in World Bank and IMF poverty analysis

15 September 2005

Executive summary

“(We need to) end the analytic near - monopoly of the World Bank (and the other multilaterals) on the details of pension reform, privatisation of water systems, the ideal bank deposit insurance system, and so many of the other nitty-gritty issues of economic and social reforms. Bank staff should take much more of a hands-off approach to policy design” Nancy Birdsall, Director of Center for Global Development

For too long the policies of developing countries have been based on the priorities of external donors. Many studies have confirmed that these have often been misguided or inoperable. The structural adjustment years at the World Bank and IMF pushed policies that left the poor and vulnerable even more poor and vulnerable. But times have supposedly changed. 

The Bank and the Fund have acknowledged that policy should be based on evidence and are developing tools and methods to analyse which policies are needed to reduce poverty. Poverty and social impact analysis, introduced in the last four years, is one such approach. It is important because it can contribute to reforms being better designed, strengthen national institutions, and make policies more likely to be implemented. A number of these results depend, however, on whether the analysis is carried out in an open and inclusive manner.

Both the Bank and the Fund have made strong commitments to principles of participation and country ownership in their dealings with developing countries.  Both have also made strong commitments to making sure that their policy advice is informed by analysis of the potential impacts of reforms.

But these commitments are not translating to action. 

The evidence that Eurodad has gathered for this report questions whether the Bank and the Fund really are stepping away from ideology-based policy influence to evidence-based policy advice. We have found that PSIA to date has been wrongly focussed, inadequately disseminated and without clear effects on decision-making. There is a danger that PSIA is being used to justify the reform decisions the IFIs have already made in their own lending programmes.

This report is based on both primary research and an extensive literature review, as follows:

• 4 case studies of World Bank [1] funded PSIA in Ghana (electricity reform), Nicaragua (education, fiscal and trade reform), Mali (cotton reform) and Vietnam (state-owned enterprise and WTO accession). [2]  
• A targeted survey of sixty-five civil society policy and advocacy organisations from twenty developing countries. [3]
• Interviews with approximately 25 staff members from the World Bank, IMF and bilateral donors.

This report finds that:

• The World Bank has invested considerable resources into PSIA but much improvement is needed in the way research is done if it is to be useful and effective. 
• The IMF has failed to take the PSIA approach seriously. The small PSIA team established in September 2004 is poorly resourced and has produced extremely limited research. 
• A large majority of civil society groups involved in Poverty Reduction Strategy debates have not heard of PSIA, let alone had an opportunity to engage with it. Of the sixty-five targeted policy and advocacy CSOs worldwide surveyed by Eurodad, only one quarter had even heard of PSIA and only two had any knowledge of how the PSIAs in their country were selected. This was also seen clearly in each of the Eurodad case studies.
• Many developing country governments are beginning to see PSIA as a donor imposition, rather than something that can assist them to consider, implement and monitor their own policies.
• Most PSIAs only examine minor policy variations or mitigating measures, rather than different policy options. An example is in Ghana’s electricity sector where the study only focused on lifeline electricity tariffs for poor households. 
• The World Bank and the IMF continue to choose the topics for the majority of PSIAs. This was the case in Vietnam, Nicaragua and Mali, but in Ghana progress has been made in using national structures to select which issues to study. 
• Macroeconomic frameworks continue to be off the agenda for PSIA despite the increasing debate around the impacts of stringent IMF policies on growth and poverty reduction. The IMF has limited its analysis to date primarily to examining electricity and petroleum pricing without questioning the defined parameters of fiscal space.
• In many cases research has been carried out too late to inform decisions. Examples include the study on WTO accession in Vietnam and the PSIA on fiscal reform in Nicaragua. 
• PSIA studies have been weak on social, political and institutional aspects of reforms, with an over-emphasis on economic aspects. The PSIA on electricity reform in Ghana was the only World Bank funded study that was led by a social development specialist. 
• PSIAs are almost always carried out by external researchers.  This overlooks local knowledge and experience and misses opportunities to build the capacity of local researchers and institutions. Of seven Bank-funded PSIAs in our four case-study countries, only one in Nicaragua had one national consultant as part of the primary research team. And in Mali although a local research was contracted to collect primary data, the data was extracted from the country for analysis in Washington. The IMF has no facility for using national researchers for PSIA.
• PSIAs have not been transparently disseminated. Only 10 of the over 100 PSIAs that have been funded by the World Bank are currently publicly available on their website. Where documents have been disseminated, it is often been after the research had been finished and policy decisions taken. There is little or no effort to undertake proactive (non web-based) dissemination in local languages.

Recommendations:

If done in a manner that alienates its clients, policy research reports may gather dust rather than get implemented. To increase its chances of being taken up by policymakers, researchers need to focus not only on the quality of the analysis, but also the process by which it is carried out. PSIA should be carried out through national structures and institutions in a transparent manner that encourages political debate and creates momentum for change. The principles that official development agencies have committed to for Poverty Reduction Strategies and PSIA recognise this, but are currently not being implemented. Currently therefore PSIA and other in-country research are not realising their potential.

The World Bank, the IMF and other donor agencies need to make many changes in how they contribute to decisions around policy reform; these include a clear commitment from the World Bank and IMF to:

A transparent PSIA process, which facilitates national ownership and policy dialogue;

Ensure that different policy options for reform design are taken into account;

Increasing analytical capacity, and shorter-term strategies to optimise local knowledge and local research resources. 

The IMF must also commit to undertake PSIAs of macroeconomic frameworks, especially in the context of a growth in aid resources and the urgent need to increase financing to meet the MDGs.

[1] None of the case studies selected were IMF led because of the minimal amount of PSIA work the Fund has completed to date.

[2] Criteria used in selection of the case studies were: Geographical spread, stage of PSIA, level of debate around issue and Eurodad member capacity in country.  Eurodad case studies examined the PSIA process in terms of agenda setting, participation and dissemination, whether it informed policy makers with different policy options, in time for decisions to be taken and the impact on policy dialogue and the reform design. Case study reports are printed separately and are also available on www.eurodad.org/psia

[3] This survey targeted policy and advocacy organisations. Organisations were contacted using a snowball approach and using multiple entry points. The list of organisations that responded and an example of the questionnaire – that was adapted for each country - can be found at the above web address.

Full report : 2,16 Kb