The Big Push Back!

Posted on 11 October 2010

By Rosalind Eyben

In my book Relationships for Aid, I wrote about the international aid culture that ignores power, relations, the partiality of knowledge and complexity, and pretends there are no surprises and unplanned consequences.  

In the last couple of years, it has only got worse. British government aid (DFID) is  now imposing extraordinary demands in terms of reporting against indicators of achievement that bear little relation to the manner and possibilities donor-funded activities have for supporting social transformation.   Researchers and NGOs in other European countries report a similar phenomenon. And because the pressure is coming from international donors, we know that the same trend is being experienced all over Aidland.

Theoretical and contested concepts such as civil society, capacity or policy become reified and then numbers assigned to the reification e.g. ‘state the number of policies influenced’.   Answers are required to absurd value-for-money questions in which institutions are considered as if they were motor cars.

Last year a government donor organisation asked me “what evidence exists of the relative cost, effectiveness, efficiency, impact and quality demonstrated by civil society organisations, in comparison to the UN or profit-making organisations?” That was the moment when I decided it was time for a big push back.

Last month, I organised a meeting for those increasingly worried about the donor trend to support only those programmes claiming to deliver easily measurable results rather than to support transformative processes of positive and sustainable changes in people’s lives.   In my invitation, I wrote that many development practitioners cynically comply with the performance measurement demands, often with a nod and a wink from a sympathetic bureaucrat equally despairing of what is now required.

Compliance is accompanied by secret resistance. People carry on doing what seems most appropriate according to their own judgement. But compliance and resistance consume energy and enthusiasm. The methods demanded of us to be more accountable are actually having the effect of our becoming ever less responsible for seriously enquiring of ourselves how we can most usefully contribute to transformative social change and be held accountable for our actions in that respect.

My invitation struck a strong chord.   One senior official from an international development organisation phoned to say ‘We negotiated for several months with [a government donor] and they themselves knew it was ridiculous what they were asking for. In the end it comes down to money and for several hundreds of millions of dollars we had to agree.’

 Of course, the trend is more widely manifest. Someone in Australia e-mailed that ‘The front line local government community development workers, bushfire reconstruction teams and indigenous workers all face exactly the same issues’. A community worker in Ireland wrote ‘This year so far we have had 8 different external evaluations…. Each one is as useless as the next and I am forced to play dumb number games. Apart from retaining most of our funding by dancing their mind-numbing jigs, there has not been one added insight or benefit to young people from this mountain of paper’.

But the problem is accentuated in the international development sector.  Aid has been around for a long time and there is increasing pressure for quick ‘wins’, particularly with the current shift to the right in European politics that puts aid flows at risk. There is also a reality gap between citizens in the North providing the money and those in the South who are meant to be benefitting. 

People apparently have little appetite for complex messages. Consequently donors and those they fund have been complicit in pretending development is simple.  Because NGOs are competing among themselves for financing they comply with donor requirements and thereby confirm and reinforce the trend.

More broadly, the prevailing neo-liberal ideology of everyone supposed to be pursuing their own self-interest has created an environment of suspicion; the desire for quantitative data arises when there is a lack of trust. Agencies or individuals seek to cover their backs through spurious statistics. The cultural values and behaviour of the international finance sector (and look where that has got us to!) have permeated into development work. Multinational accountancy companies are now running programmes for strengthening civil society and jostling to be the watchdogs of aid.  

Participants welcomed the possibility of collective research and action to dialogue with donors and create more space for transformative development practice.  The Participation, Power and Social Change team at the Institute of Development Studies  is currently exploring how to take forward collective advocacy, communications and knowledge sharing among an informal network of practitioners and researchers pushing back against this devastating trend and pushing forward to make aid work better for people living in poverty.

Rosalind Eyben is a Research Fellow in the Participation, Power and Social Change team at the Institute of Deveopment Studies. She can be contacted at r.eyben@ids.ac.uk.


19 responses to The Big Push Back!

  • [...] PS 11 October 2010. See the latest posting by Ros Eyben on this topic here, on the Hauser Centre blog [...]

  • Irene Guijt says:

    Hurrah – the more debate on this the better! I’m so sorry I was unable to attend the meeting, Rosalind. Last week at the European Evaluation Society quite intelligent debate about the need to maintain the space for plurality of accountability forms, to unmask the spin on supposedly ‘more rigorous’ evaluation processes that privilege some methods over others, and to start not with method but purpose of evaluation and users. Prof. Dr. Jennifer Greene gave a very wise keynote on the need for contextually-responsive methodological and method choices. There were also very interesting presentations by aid agencies working with some of the currently privileged methods – RCTs and quasi-experimental methods – with two of the cases highlighting the very, very high costs for relatively insignificant findings.

    The irony is that the push for a certain kind of quantitative ‘proof’ of worthiness is, in fact, not well tested for the kinds of development that focus on institutional transformation. We need to evaluate these evaluation approaches! In the Dutch aid system, I am told that all development NGOs are now required to construct baselines for all their programs (and they have 5 months to do so it seems) and to work with control/reference groups. Will be interesting to see what control groups are identified for advocacy on gay/lesbian rights and mobilization of citizen action through television on basic rights, and for programs dealing with internally displaced due to armed conflict. …

  • Don says:

    Some older and more experienced evaluators apologise when they ask for the numbers and the younger ones don’t. Which is worse – cynicism or naivety? The impetus for impact to be measurable and proven is fuelled by divisive mistrust and lack of insight and rarely overcomes its own shortcomings to add to real wisdom and understanding. How does one measure happiness and love and not being alone and dying in a dignified way?

  • Mabrouka says:

    Unfortunately, as long as suspicions towards aid agencies grow (another one this week by Philip Gourevitch, “Alms Dealers,” The New Yorker, October 11, 2010), demands for evaluation and audits will grow and aid will be even more bureaucratic therefore less effective.
    It seems that pointing fingers at aid agencies is in fashion lately. Why? Is it because results are not visible while SUVs are everywhere? Or because aid agencies have inflated with millions of dollars budget while, in the South, intended aid recipients live with less than $1 a day? As Rosalind points out, international development is complex and that is precisely why suspicions grow. Pointing fingers at aid workers is easy.
    I wish aid agencies were more transparent on the day-to-day challenges they are facing.
    Don’t get me wrong I support audits (I was an auditor before working in development); I support them as long as it serves the purpose.

  • Great to see this meeting take place! (Also see Ben Ramalingam’s post on it at:
    http://aidontheedge.info/2010/09/29/pushing-back-against-linearity/)

    I would wholeheartedly welcome a shift in the cognitive frameworks with which we talk about international development. The business sector seems to have a healthier relationship with risk in their for-profit endeavours. Yet in the development sector, I’ve been observing our increasing desperation to find “evidence” will never allow us to know what is often inherently beyond logic and induction.

    In working with large corporate aid agencies over the years, I continually experienced the limitations of large-scale, donor-controlled, project-based funding, recognizing the profound need for community-driven development initiatives that were genuinely responsive to local needs. I’ve also had the unique privilege to experience the impact and potential of alternative mechanisms that directly support community leaders and that, for me, highlight the way forward for our sector.

    The RCT “gold standard” is especially troubling when one is talking about grassroots-up initiatives. Imposing expectations to “try to evaluate every single intervention” on people who are in the process of organizing at the local level is most certainly a drain on their time and scarce resources. And what so many people on the ground have told me again and again is that abstract metrics or research constructs don’t help them understand their relationship to improving the well-being of the people they serve. As members of the community, they read trends through what’s happening on the ground, rather than using any theory. It’s time for us to recognize that one can monitor not only through data, but also through dialogue.

    Let’s always consider what is the appropriate cost and complexity needed for evaluation (especially given the size and scope of the program) and aim for proportional expectations so we ensure it remains a tool for learning, not risk-reduction.

    • Lee says:

      I agree that this debate is both necessary and productive. However, I disagree with several of your contentions.

      First, you write that the biz community has a “healthier relationship with risk”. I’m going to go out on a limb and say that the biz community is just as scared as the donor community as what lays beyond logic and induction. We all love studying intuition, and their are venture capitalists who invest based on it, but for the most part you’re finding invention based on round upon round of RCTs, pilots, and focus groups.

      Second, with regards to your concerns over “the gold standard” and grassroots NGOs. The grassroots NGOs I’ve worked for get their money from individual donors, church groups, and small communities, not USAID or The World Bank. Thus, while the NGOs I’ve worked for would love to conduct RCTs for internal purposes, the fact that they don’t have the money/time to do so hardly impacts their ability to continue working.

      You might think this is a good thing–small NGOs keep getting to do their good deeds after all. While I think there’s certainly a place for grassroots NGOs, that they can be far more efficient, and have far deeper relationships with the stakeholders involved, the fact that their money often comes from uneducated donors alleviates the NGO’s responsibility for comprehensive reporting.

      twitter.com/leecohen

  • Hear hear.
    The big challenge is to find some politically viable alternative(s) to push. Storytelling and foucauldian genealogies simple won’t make the cut.

    Where do I sign up?

  • It’s great to see this discussion. One of the key challenges for the paricipants at the big push back meeting was to move on from analysis of the problem to thinking practically about how to work together to make alternatives ‘politically viable’ (as Sceptical Secondo puts it). Lots of people are already trying to do this and I hope the big push back initiative will encourage them to share their experiences of political success and tell us how they managed it!

  • Joachim Cert says:

    Certainly donor and international development agencies still need to come up with impact evaluation evaluation approaches that are more realistic and useful. However that’s not an excuse for the huge amount (US$ billions) of aid that go to the NGO community every year to be unaccounted for. I have had the opportunity to work in several organizations and came to realize that a huge number of the so called CSOs do very little “work on the field” yet waste an enormous amount of resources in international trips, conferences (very often traveling first-class, lodging at expensive hotels, touring around, etc), high end SUVs, fancy offices, lobbying, dinners and social events (with elite, not with “beneficiaries”) and nobody seems to care about that.

  • moises venancio says:

    I have just been made aware of your work on the big push back on Donor reporting. 70 percent of my work is now doing log-frames with key Donors. Your work very much has your finger on the pulse.

    However, after 16 years in the aid business no matter how problematic these results matrixes are (for various reasons) I must also say that the effort to try and focus everyone on results is welcome. All agencies (Including my own) want to do ” Peace and sustainable development” and all those grand things that read much better and allow for great philosophical and humanist ambitions. Yet such lofty ambitions encapsulate everything and nothing and do little to assist focusing and realistically managing aid as well as increasingly limited resources around concrete objectives.

    All aid agencies need much better management discipline and having a sharper focus on achieving real world results as opposed to holding meetings , workshops, mission , etc is an excellent step in the right direction

    Anyway I suppose I just wanted to say that too much focus on results is indeed rather counter productive not least as it risks turning results into another internal industry that significantly adds to work load.

    The crux of the problem is not so much the results frameworks as the lack of leadership/management and managers!! Results frameworks become relative when there are mangers capable of managing for results. In the absence of managers, results frameworks become over bearing reporting tools with limited management potential. In addition, under these conditions, they almost become another internal industry and foster increased resentment due to growing workloads.

    As such, The introduction of results based frameworks really should be pondered by Organsiations as their value is only germane/maximised if it takes place within wider Corporate shift towards stronger management cultures.

  • [...] The Big Push Back. Rosalind Eyben is “worried about the donor trend to support only those programmes claiming to deliver easily measurable results rather than to support transformative processes of positive and sustainable changes in people’s lives” — which has resulted in a system in which “many development practitioners cynically comply with the performance measurement demands, often with a nod and a wink from a sympathetic bureaucrat equally despairing of what is now required.” Duncan Green hits a similar theme. [...]

  • Girish Menon says:

    Thank you Ros for sending me information about the meeting and the paper. I found the discussions on the Big Push very interesting and relevant indeed. It is a fine balance to be maintained between how we address some basic accountability issues, often expressed in quantitative terms, while also being fundamentally responsible for contributing to and engaging with processes of social and transformative change. I can relate to many of the challenges expressed in your paper and look forward to engaging further on these discussions !

  • [...] all this is essential to making the aid system work better.  I certainly don’t agree with the whining from aid industry insiders that measurement of results is burdensome or that it distorts decision-making in an unhelpful way. [...]

  • Charlie says:

    It strikes me as completely counter-productive to be against the donor demand for better evidence of impact. Public support for development is shockingly low, with the perception that aid is wasted being top of the list of objections – because of our failure to show impact. We should be engaging the donors in a constructive discussion about what better evidence of impact looks like, not pushing back on their demand for it wholesale.

    Ultimately the communities receiving aid should judge us. This isn’t some vague truism, although it is often treated by NGO people as such. Just like in the private sector, feedback in the NGO sector would work (http://www.ids.ac.uk/index.cfm?objectid=7E5D1074-969C-58FC-7B586DE3994C885C) and it wouldn’t be difficult to design indicators to facilitate this.

    And yet many in the sector seem to prefer their unaccountable, feedback-less, non-transparent comfort zone. It’s time the NGO sector, which I work in and have the deepest respect for, got with the times.

  • [...] work better.  I certainly don’t agree with the whining from aid industry insiders that measurement of results [...]

  • [...] all this is essential to making the aid system work better.  I certainly don’t agree with the whining from aid industry insiders that measurement of results is burdensome or that it distorts decision-making in an unhelpful way. [...]

  • Leave a Response