On May 19 and 20 Grassroots Business Fund and the Aspen Network of Development Entrepreneurs co-hosted the workshop "Metrics from the Ground Up" where David Bonbright offered a presentation on surveys and client feedback. A summary of the workshop can be found in the attachment.

David Bonbright, Keystone’s Chief Executive, participated in The Center for Effective Philathropy’s conference ‘Aligning for Impact: Connecting the Dots’, 21 March – 1 April, 2009.

The CEP is a nonprofit organization focused on the development of comparative data to enable higher-performing funders. CEP’s mission is to provide data and create insight so philanthropic funders can better define, assess, and improve their effectiveness and impact.

Members of the Keystone team participated at the Perspectives on Impact Evaluation: Approaches to Assessing Development Effectiveness conference organised by NONIE, 3ie and AfrEA in Cairo on 31 March-2 April 2009.

Keystone team members presented and participated in various pre-conference workshops and sessions where they promoted the concept of constituency voice and the emerging methodology of comparative constituency feedback.

A team supported by the Rockefeller Foundation and comprising members of iScale, Keystone, the International Development Research Centre, EvalNet and RMIT C.I.R.C.L.E has launched an initiative for Impact Evaluation for Improving Development (IE4ID). The team has developed an action agenda document which sets out the requirements for impact evaluation to address the changing realities of development in the 21st century.

First, it describes how we need to rethink impact evaluation by focusing specifically on the nature of development, and how impact evaluation processes and findings can and should contribute to better development.
Second, it describes how we need to reshape IE4ID, using different methods and strategies to rigorously conduct and support use of impact evaluation.
Finally, it identifies essential steps to fundamentally reform the enabling environment of impact evaluation for improving development. International cooperation will be required between commissioners and practitioners for IE4ID to occur in this way.

Rethinking Impact Evaluation

1.    Impact evaluation can and should contribute to improved development
Improving the quality of information is important, but it is not sufficient for impact evaluation to make significant contributions. Impact evaluation of development should be deliberately undertaken for development.

2.    Impact evaluation can and should suit the nature of development
Development initiatives in the 21st century are often interrelated, complicated and complex.

Reshaping Impact Evaluation

3.    Impact evaluation can and should be embedded within robust systems of monitoring, assessment and learning
Evaluations must be embedded in transparent and effective systems for impact planning, assessment and learning that include all relevant stakeholders, including primary consitituents.

4.    Impact evaluation can and should produce a comprehensive picture
Evaluation must provide balanced assessments

5.    Impact evaluation can and should explain how and why impacts occur
Impact evaluation needs to assist knowledge translation about what works, under what conditions, how and why, and hence how  success might be achieved in other places and times

6.    Impact evaluation can and should draw from methodological developments in the natural and social sciences
Efforts to improve the rigour and utlity of impact evaluation are hampered by conceptualisations of science that are inaccurate and outdated.

Reforming Impact Evaluation

7.    Rethinking and reshaping impact evaluation requires fundamental reform
This paper is a call to action to those who want to make impact evaluation relevant,
credible, and useful for improved development. To those who want to make a difference, to
those who want to bring about change, we extend an invitation to become involved.

In November 2007, in the USA, Keystone, the Alliance for Children and Families (ACF) and United Neighborhood Centers of America (UNCA) started a process for exploring the possibility of implementing a comparative constituency feedback survey for children and family service organizations.

Out of this comes a white paper written by Keystone’s David Bonbright, David Campbell of Binghamton University, USA and Linda Nguyen of ACF, which wrestles with a seeming paradox. There is an enduring and widely held conviction among human services leaders that feedback from primary constituents is of great – perhaps even paramount — importance in their work. And yet in practice human service agencies do not find current formal feedback practices to be helpful.
This report shares findings from an ongoing learning and innovation project on the human services sector in the US and provides learning which supports and deepens our understanding of the challenges reflected in the multiplicity of feedback purposes. It has six principal findings:

1. Measurement is a core management function. All study respondents participate in some kind of formal measurement activity and pursue a core set of measurement activities.

In an article entitled ‘Proving or Improving?’ published in the March 2008 edition of Alliance Magazine, David sets out his views on the debate between the use of either context or scientific based metrics to measure impact. He identifies the key shortcomings of the scientific experimental approach before going on to discuss the key things organisations can do to ensure they choose the right impact evaluation metrics. He furthers his point by highlighting the importance of collecting and publishing the views of a project’s primary constituents on an organisation’s progress measures. This means that the entire ecosystem is visible to all creating conditions approaching consumer voice, increasing the voice of the primary constituents and the accountability of the implementing organisation.

In June and July 2007, Keystone was commissioned by Humanity United, a component of the Omidyar Network, in collaboration with the Social Venture Technology Group and Jed Emerson to develop a ‘blended value’ valuation and reporting framework for economic enterprises (in this specific case a planned luxury tourist lodge in Rwanda’s Akagera National park) that embodies respect for the environment, meaningful benefits to area residents, and real economic value for dollars spent.

The resulting framework presents an overview of the various economic, social and environmental considerations that investors and developers should keep in mind as final business plans are drafted and operating contracts in place. It also offers a set of metrics that may now be brought together with the specific financial and operating plans for the venture in order to assess the full value creation that might be realized from this enterprise. To read the full report, click here.

A study from the William and Flora Hewlett Foundation in collaboration with McKinsey&Co, released in November 2008, is looking at the enduring question of how to strengthen the informational base upon which funders make their social investment decisions.

The study, titled "The Non-Profit Marketplace:Bridging the Information Gap in Philanthropy"  explores the issues of:

1. How can we help donors make smart philanthropic decisions?

2. How can we ensure that the strongest, most effective nonprofits get the resources they need?

Keystone is currently accepting applications for a Project Manager to work in the planning phase for the Agriculture Learning and Impacts Network (ALINe) and for a Project Administrator. Click here to read the job descriptions.

Over the coming months, Keystone, working with the Bill & Melinda Gates Foundation and the Institute of Development Studies, will design and plan an Agricultural Learning and Impacts Network (ALINe) to be launched at the beginning of 2010 that will seek to transform evaluative thinking and practice in agriculture. The planning grant website can be found at www.alineplanning.org

The Bill & Melinda Gates Foundation will, over the next few years, invest unprecedented levels of resources in transforming smallholder agriculture.  Their extraordinary goals demand an impact planning, assessment and learning (IPAL) system for helping the Gates Foundation understand how to maximize and accelerate the attainment of those goals.  The ALINe project seeks to bring about the required step change in evaluative thinking and practice in agriculture to keep pace with the Foundation's goals, providing evaluation-related support to Foundation grantees, the Foundation itself, and the wider field of agricultural development.

A key feature of the planning year will be the implementation of comparative constituency feedback surveys that will collect the perceptions of smallholder farmers on the performance and impacts of the Foundation’s grantees.

For more information please contact Keystone's Chief Executive, David Bonbright.