Wrap-Up of The Evidence Project—Building a Theory of Change

I have not offered an expanded Benefits Diagram (as Freedom from Hunger calls its theory of change) since Theme Six, when I completed the discussion of “intermediate outcomes.” Since then, I have explored the evidence pertaining to the business cases for Credit with Education and Saving for Change and the evidence for “ultimate impacts” (reduced poverty, increased food security and improved nutrition). Now it is time to pull all this together in the Benefits Diagram.

The Benefits Diagram is simplistic. In part, it is simplistic because the diagram must serve as a communication device, for sharing within and outside Freedom from Hunger what the organization is trying to accomplish with its program strategy and its partners’ delivery models. The diagram could be much more complex, but it would not communicate well; viewers would find it difficult to distinguish and remember the salient features. Also in part, the simplicity reflects the lack (so far) of detailed information about the possible cause-and-effect pathways and how they interact with each other.

The Benefits Diagram is not a complete theory of change. To be complete, the diagram would have to include all the interactions among the internal components as well as their interactions with external factors beyond the span of control or even knowledge of the actors shown in the diagram. Some external factors deter or enhance implementation, such as the pre-existing social capital in the communities from which the credit or savings groups are recruited or the other sources of financial service that may compete with the program’s services. Other external factors beyond the program’s influence cause or suppress the intermediate outcomes or ultimate impacts we seek, such as weather variations and food price fluctuations, which weigh heavily on household food security. A complete theory of change would be too complex (for me at least) to portray in a single, intelligible diagram. However, a careful reading of the posts in each Theme should reveal the majority of these missing interactions and external factors. Look at the Benefits Diagram shown below as a sort of executive summary.

The purpose of The Evidence Project has been to complete or fill in the diagram with new information reflecting the evidence I could find regarding the likelihood of the various impacts that the Benefits Diagram predicts. Colors represent the confidence we can have in the evidence that certain household- and individual-level impacts do in fact occur in response to participation by women in Credit with Education and Saving for Change programs (in one or more of their variations). The width of an arrow pointing from cause to effect represents the likelihood the impact will occur; that is, the relative frequency of the impact’s occurrence. The evidence that an impact actually occurs may be very strong even though the circumstances which create the impact may be relatively rare.

I will illustrate the point by walking you along the main cause-and-effect pathways.

In general, the color blue combined with thick arrows from the groups to the intermediate outcome and onward to the ultimate impact altogether indicate a well-documented, reliable (= frequently traveled) causal pathway from the intervention (services to groups) to the ultimate impact. We see this encouraging combination in the pathways from both group types (credit-led and savings-led) to “More Savings & Consumption Smoothing” and onward to “Increased Food Security.” The evidence indicates that these two pathways are the strongest, most robust components of the Benefits Diagram. That is, the group-based microfinance model seems to most reliably lead to increased household food security. This is very encouraging for an organization named Freedom from Hunger! The chosen program strategy for the past 24-plus years seems to have been a fortunate choice.

But wait—there’s more!

There are less well-documented and somewhat less reliable pathways in dark green from both group types to “More Social Capital and Self-Confidence” and onward to all three ultimate impacts. The width of arrows from the groups to the intermediate outcomes is based on detailed evidence. I admit that the widths of the arrows onward to ultimate impacts are based less on hard evidence detailed in The Evidence Project posts and more on my logical deduction from the literature and even conventional wisdom. In the case of this particular pathway, it seems logical that social capital and self-confidence would be important, even necessary, for achieving any of the three ultimate impacts, but relatively less important than increased household income is for “Reduced Poverty,” or increased savings is for “Increased Food Security,” or better health practice is for “Improved Nutrition.” The evidence does not support a definitive statement, but it seems very likely that the cause-and-effect relationship between group participation and household food security is made stronger, perhaps even made possible, by the parallel but synergistic pathway from group participation to more social capital and self-confidence. I cannot say whether social capital and self-confidence operate independently of each other or whether increase of either or both is a necessary precursor for increased household food security or just an accelerator of the impact of the group-based microfinance service.

The reader may prefer different widths for some of the arrows, and I invite such second-guessing. The Evidence Project, for all its posts and hyperlinks to evidence-laden papers and websites, is just a first effort in what should be an iterative process of making better and better inferences from better and better evidence. There are many gaps and weaknesses in the evidence I have found. This is a work in progress, even though my watch as leader of the project is drawing to a close.

Regarding the other pathways, logic indicates a strong cause-and-effect pathway from “More Profitable Business/Household Income” to “Reduced Poverty” and from “Better Health Knowledge & Practice” to “Improved Nutrition.” Moreover, the hard evidence indicates that both ultimate impacts have been generated by a few microfinance programs, including Freedom from Hunger’s partners. However, the likelihood that we will observe these impacts in most microfinance programs is diminished by dependence on circumstances that are not often present, even in the programs of Freedom from Hunger partners. Extra program features are needed but are often not present.

For example, the microfinance intervention must be targeted to serious entrepreneurs in quasi-active economies if we expect the majority of borrowers/savers to start and grow truly profitable enterprises that truly raise household incomes. The fact that this is often not the case explains the thin arrows from the groups to “More Profitable Business/Household Income.” Moreover, the microfinance services must be targeted to serious entrepreneurs who start below a relevant poverty line if we expect the majority of borrowers/savers to rise out of poverty. Again, this is often not the case, even for Freedom from Hunger’s partners (though the program placement bias toward rural areas mitigates the lack of specific targeting to poorer individuals).

Likewise for “Better Health Knowledge & Practice,” something more is needed but often not provided. The microfinance intervention must integrate in some meaningful way with health/nutrition education and services that address the most prevalent causes of malnutrition, especially in young children, if we expect the majority of client households to experience improved nutrition. The fact that this is often not the case explains the thin arrows from the groups to “Better Health Knowledge & Practice.” Microfinance interventions in general infrequently provide or link to health education and services. Even in the programs of Freedom from Hunger partners, education may not be provided reliably well or at all. And when health education is provided, it may not be relevant to nutrition, particularly of young children.

Finally, regarding the pathway from the groups to “Decreased Cost of Borrowing and/or Saving” and onward to “Reduced Poverty,” herein lies the biggest difference between the experience of credit group members vs. savings group members. I concluded in Theme Two that independent savings groups offer a better deal to members than credit groups dependent on formal financial service providers. By “better deal,” I mean the savings group members have autonomy as a group to set their own rules and to be treated as individuals and to reap the financial rewards of lending out their own savings. They get a return on their savings that could never come from a deposit-taking institution. The savings group business model is centered very much on the group itself, mostly in isolation from other groups and external money. This is both its strength and its weakness.

Also in Theme Two, I was surprised to find that credit groups served by microfinance institutions (MFIs) do not necessarily get a much better deal than the members would get by going to a moneylender. The reason is the moneylender is usually in a fairly weak position to enforce the terms of the loan when the borrower gets in trouble, because of the social obligations that often bind borrower and moneylender together in some way. Hence, while charging a fee that works out to be a much higher annual interest rate, the moneylender is much more likely to extend the loan’s duration (without additional charges) in direct response to individual circumstances, thereby reducing the effective interest rate to not much more than MFIs typically charge. Moreover, the moneylender often can provide smaller amounts when they are needed more easily than the MFI can. On the other hand, the MFI has the discipline and the resources of a commercial institution; its offer is more rule-bound and reliable and usually for much more money than can be borrowed in one loan from relatives, friends or even moneylenders—or from a savings group.

I was also surprised by how complicated this pathway can be and how little research has been devoted to understanding what are the relative costs and benefits of the financial service options, both formal and informal, that are available to a poor household (hence the light green color). There is real need for some dedicated researchers to pick a few communities in different parts of the world and map out the financial service “ecosystem” of each one with careful comparison of the explicit and implicit costs and benefits of the available options. Without the benefit of this deep understanding, I do believe the little available evidence indicates that poor households go to MFIs not to replace their other options but to supplement them with a new kind of service that meets some of their financial service needs better than the options already in place. Thus, they do not get a better deal so much as they get a different deal.

Perhaps the scant attention paid to the assumption that microfinance offers a better deal is related to the probable fact that the money saved by borrowing or saving in MFIs and savings groups is insufficient to have much impact on household income and poverty reduction. Notice the arrow to “Reduced Poverty” is relatively thin.

I leave you to study the diagram and even play with it a bit. I have left Freedom from Hunger with a more detailed version that can be manipulated in light of different assumptions and different evidence.

My next and last post of The Evidence Project will provide parting thoughts about what the evidence-to-date tells us.