The BRAC-CEGA Learning Collaborative (BCLC) is a partnership between CEGA and BRAC that has the goal of institutionalizing rigorous impact evaluation within BRAC. Similar to EASST, BCLC builds capacity through hosting BRAC researchers as visiting fellows at UC Berkeley and funding collaborative research projects. The BCLC worked with BRAC Bangladesh from 2012 to 2015, and has been recently re-launched to work with BRAC International in Uganda with generous funding from the International Development Research Center (IDRC). 
 
To keep up the momentum of the collaboration, BCLC launched a travel grant competition in the spring of 2017. The grant was designed to award CEGA faculty affiliates and their PhD students to visit BRAC field offices to develop new research projects, start new partnerships, or to bolster existing partnerships with BRAC.  In the research sphere, face-to-face meetings catalyze the collaborative process, by building trust and initiating relationships that are difficult to forge virtually and across time zones-- it is hoped that these grants will lead to long-lasting collaborations.  
 
The following post was written by Gregory Lane, who was awarded a travel grant to Bangladesh.  Lane is a PhD student at UC Berkeley's Department of Agricultural & Resource Economics (ARE).  

In late 2013, a team of UCB researchers (including Elisabeth Sadoulet, Alain de Janvry, and myself) began collaboration with BRAC Microfinance’s Research and Development Unit (RDU) on developing a new financial product.  The goal was to produce a new tool that could be used to help clients effectively respond to unforeseen income shocks. The more traditional microfinance products already used by BRAC were too rigid to be used in this manner, as the institution requires full repayment of any previous loan before offering additional credit. After a year of consultation, we produced a product called the Emergency Loan. This new loan is a pre-approved, index-based credit product that is to be made available to qualified BRAC clients in the event of a flood (later to be expanded to other types of income shocks). Its purpose is to remove some of the downside risk arising from weather shocks, thus encouraging greater investment in productive, but risky technologies (high value crops), while also allowing faster recovery. 
 
A full-scale evaluation of the Emergency Loan via an RCT began during the summer of 2016. However, during the first year of the experiment we encountered some administrative difficulties that limited the power of the experiment. Specially, in certain branches the Branch Manager did not spend enough time informing eligible clients when the loans were made available (i.e. when the floods had exceeded the pre-determined threshold). Furthermore, we found that BRAC’s Loan Officer incentives were not aligned with ensuring that clients had the opportunity to take an Emergency Loan.  First, Loan Officers perceived the Emergency Loan as risky and they feared that lower repayment rates would reflect poorly on them. Second, Loan Officers were only directly incentivized to disburse traditional loans. In some situations, this caused Loan Officers to encourage clients to wait until a traditional loan became available rather than take an Emergency Loan right away.  In all, these issues combined to reduce the number of loans that were actually disbursed.   
 
To address these problems, I traveled to Dhaka in May 2017 with funding from the BRAC-CEGA Travel Grant to work with BRAC’s RDU to implement changes for the second year of the experiment. Together, we agreed to implement several adjustments. First, both Branch Managers and Loan Officers would have Emergency Loan disbursals count towards their yearly incentives for loans disbursed.  Second, a memo was distributed to all branches that management would not weigh delinquent Emergency Loans as heavily in the evaluation criteria for each Loan Officer and Branch Manager.  Third, we informed branches that there would be regular in-person check-ins from the head office management to ensure that clients were well informed about their eligibility and that after a flood all eligible clients were notified about the loan activation. Together it is hoped that these changes to the incentive structure will ensure that every eligible BRAC client would have the opportunity to take an Emergency Loan should the need arise.   
 
During this time in Dhaka, we also began discussions on possible future collaborations between UCB and BRAC’s RDU.  In particular, we explored the possibility of examining the effects of bKash (a mobile money platform) integration with BRAC’s microfinance operations, a process that will begin in the coming years. The project would examine how this integration changes participation in microfinance, savings rates, and overall portfolio health.   

EASST seeks to train researchers in rigorous impact evaluation methods with the ultimate goal of producing high-quality, locally-generated evidence for policymaking. Impact Evaluations are often considered the most effective way to show causal links between a program and its effects on populations. However, there remain several debates on how findings from an impact evaluation in one context can be successfully translated to another context.

J-PAL’s  Mary Ann Bates and Rachel Glennerster call this debate “the generalizability puzzle” – and put forward a compelling “generalizability framework” for policy makers to use in their exploration of whether a solution would be appropriate for their context. The authors elegantly argue that focusing on underlying causal mechanisms and specific “human behaviours” behind why an evaluation was successful, married with crucial local data, would be the best way to translate findings to other contexts. They provide the example of a study that found that providing lentils proved an effective incentive to people’s decision to vaccinate in rural India. It would be hardly possible to pick up this program and drop it into another context—different cultures’ food preferences and ways of accessing food are different, as a start. But there are valuable lessons embedded in the mechanism behind why the incentive worked that could apply to increasing demand for preventative care measures elsewhere.

Bates and Glennerster provide several examples of how to use their framework to apply the findings of particular interventions to other contexts. In one example, they discuss J-PAL Africa’s work to scale up the “Sugar Daddies Risk Awareness”  HIV-prevention program that was successful in Kenya in the Rwandan context. J-PAL Africa worked with the Rwanda Biomedical Center (directed by EASST fellow Jeanine Condo) to collect descriptive data. This program, which involves showing teenagers a video revealing that older men have higher HIV rates—significantly reduced the number of sexual relationships between teenage girls and older men and therefore girls’ risk of HIV transmission in Kenya. However, working with the RBC’s data revealed that most teenage girls in Rwanda already knew that older men had a higher relative risk of HIV. Along with this, teenage girls tended to overestimate men’s HIV risk as whole. This shows that if the Kenya program had been dropped into Rwanda without careful consideration of the mechanisms at play in the Rwandan context, it may have resulted in unprotected sex increasing because of girls’ realizations that HIV risk wasn’t as high as they thought. Therefore, J-PAL Africa recommended pursuing different mechanisms for addressing this problem in Rwanda.

The authors conclude that, “if researchers and policy makers continue to view results of impact evaluations as a black box and fail to focus on mechanisms, the movement toward evidence-based policy making will fall far short of its potential for improving people’s lives.”

​To read the full article, click here.