Putting research into practice

This blog offers some reflections on communication between researchers and practitioners. Links to papers and presentations from the February 2023 private sector development research workshop, hosted at BII in association with PSDRN, CEPR STEG and IGC, can be found at the bottom of this page.      

We co-founded the Private Sector Development Research Network (PSDRN) to encourage interaction between academics and the staff of development finance institutions who can put research into practice. At BII we sincerely appreciate the time that individual academics volunteer to speak to us, for which they receive little reward in terms of career advancement. And we are very grateful to institutions, such as the IGC, J-PAL and VoxDev, that work hard to make research more accessible to the likes of us.  

But as the panellists who spoke at the start of the research workshop said, we practitioners don’t always get what we want from research. How could we get more out of research?  

First, a caveat. This is a big picture and I only see a corner of it. Very many academics interact with policymakers in different ways. Our concerns as a DFI also differ from others development organizations – we are generally not wanting evidence to help us design interventions (such as the fantastic example of “teaching at the right level”).  

But like all practitioners, we really want to know what we should be doing, and why. Prioritisation requires information about relative importance.  

Academia tends to advance one paper at a time, with one idea or question per paper. Theoreticians try to simplify problems down, but we want policy decisions to be taken after all things are considered. Surveys and literature reviews go some way towards synthesising knowledge, but they summarise the progression of a literature that consists of papers written to appeal to journal editors. Surveys rarely attempt to help make sense of things for practitioners. They do not say: “given where we are and what we want to achieve, based on what we know here is what you should be doing in order of importance”.  

When it comes to in-person interactions, rather than presenting individual papers, academics could give a more wide-ranging talk about their area of expertise. One could imagine a much less technical approach that describes what we know (with only a high-level view of how we know) and really concentrates on the “so what?”.  

Budget allocations must consider costs and benefits. Impact investors seek (or should seek) the highest social returns on the dollar. Researchers are increasingly trying to combine estimates of costs alongside estimates of benefits. That’s difficult to do (costs are complicated) but it’s a helpful direction. Because impact investors are often left having to take decisions based on informed subjective judgements, they look for benchmarks to help them assess what “relatively good” looks like. BII’s Impact Score has a component called “intensity”, where we try to identify and reward investments than will create more jobs per dollar invested, or avoid more carbon, than average. Descriptive statistics that help us make comparisons, and get a sense of relative magnitudes, are very helpful.       

One of the reasons why DFIs and other development organisations employ economists is to help them translate evidence into recommendations. Some questions – the relative importance of very different objectives – cannot be answered by research and really require an institutional position. But we could use more help.  

All this would be a lot of work, and at present I don’t think academics would be well rewarded for it. Some university systems reward papers that influence policy, but as far as I know there is no incentive for spending time putting together guidance for practitioners.  And it would be really difficult. It is easy to answer big policy questions with brash overconfidence – reaching subjective conclusions carefully and with good reason is different and not something academia trains you for.[1] 

Perhaps we don’t get the answers we need because researchers don’t know the questions we are asking. BII Chief Impact Officer Liz Lloyd mentioned how we’d been searching the VoxDev trade literature review to learn whether we should be helping low-income countries import food, without success. But how could trade economists have known that’s an important question for us?    

I am not going to attempt a definitive list of unanswered questions here, but perhaps a couple of examples might help.  

A basic role of DFIs is getting credit into the hands of firms. Should our approach respond to firm size? Before thinking about that question, we need to clear away some related questions. First, there are different ways of doing things. Putting equity into a bank so it can expand its overall operations will have a certain effect on the volume of SME lending; a directed credit line to a bank, earmarked for SME lending, will have another. Let’s just think about the impact of a dollar borrowed by a firm, however it gets there. Second, was the borrower credit constrained? If we have no ability to discriminate, then background probabilities matter (what proportion of firms of different sizes are credit constrained?) but if we are able to identify credit-constrained firms, that’s less important. For ease here let’s consider borrowing by credit constrained firms (but we’d want to explore both).  

Suppose we have $1m to lend, and could split it between one large enterprise, 10 medium enterprise, 100 small enterprises, or 1000 microenterprises. What do we want to know? Understanding the impact on the borrowers would be a good start, but the overall (general equilibrium) effects would be better.[2] Which would have the largest impact on growth and productivity? We know very few small firms grow, but perhaps a few more growing would add up to something significant. Or not.  Next, who benefits? How are benefits shared between proprietors, employees, or customers? And who are they? Those are the questions we want answered to figure out how much effort we should expend on finding ways to lend to firms of different sizes.    

Other questions are less operational and are more about the role of development finance in overall development policy. Whenever there is a government inquiry or report written on that sort of question, we explain how important growth is for poverty reduction, including the eradication of extreme poverty, and civil society organisations respond by saying that growth is not guaranteed to reduce poverty and hence investment must be targeted at extreme poverty. Is that right? Perhaps the main reason why growth does not always reduce poverty is that governments do not always collect taxes and spend them on well-designed social services (something that is rather out of our hands). I am sure there are other potential explanations.    

One way to let researchers know what questions we care about is to talk to each other more often, but perhaps we could do more. Perhaps for every literature review, practitioners should write an unanswered questions review.  

[1] It is not obvious that anyone is trained for it or can agree on what it looks like when we see it.
[2] Some great papers in that spirit are Rotemberg Equilibrium Effects of Firm Subsidies; Bazzi et al. Firm Entry Barriers, Growth Constraints, and Job Creation; Cai and Szeidl Indirect Effects of Access to Finance

PSDRN / IGC / CEPR STEG private sector development research workshop 

Policy Panel: Putting Research into Practice: What We Need and What We Get. [Video] 

Panellists

• Prof. Adnan Khan, Chief Economist and Director for the Economics and Evaluation Directorate at FCDO;  
• Liz Lloyd CBE, Chief Impact Officer at BII; 
• Denis Medvedev, Director of Economic Policy Research at the International Finance Corporation (IFC), World Bank Group 

Research presentations   

• Roberto Fattal Jaef: Deconstructing the Missing Middle: Informality and Growth of Firms in Sub-Saharan Africa (with Kaleb Abreha, Xavier Cirera, Elwyn Davies, and Hibret Maemir). [paper] [video] 
• Sandhya Garg: Financial Access and Gender-wise Entrepreneurship: Evidence from Rural India (with Samarth Gupta and Sushanta Mallick) [paper] [video] 
• Chhavi Rastogi: Entry along the Supply Chain: Removing Growth Restrictions on Firms in India. [paper] [video] 
• Nicholas Trachter: Sectoral Development Multipliers (with Francisco Buera) [video] 
• Alexandros Ragoussis: The Great Upgrade (with Jonathan Timmis) [paper] [video]