The East African Social Science Translation (EASST) collaborative started paying attention to bringing researchers and the media closer beginning at its launch back in July 2012. At that time, EASST organized a workshop for both researchers and journalists, alongside its launching conference in Kampala, Uganda. The workshop was on how researchers can communicate their research results with the media and how the media should report results from rigorous impact evaluation studies, hence, translating the findings into inputs for policy making.
Few weeks later I got a call, and the woman on the other end told me that she hosts a radio show on Afro FM (www.afro105fm.com), one of the FM radio stations in Addis Ababa. She called to invite me to one of her shows. My immediate reaction was to say no but I quickly remembered the discussion at the Kigali summit. Then I decided to take up the offer of being a guest on the hour-long live show on a Monday afternoon.
I was not sure what to talk about when I heard that the name of the show is Justified Nature. But we managed to come up with a specific topic for that day and decided to focus on justice and ethics in conducting research. Among the issues we discussed was researchers’ ethics when it comes to running experiments and collecting data for rigorous impact evaluation.
In my opinion there is a limitation on both researchers and research subjects in knowing, as well as in respecting, rights. People, particularly in rural Ethiopia, perceive that such studies are conducted by the government and think that they are obliged to ‘obey’ whatever ‘order’ that they have been given.
Researchers sometimes take advantage of this misconception and proceed with experiments or data collection without explaining that the participants have every right to take part in, refuse to take part in, or even drop out in the middle of the study if they don’t feel like it. There is also a weak ethical review system in the country that protects the right and wellbeing of research subjects. With the exception of the health sector, I have never come across an ethical review process to conduct a research, at least locally.
In addition, we raised and discussed the issue of result manipulation by investigators who often report the result that they would like to see rather than what their data suggests. This could be done for reasons like pleasing donors or politicians, or just for the sake of getting published. But such trends will cast a very serious shadow of doubt on research in general.
I used this opportunity to talk about transparency in research and more specifically about the idea of pre-analysis planning. We discussed how important it would be if researchers were required to register their planned research and outline of their research protocol, particularly those in experimental impact evaluation studies, to ensure that the researchers would stick to their planned study. This way researchers will be able to report on the findings based on the research they had set out in the first place, instead of reporting whatever appeared to be significant at the end. In other words, what this means is that researchers conducting experiments should tell us what kind of experiment they will be running and what kind of analysis they will be conducing using which variables as well as the kind of results they are expecting. Once they specify this they will be conducting the study as outlined in the plan and will tell us what they found at the end. This makes the finding more believable and will leave no room for investigators to manipulate the results in their favor.
Even if documenting pre-analysis plans is at its early stage, discussing it in public like this will help speed up the creation of a demand for it. Once decision makers and the public are aware of it they will start to push for the process to be required for those conducting research; to have a plan at the start of the study as well as to go about their study as planned. This way we can improve the quality, reliability and legitimacy of impact evaluation studies a bit more.