Episode 73: Danila Serra

 

Danila Serra

Danila Serra is an Associate Professor of Economics at Texas A&M University.

Date: June 7, 2022

A transcript of this episode is available here.

Episode Details:

In this episode, we discuss Prof. Serra's work on ethics training for police in Ghana.

“Proud to Belong: The Impact of Ethics Training on Police in Ghana” by Donna Harris, Oana Borcan, Danila Serra, Henry Telli, Bruno Schettini, and Stefan Dercon.

OTHER RESEARCH WE DISCUSS IN THIS EPISODE:

 

TRANSCRIPT OF THIS EPISODE:

Jennifer [00:00:08] Hello and welcome to Probable Causation, a show about law, economics and crime. I'm your host, Jennifer Doleac, Texas A&M University, where I'm an economics professor and the director of the Justice Tech Lab. My guest this week is Danila Serra. Danila Serra is my colleague. She is an associate professor of economics at Texas A&M University as well. Danila, welcome to the show.

 

Danila [00:00:30] Thank you so much for having me. I was waiting for this moment.

 

Jennifer [00:00:36] So glad to have you on the show. So today we're going to talk about your research on a police training program in Ghana. But before we get into that, could you tell us about your research expertize and how you became interested in this topic?

 

Danila [00:00:49] Yes, so much of my research has been on corruption and accountability or more generally. And how did it all start? I mean, I guess it started already with my undergraduate studies. I was always interested in corruption and crime and probably comes from the fact that time I grew up in Italy. I don't know if you have noticed that there are lots of Italian economists working on crime, corruption, and so probably does come a little bit from your experience growing up and what puzzles you growing up. I mean, I've always been interested in corruption. My research has evolved over time, but much of my work has been on governance and accountability. Some of it has been in like very micro based. So I've been working on motivations and incentives and responsiveness to incentives using also, you know, laboratory experiments really to look at what motivates individuals at the micro level. But then my work, that is more development development economics. So my work in the field is being more generally on accountability of service providers with a focus on education and health, primarily up to this point.

 

Danila [00:02:12] So this is my first experience working with police officers, I must say. So I'm really excited about this project. So my interest is in how to incentivize public service providers to provide a better service and so my angle coming into this project is to really see police officers as service providers. So I see the police as a sector of the government and so this has been my approach. And of course, you know, I have soon discovered that there is a huge literature on policing its own field within economics. And so I'm making my way really getting more familiar with this specific literature on police officers.

 

Jennifer [00:02:57] Yeah, it's my side project to get all of my colleagues to eventually write the current paper. So great to have pulled you into this world. So your paper is titled "Proud to Belong: The Impact of Ethics Training on Police Officers." It's coauthored with Donna Harris, Oana Borcan, Henry Telli, Bruno Schettini and Stefan Dercon. This paper describes the results of a field experiment Iran with police officers in Ghana. So let's start with some context. Tell us a bit about that country. And since it is the primary focus of your paper about the corruption problems in the public sector there?

 

Danila [00:03:32] Yes. So Ghana is a very interesting country. It's a country in West Africa and it has experienced lots of progress, both in terms of economic development and economic growth in the past 20, 30 years, but also lots of improvements with respect to poverty reduction and human development indicators, including education, adult literacy, but corruption remains a big problem. And this is not unique to Ghana. It is it is a corruption is rampant in most of the developing world and definitely in sub-Saharan Africa.

 

Danila [00:04:10] And so, you know, it's difficult to quantify corruption. And I think this is one of the challenges in doing research on corruption. It is it is very difficult to measure corruption and be able to say Ghana is especially bad compared to neighboring countries, for example. That's difficult to say because, of course, corruption is legal, but there are many, many ways to a police have an idea of how predominant corruption is in Ghana and in other countries. There are corruption indicators. I'm not sure how familiar your audience is with such indicators, but there are rankings of countries around the world based on perceptions of corruption provided by citizens, businesses as assessment rankings. And so Ghana, I would say, is kind of in the middle of the pack, if we look at, for example, the corruption perception index that is provided annually by Transparency International, I encourage everybody to go check the rankings of their own countries on this in this index. But the police in particular is a sector of the government that is perceived at least to be extremely corrupt.

 

Danila [00:05:21] This is based on a survey of citizens survey done with Afrobarometer, the corruption barometer also produced by Transparency International. So it is whites I mean, the opinion or the perception of rampant corruption within the police in Ghana is widespread, according to the The Corruption Barometer, which is a survey of citizens conducted by Transparency International really, the majority of the Ghanaian population perceived most or all the officers of police officers to be corrupt. And they also have no problem saying that often at least I think that they die, if I remember correctly, suggests about one third of the times that citizens encounter police officers. They are asked to pay bribes. So that's a very, very dangerous problem.

 

Danila [00:06:16] Of course these interactions with corrupt officials also undermine trust in the institutions and corruption in the police in particular is a problem because if you if you encounter corruption in other environments, in other sectors, you would imagine that you could go to the police to report whatever misbehavior you witness or you experience by if you know, if you perceive the police to be corrupt, of course, there is this additional problem. There is an additional constraint to your willingness to do something about corruption or to, you know, condemn corruption publicly, often for corruption.

 

Jennifer [00:06:55] Yeah, that's really interesting. Are there other challenges to addressing corruption in the in policing in particular? I think in the paper, you you talk about the hierarchy within this organization to.

 

Danila [00:07:08] Yeah, I think in in many countries, definitely in many developing countries and this is true for Ghana. The police has a hierarchical structure most of the time due to links to the military. And so the police often works in this hierarchial way where there is a system of order and command and respect towards your superiors. This makes it extremely difficult and definitely, definitely, I believe, more difficult in the police than in other sectors. For example, how the location makes it more difficult for lower level officials to say no to potential orders coming from the top that have to do with, for example, demanding bribes or bringing certain amount of money back at the end of the day, once you are out, you know, traffic stops or patrolling. So this hierarchial l system makes it more difficult, I believe, for lower level officials that maybe want to act honestly, to contrast with any orders that come from the top. And the risk of being punished or transferred or even fired is probably higher for police officers that from, you know, public servants in other sectors.

 

Jennifer [00:08:28] And so when we're thinking about how corruption manifests in policing, it sounds like we're mostly talking about bribes. Is that right?

 

Danila [00:08:36] Yeah. This is at least the kind of corruption that we know more about because this is the kind of corruption that citizens are willing to talk about or that they experience in their lives, in their everyday life. So this is what we know more about. It doesn't mean that there are no other forms of corruption, but at least from, you know, the citizen surveys that I mentioned before, citizens are asked about their encounters with the police and they're primarily talking about traffic police and road stops, roadblocks, and how common it is to be stopped by the police and being asked to pay a bribe for reason. And so this is at least we have more evidence from survey from survey data. And again, I mean, when I want to emphasize again how difficult it is to measure corruption or to have data on objective data, field data on corruption. So what we can work with, at least in Ghana, this study is what we know from surveys of citizens. And so this is what really emerges. And then there are also, you know, policy briefs of reports that documents, again, this tendency to have roadblocks and police stopping you and demanding bribes.

 

Jennifer [00:09:52] So before this paper, given all of your other work on corruption previously, what did we know about how to reduce corruption more broadly, like outside the police?

 

Danila [00:10:02] Now, this is a very, very important question. Interesting question about. Also very hard question to answer. What we know about controlling corruption is that, you know, what we could imagine from theory actually works on monitoring, auditing enforcement, punishment. They do work, so we know what works.

 

Danila [00:10:25] However, the problem is that in countries where corruption is systemic interventions that really act on top down, auditing and enforcement are very difficult to implement because you have to to imagine a system where corruption basically permeates every level of the bureaucratic system, every level of the government. And so it's it's very easy to get out of punishment, to get out of monitoring by paying bribes. And so any kind of system that we know work, any kind of crackdown not coming from the top could work, but it's very difficult to implement. And so the challenge is, what do you do when you're facing a system where bribery and corruption exists at every level, even if, you know, you create an anti-corruption agency, for example, that is supposed to be independent from the government and maybe provide auditing and monitoring. We know from a variety of cases in the developing world that these are usually failing. They are unable to do their job.

 

Danila [00:11:33] And so my approach and the approach of others, of course, is trying to find different ways to fight corruption that doesn't really ways that do not rely on top down auditing and monitoring. And so my focus specifically has been on bottom up interventions and involvement of citizens or service recipients of service provided by public servants. And so that's been that's been my interest at trying to find ways to fight corruption or try to at least pass theme parks, the ways to fight corruption maybe from the bottom up or through mechanisms that do not rely on monetary incentives. Of course, I'm not the only one that also I mean, the research on corruption is vast and growing. There are different ways that are different things that researchers are trying to implement.

 

Danila [00:12:28] For example, the use of technology is what I think we've seen concrete effects, for example, you know, biometric identification to access benefits of social programs. This has been found to be effective. However, again, the problem is implementation. The problem is, you know, how to implement in in contexts where resources are limited and institutions are poor, how to implement the systems that we know work. And so that's a big challenge, I guess, in the fight against corruption and also in the study of works besides, of course, measurement problems that I mentioned before.

 

Jennifer [00:13:07] Yeah. And then turning to the specific topic of this paper. What did we know about the ability of training to change the way police officers perform on the job?

 

Danila [00:13:18] Yeah I had this super interesting area of research, in my opinion. I'm going to be honest. When I went into these projects, I thought that there would be already a huge literature on these challenges, I think. I mean, I think we talked about this at the very beginning. I imagine that outside of, you know, the developing world of the Global South, I imagine that there would be strong evidence in favor or against the effectiveness of training programs for police officers.

 

Jennifer [00:13:50] We would have figured this out already.

 

Danila [00:13:54] I imagine that there would be plenty of studies in the U.S., for example, and I was extremely surprised to see that it very little.

 

Jennifer [00:14:03] Yes.

 

Danila [00:14:04] Especially in terms of causal effects causal impacts are some descriptive studies. We know very little about the causal impact of training on the behavior and attitudes of police officers. There are some studies on that impact, very few, even the impact of procedural justice training, which is a little bit different from what we do.

 

Danila [00:14:27] So we we are we are focusing on an ethics and intent, integrity training. There are some studies, for example, by Emily Owens and others in the U.S. working with, I think heavily was working with the Seattle police.

 

Jennifer [00:14:41] Yeah, that's right. Mm hmm.

 

Danila [00:14:42] Right. And there is a study in Mexico as well by Canales and others also on procedural justice training. There are a couple of studies in the U.K. really. We're talking about a handful.

 

Jennifer [00:14:54] Yeah.

 

Danila [00:14:54] Yeah. Really a handful of studies and the evidence is promising at least this is my reading of the literature. The evidence is little, but the there seems to be reasons to believe that training should work at least what we know about procedural justice. This study showed that it seems to improve behavior or the use of force, for example, of the police, but also citizens attitudes towards the police and vice versa. So there is reason to believe that training could work. I think it is a very, very promising area of research. I think we need to do more work to understand what kind of training works and under which conditions and how to implement these training programs.

 

Danila [00:15:37] The evidence is little, but I think one of the limitations of the existing studies is that outcomes are measured in the short term weeks or I think at most six months after the training. So there is this, you know, again, this call for more research and trying to look at long term insight of training programs. In our case, we are going we we measure outcomes 20 months later, almost two years later. So I think that's one of the good things about our study that we really we are waiting a little bit longer, possibly because of COVID. I mean, this is not something that we planned.

 

Danila [00:16:13] I'm going to be perfectly honest. We didn't plan to go back and look at the impact almost two years later, but COVID forced us to to do so.

 

Jennifer [00:16:22] Yeah. Silver lining.

 

Danila [00:16:24] Yes.

 

Jennifer [00:16:25] Okay. So, yeah, I agree that the literature on police training is remarkably thin, so why don't we know more than we do? It is mostly a data problem. It's just mostly an identification problem. When you first started thinking about testing this program that I guess you all design in Ghana, we'll talk about that in a moment. What were the main hurdles that you had to overcome to figure out how to change behavior in this space?

 

Danila [00:16:49] Right. I think the main problem with identifying the impact of training programs, especially in the context of the police, is that there is an issue of, you know, if we are looking at impact of existing training programs. Right. Because I'm sure that there are plenty of existing training programs for police officers. I think one of the issue is that you could imagine that if there is heterogeneity with respect to which, for example, stations or locations implement the given training program, of course, that might be due to a difference. Something special about the districts that decide to do so in Ghana, for example I would imagine that if there was a police district that decided to go with our proposal to implement training for their officers, maybe this district was already different. So maybe the people in charge of the districts already more invested in reducing unethical behavior of the police. Right. So there is an issue of in as as we know, in order to the same part of a program, we want to make sure that whatever is exposed to the program is randomly selected or it is not different from people that are not disclosed in a way that would compromise the identification.

 

Danila [00:18:11] Right. And so being able to work with the police in a way that allows you to have randomly selects officers who are trained and officers were not trained is a big challenge. Right I mean, I think this is a challenge in any kind of program implementation, working with government agencies. They have a special idea. They have very, very clear ideas on who should receive the training, who are the target officers in this case. And so there is an issue of having to convince them of the benefit of randomization, in our case.

 

Jennifer [00:18:52] At least some sort of phased roll out that gives you --.

 

Danila [00:18:55] Yes.

 

Jennifer [00:18:55] Allows you to get some traction. But yeah, in a lot of cases, they just like they train everybody at once or everyone gets this training when they're--

 

Danila [00:19:01] Yes.

 

Jennifer [00:19:01] In the police academy. And so it's just impossible then to you don't have a comparison group, right?

 

Danila [00:19:06] No.

 

Jennifer [00:19:06] Or a control group.

 

Danila [00:19:07] Exactly. Or if they make the training is voluntary. Right. And so they just say, okay, this is an option. And they let the district heads decide if they want to do it or not, of course, will be another problem. And so there is a lot of work that goes into there was a lot of work in our case that went into creating a relationship with the police and communicating to the police to let us help them in a way that could allow us to measure the impact of the training in a scientific way. And so that took a lot of work, but of course, you know, for us, going into also the design of the intervention was challenging. So even abstracting from the implementation challenges and the cooperation with the police, even in coming up with a program, a training program that we thought theoretically even that could work in this environment was a challenge.

 

Danila [00:20:05] And this is because we're dealing with a setting where we know that unethical behavior and corruption in particular is pervasive. And so we we were dealing with the context where we know that the social norms and the organizational norms had been corrupted. And so we have to think very, very carefully about what kind of program, what kind of training program should have a chance of succeeding in this environment where you are not only dealing with your peers potentially behaving unethically, but also your superiors potentially behaving unethically and demanding that you do the same. And so then, you know, the challenge is how do you break this cycle? How do you persuade officials to behave differently from what is the norm and potentially of risking punishment from peer or so from superiors? So there was a big challenge. And and the design, I mean, it was it was at the core of our design and the way we thought about the program.

 

Jennifer [00:21:13] Yeah. So we'll talk more about that in just a few moments. But I do want to just flag also just data challenges here.

 

Danila [00:21:19] Oh, yeah, I know the measurement.

 

Jennifer [00:21:21] The measurement piece. I mean, I know in in the U.S. where we have, you know, not great data on policing, but, you know, it's getting better there still, as you said in many studies, you know, sometimes they just survey people on the way out of the room, like, how did you feel about the training? Did you think it was useful? More like a, you know, an evaluation. And sometimes they get the it can get data six months out but not two years out. We really want to know, is does this affect the likelihood of making an arrest? Does it change, you know, racial discrimination in stops? Does it change use the force?

 

Danila [00:21:54] Absolutely.

 

Jennifer [00:21:55] And then, of course, you've got challenges in Ghana that are different. So you want to say a little bit more about about that.

 

Danila [00:22:01] Yeah, absolutely. I mean, for for for those of you who work on policing in the U.S., you're used to these very high quality data from the field, have data on police behavior.

 

Jennifer [00:22:16] And even data on crime, right?

 

Danila [00:22:18] Yeah, exactly. And crime complaints against the police use of force, all of these like beautiful data. There is none of that in Ghana. You know, going into this study, we had no kind of baseline data or administrative data to start with. And also we knew that we would have none of this data down the line. So we do not we do not have I mean, we're trying to get some data police on accidents or find since we are targeting traffic police but, it's really something not accessible to us I believe maybe it exists I don't know at this point point.

 

Jennifer [00:22:57] But maybe not.

 

Danila [00:22:58] You know, but maybe not. And so we knew going in that that would be a challenge. And so we had to think about measurements like how would we measure outcomes in this case. And so that was part of our again, one of our design challenges. So think about, of course, survey data. So we we had to conduct a survey of police officers and we did that with at baseline and in line. But also we had to think of some measure. I mean, we knew that we will face criticism by head only at survey based outcomes, even though, I mean, I believe that at least the time lag between the intervention and the survey and the baseline line, help us in a way, but still, you know, self-reported opinions or attitudes of police officers are not, you know, the best the best source of data.

 

Danila [00:23:54] But that's when we we decided to also have an incentivize game as part of our end line survey. So I know we've talked about this a little bit more later, but I want to say so this is something that I didn't mention. When you asked me about my research, my background is behavioral economics, experimental economics. So as I mentioned a little bit, some of my work has been in the lab using lab experiments or incentivized games. And so we brought some of these to the field in Ghana and we had police officers play a cheating game that we we can talk about, about the game a little bit more later. But the idea was to have an some measure of unethical behavior and see whether we could move that measure with the training.

 

Jennifer [00:24:41] Yeah, very neat. Okay. Let's talk about this training program that you and your colleagues designed, your own ethics training program. It did not exist before. So tell us about the program. How did it work?

 

Danila [00:24:53] Yes. And I want to be very upfront and clear. I mean, most of the credit for the training goes to Donna Harris and Bruno Schettini, who really worked very hard to design this, training, all the modules of the train, consulting with social psychologist and a number of people to really have a product inclusive - inclusive of, you know, role being, exercise, esteem, building, exercise. You see a very complex training program. But what I can talk about is what it came from the idea and the theories behind the design. So we started again. I told you before we we were aware that we were facing an environment where organizational norms are corrupted. I've been corrupted for a while. And so we knew that any kind of training program, especially ethics or integrity training program, cannot just be based on, you know, making officers aware of of the law.

 

Danila [00:25:52] For example, what is legal, what is not legal, how you should behave. That's not what we wanted to do. We wanted to design something that would induce introspection and looked at the literature in economics, on identity, in organization. So seminal work of seminal work of our catalogs and content, but also work on mission matching. I'm thinking of papers by battling attack. So here the idea of this work and also work on interesting motivations is that individuals join organizations for for different reasons. And if individuals feel like their motivation or what is called intrinsic motivation matches the motivation or the culture of the organization, they do their job more, more effectively, more efficiently. They work better and they require less pay. So the idea is that there is the organization in this case the police, as its own mission.

 

Danila [00:26:56] We want to think of them mission as as one of service provision and honesty and professionalism. And we also assume that at least some of these officers originally joined the police because they wanted to serve the public, they wanted to help their community. And maybe that's not what they ended up doing because they were faced with norms that were actually in contrast with the original mission of the organization. Right. And so the first thing that we wanted to do is to reactivate those original motivations, knowing that not everybody, of course, not all officers joined the police because of these, because they want to be good police officer, because they want to they want to help or serve the community. Right. But we assumed at least some of them want to do that. Some of them joined the police for that reason. And so we wanted to target those intrinsic motivations that may have been lost.

 

Danila [00:27:55] And we wanted to reactivate such such motivations by having officers remember and think about why they joined the police and also reflect on how their ideal aspirational police officer that they wanted to be differs from the average police officer in Ghana. And there were lots of discussion. I mean, this is this is what we designed the first day of the training to be based on discussion and brainstorming about the gap between this aspirational officer that they wanted to be and the reality of the situation in Ghana. So there was lots of discussion, but also this reactivation exercise was based on practical techniques, helpful setting and also communication techniques. So there were lots of, you know, role playing, exercising, where police officers were pretending to be citizens and interacting with citizens and trying to find solutions on some of those problems that they had identified in their brainstorming phase of the training.

 

Danila [00:28:58] This was day one of the training. So we as we discussing the paper, it's only happening two days. And the first day was was aimed at reactivating this type entity, the intrinsic motivation linked to the identity of police officer, seen as a service provider. That's what we aim to do at least. So making them feel that they are a service provider, their service provider more than law enforcer or crime preventer, they are there to provide a service to their community. So there was, again, a theoretically base on this research on identity and intrinsic motivation and mission matching. The second day of the training tackled the big problem of corrupting social norms and the fact that you cannot be alone in this fight against corruption. Right? If the social norms are biased and also social norms are up for grabs, you cannot or you will never be the only one fighting the system otherwise the risk, the risk is very high.

 

Danila [00:30:00]  So beliefs about others willingness to fight with you, others willingness to work together to change a system are really important. So the second day of training aim to create this group identity a new group identity of agent of change. The purpose was to have participants in the training believe that change is possible together and that they are not alone in wanting to fight the system. And so there was, you know, they all again, lots of exercises, brainstorming, video, watching all the students in them. In Brazil, Bruno Schettini, which was the facilitator of the training sorry, I should specify this Bruno Schettini he was also coauthor of the study. He's a police officer, an inspector, he had experience in training officers successfully in Brazil. And so he was the facilitator of the training. And as part of the training, we showed a video that Donna and Bruno shot in Brazil interviewing officers there as well. So Bruno also brought his experience and there was a lot of discussion about the Ghana experience and how it differs from the Brazil experience. And again, it was very interactive, was always meant to be a kind of let's discuss and talk together about what can be done rather than sort of a lecture based training.

 

Jennifer [00:31:26] Yeah. And so okay, so you had those two days and then.

 

Danila [00:31:29] Yes.

 

Jennifer [00:31:29] Everyone stayed in touch, right? You had WhatsApp groups set up?

 

Danila [00:31:32] Yes. Yes. So we we thought it might be important. Again, this is not something that we can test empirically. I wish I wish you could test how important it is for them to stay in touch, but we do that. So again, future interventions, you know, maybe we could do that, but we thought that after this two day of trainings, but the way these two days of training, they happened in two different week, consecutive weeks.

 

Danila [00:31:56] So officers came on one day of the week in groups of 30 to 40 from different districts and then came again the following the following week. So again, it was training was for Ghana police, it was not the specific districts and so they were all mixed up different districts. Yeah so we thought that after these two days of training, it might be important to facilitate communication between these these officers, these new agents of change that we aim to have to have created. And so we put them together in a WhatsApp chat. So that was our first reinforcement of the intervention. So we we implemented two reinforcements, we call them, one is these WhatsApp groups and then also in collaboration with the police, we had an awards ceremony about eight months after the intervention. The intervention happened between April and May 2019. And yeah, there were ceremony happening December of the same year and there was a very formal award ceremony.

 

Danila [00:33:00] We were not there. It was the police implementing our behalf and we gave all the trained officers certificate of completion. And also importantly, we gave them a pin, a special pin that we created for them and on the pin it was these are the symbol of Ghana, the eagle. And also it was an agent of change as a reminder of this new identity that we are, we created the training.

 

Jennifer [00:33:27] Mm hmm. So who are the main targets of this intervention? Who is eligible to participate in this program?

 

Danila [00:33:33] All traffic police officers working in the greater Accra region of Ghana. So this is this is not the study that we did in all of Ghana. We say Ghana. But in reality, we did this in one of the 11 police regions of Ghana at the most the most urban the Accra region.

 

Danila [00:33:51] And so all the traffic police officers working in this region were eligible, except that we randomize them, so not really, but the target population was traffic police officers operating in the region and the only exclusion here so we, we the randomization that we did was done at the police district level. There were 32 police districts plus the headquarters, so the only exclusion criteria here was that officers that we're working in the headquarters, which is a special district with many more officers and these police officers have different functions, not just traffic police in the sense traffic policing, but also other functions, formal functions. So we we decided not to include these officers also because we were afraid of contamination. These officers, they tend to substitute other officers in the district when needed. They go around a lot. And so the only exclusion criteria was that these officers, the headquarter officers, were not included in the randomization in this study. So they could not be trained at this point. But in general, the target population was all traffic police officers working in this the greater Accra region.

 

Jennifer [00:35:05] Okay. And so tell us more about exactly how you implemented this as a randomized controlled trial. So it's great you have an RCT it makes it really easy to measure causal effects. So how did you select who participated in the training?

 

Danila [00:35:18] Right. So we randomly selected two third of the police districts, traffic police districts in this area. And so 21 out of 32 traffic police districts, we randomly selected them to be our treatment districts. So these are the districts where we implemented the training. However, from the very beginning we were told by the police, by our partners, that we couldn't disrupt the work of police officers.

 

Danila [00:35:48] So they basically told us, okay, so you can do this in two weeks, you plan to do them, which so we had planned for end of April, beginning of May, but the very beginning they were too they told us, you can implement this training program, but you are not gonna change the duties or affect the duties of the police officer. So only officers who are free, who are not assigned to be on the roads, on those specific days, on the specific weeks sorry, are able to participate. So we anticipated that about half of of the officers in the treatment districts would be able to participate and that's why we oversampled treatment this way so 21 out of 32 districts were selected to be treated.

 

Danila [00:36:35] As for who came to the training that was decided based on these predetermined duty rosters. So this duty rosters are made up at the beginning of the month and they determine where officers are in a given week for one or two weeks. And our understanding and see so this is we are treating this as this allocation of this determination of participation in the training as, you know, as good as random. Because we have been told from the very beginning that the assignment of the duties to officers follows a route, a system, kind of a lottery. And this is confirmed by our survey data of of officers. We asked them how they're assigned to duty posts. So out of the officer it's about 300 officers working in their treatment districts about half of them as suspected, were able to participate or came to the training and that we have the officers in the control districts that were not invited to participate nobody from those came.

 

Jennifer [00:37:41] Yes. It's kind of neat that you've got this this two levels of that actual randomization and then the sort of quasi randomization. And so in your actual analysis, you analyze a bunch of different ways, which is great. And so you do some where you just you basically do what we call an intent to treat, where you just compare the treated districts to the control districts. And that's like that's the randomization you actually controlled. But we know that estimates can be a little watered down because not everyone participated. But then you do a bunch of work to convince us that who actually participated, who was off duty on those days looks random. And then you can actually use that additional information as well and kind of use being in a treated district as an instrument. For actually participating. Am I getting all that right?

 

Danila [00:38:30] Yes, yes, exactly. So our preferred specification, I would say if we believe that the assignment of all users to the duties to different locations is random, our preferred specification would be one where we essentially have three groups of officers. We have the control treatment officers, and they were completely, as you say, it's a control group designed randomly by us, so we have full control over them. But then we also have within the treatment districts, we have two groups of officers, we have those who were trained and those were not trained. And we like that specification, so we like to compare it trained to the untrained in the same treatment districts, because that allows us to see if there are any spillover effects of the training. Right. Of course, it is a function of us believing that this participation in the training was indeed as good as random. We have reason to believe that it is. So the best that we can do is look at differences of statistical significance, specifically significant difference between these two groups of officers and which is what we do.

 

Danila [00:39:40] But then, as you say, the most conservative way of looking at analyzing the data is to do an event to treat or estimating and to treat the facts, where we just compare treatment districts and control districts. And then, as you say, again, we can also look more specifically at the same controls, try to control specifically for the selection problem by estimating these leads. So by using assignment to treatment as an instrument for actual participation in the treatment.

 

Jennifer [00:40:10] Yeah, I think the combination of, you know, this institutional knowledge you have, everyone's telling you that their duty assignment is basically a lottery. And then also you have this data, pretty rich data on the characteristics of the officers and you don't see differences. It's like that combined. It's really compelling, I think. So the kind of story one might be really worried about going in, like who's actually participating. But I was convinced that it looks.

 

Danila [00:40:36] Okay Good.

 

Jennifer [00:40:35] It looks random, which is kind of. Yeah. Okay. So tell us a little bit of the backstory here. I mean, this this is like such a cool intervention and you're able to implement this randomized trial in partnership with the police in this country that you've already told us has, you know, corruption problems and a variety of, you know, it's tough to get data and all this stuff. So how did you pull this off? How did the partnership with the police forces in Ghana actually come about?

 

Danila [00:41:03] I think it's a very interesting story in the sense that when we started thinking of doing some work on the police and police officers in Ghana, initially we were thinking of working with an NGO at Transparency International, for example, and doing something really about the citizen side and drivers and information and complaints and of intervention awareness.

 

Danila [00:41:29] So it was started not thinking about training and thinking more of a bottom up, bottom up intervention. And then soon we realized that we could never work, do any kind of research on the police without the police knowing. So everybody told us in the field from, you know, did the NGOs to the enumerate or so the team were trying to recruit in the field that the first step for any kind of research on the police was to cooperate or collaborate with the police or seek a relationship with the police. So we were convinced that that was the way to go. Now, I would say a big reason for our success is that when our team members Henry Telli is our Ghanaian native and is the count economist for Ghanat the International Growth Center. So he was, you know, based on local and as well as our team leader or field manager that we we hire pretty soon pretty [00:42:37]soon [0.0s]  was there as well. I mean they were instrumental because we realized that in order to have a conversation with the police, we had to go in person. This could not be done by email or by phone in needed. I mean, there was a lot of talking done in person and many times through unannounced visits. And if anybody is interested in this kind of, you know, how how can you do work on the police in this kind of context is a very interesting article by Justice Tankebe, which is a criminologist, University of Cambridge, who specifically talk about the process of doing research with the police in Ghana and how important it is to just show up and try to get meetings on the day on this part rather than try to plan in advance.

 

Danila [00:43:31] Then sometimes you plan in advance and then nobody's there to talk to you. So there was a lot of, you know, kind of a surprise visit. I would say that we were lucky in a sense, because when we first approached the police, they were interested. They were talking at least of reforming the police and I know that this is common like many times, there are talks about reforming the police in this context, but the new inspector general, the post the police was really, you know, drafted a document that was this document talking about transformational agenda. And one of the priorities of the police was to improve perceptions of the police. So there was a vested interest in doing something to improve relationship with citizens or at least improve the way the citizens see the police. And so we came in in the scandal that there was 2017 believe it or not, a long time ago, it was fall October 2017, winter 2018, when we started talking with the police and in particular the Ghana police has their research and planning division.

 

Danila [00:44:43] So there were people there that understood research, like how our partner in the police at the doctorate. So there was an understanding of the research process and the importance of research. That was a good thing for us to be able to this to explain the importance of randomization, for example, and decided we couldn't go in and implement this program across the board. There was an understanding of the importance of just randomly selected districts to be recipient of the training. And also we told them, you know, we will we were going to test the impact of the intervention. We didn't tell them how or what we would do in terms of measurement of outcomes.

 

Danila [00:45:26] But we we told them that if we found a positive impact, we would definitely, you know, be willing to go back and train trainers to replicate and also train officers where the control group. And so that's still a plan. So we still plan to to do that.

 

Jennifer [00:45:45] That's great. So what data were you able to use for this project? You've already told us the administrative data doesn't really exist, so you couldn't get it at the very least. So what? What data do you have?

 

Danila [00:45:58] So we have survey data. So we conducted a baseline survey of police officers. The baseline was done in October 2018, which was several months before we actually implemented the training that was in the spring of 2019. This survey was designed in collaboration with the Research and planning division of the police, the Ghana police to the Accra, the Accra region. So it was a very comprehensive survey. They wanted to know about job satisfaction, for example, of officers managerial practices. So we included lots of questions just to prove to the police, to our partners, that we could be useful to them to quantify information that they were interested in having and so we the baseline was very comprehensive. But crucially, as part of the baseline, we asked about perceptions of corruption and unethical behavior. And we never went in and asked officers whether they engaged in this kind of behavior. This is common practice. In in this kind of sensitive survey regarding sensitive matters is not really ask them for their own experiences. But we did ask about their perception of unethical behavior or corruption or witnessing a one at fault behavior in their districts or among colleagues and our own reporting of corruption. So we have that information.

 

Danila [00:47:24] And we also asked why they joined the police. This is an important question because we as I said before, we we assumed at least some of them joined the police for the right reasons. And so we wanted to see whether that was the case. And also, we want to make sure that that motivation to join the police, that intrinsic motivation to join the police that we care about, is balanced across control and treatment districts and cross-trained and untrained AC service. So we asked about that. Also, we have them play an incentivized cheating game because we were afraid. I mean, we the survey was done in their place of work. So we visited the police district and we we were authorized to survey our officers during the week based on their i their duties. So that's what we did. But of course, we're concerned that they will now feel that they will feel observed by their supervisor, for example.

 

Danila [00:48:21] So not comfortable in telling us how they felt about behavior, the common behavior in their districts. And so we thought it was important to have some more objective measure of unethical behavior and especially something like, you know, individual propensity to engage in unethical behavior. So we have them play a cheating game and incentivize cheating game, which is a mind game. So cheating games, incentivize cheating games have been introduced, I would say about ten years ago. And they usually have to do with the rolling of a dice, although you will see at end line we went back, we couldn't then give officers the dice and so we had to do a different kind of game, but a baseline we did. So we we gave each officer a dice and a cup and they they were sitting private with some privacy screen.

 

Danila [00:49:18] And we asked them to think of a number between one and six and just have that number in their mind without telling us or writing it down anywhere. And then we asked them to roll the dice in the in privates and to write down on a form that we give them whether the number that they rolled was the same as the number in their in their mind. And so the important thing here is that there was an incentive to lie, because if they also say that the numbers matched, we will pay them 40 Ghanian cedi, which we estimated it would be the equivalent to about three and a half hours of work. So ten half times the hourly wage of police officers based on our baseline data. So kind of considerable amount of money, there is an incentive to say that the numbers match. Importantly, if you do say that the numbers match, but it is not true. It is not verifiable, right?

 

Danila [00:50:18] There is no way for anybody to know whether you are telling the truth or not. Right. So there is an incentive to lie. And the interesting or the exciting part of this is that you can really test whether there was lying or not by comparing the empirical distribution with the theoretical distribution. Right. So theoretically, we should see I mean, the probability of a match is one sixth, right? So the probably the two data, two numbers match is 16.6%. And so we can see whether the percentage of matches in the field is significantly higher than that. And also, we can test for balance in willingness to lie, to behave unethically. If you think of this as a measure of willingness to behave unethically, we can see if there is balance across treatment and control districts and trained and untrained officers are based at baseline.

 

Danila [00:51:15] And so what we find here is that 60% of officers across all districts with no significant differences, they say that the numbers match 60% significantly higher than their predicted theoretical prediction. Right. And so this is evidence of cheating. So cheating is widespread, much higher than what we see when we play this game with university students, for example, in that in labs. And importantly, there is balance. So 60% of control officers, 60% of officers that are trained that will be trained later on. This is a baseline. And 60% of all citizens from the treatment, these just who are not trained say that their numbers match. This is important for us to be sure. I mean, at least this is what convinced me that the assignment I mean, the participation in the training was as good as random in the industry because I didn't really see when I saw this data, when I when I saw this specific data, my worries about selection kind of went the way.

 

Jennifer [00:52:21] Yeah, yeah. These games are really neat because it really is. It's like in any individual case, there's no way you can tell if someone's lying.

 

Danila [00:52:29] No.

 

Jennifer [00:52:29] So that means that they'll be free to lie if they want to. But on average across you can compare group averages basically and then that reveals the cheating. Awesome. And then what's the cheating game that you played? Well, I guess first, tell us about the survey that you conducted later.

 

Danila [00:52:45] Yes. At end line so we were supposed to go back a year after the training and replicate basically that baseline survey, one year after the training, a year and a half after the baseline. However, COVID happens exactly when we were training enumerators in the field and we waited a while thinking naively that COVID would be over by summer.

 

Danila [00:53:08] As we know, that was not the case. And so at the end we decided to just change our data collection strategy and collect data by phone. And so we had to redesign our survey instrument to make the survey much shorter. And to be able to be implemented by phone. And so we also had to change the game. But in any case, we had very specific ideas with respect to which outcome variables we wanted to measure. And so in in our survey wnd line survey, we really focused on asking questions that would allow us to generate some indexes of attitudes of officers and behaviors. And also so we especially were interested in any kind of change due to the training, any change in identity or beliefs. So we we care about officers telling us that they see themselves as service providers rather than crime enforce.

 

Danila [00:54:06] So law, crime prevention, law enforcer. We care about them telling us that, for example, they're the most important quality of police officer is honesty and professionalism. We care about them telling us that organizational norms can change, due something that we really acted upon the second day of training. So we ask specific questions that would allow us to construct indexes of attitudes. So we aggregate these these answers to these questions to generate five outcome indexes. The most important one for us, the most important, as in the one that we believe is more closely linked to the content of the training is what we call a value index. And we also ask questions about reporting of corruption and monitoring of subordinates and perceptions of corruption. And we also have a measure of relationship with citizens, because we have a full module of the training that was really about communication skills, how to talk to citizens after happening, for example.

 

Danila [00:55:11] So we have also an index of that. So we have five outcome indexes based on our survey measures. And then we have a game again. So we wanted to do a cheating game. However, we we had to change the game because we couldn't give them a dice. Now it's on the phone. So we say to do a coin toss game. So this coin toss game is taken by people, by arbiter source. And so this basically asks participants or as participants to get a coin. And everybody had won and tossed the coin four times. And tell us how many times they got tails. So now they could answer between zero and four. And what we can do, again, is to compare the empirical distributions or the percentages of officers who told us zero, one, two, three and four. We did two radical distribution, which is, in this case, a binomial distribution.

 

Danila [00:56:07] And we can see, first of all, if absent the training, there is cheating as measured by significant difference in the two distribution, the empirical time and theoretical. And then more importantly, we can see whether the training shifts the distribution. And so if for trained officers in treatment districts, we see that the distribution suggests less cheating or is closer to the theoretical distribution, which is exactly what we do. So in that, in the empirical section of the paper, we have game generated outcomes in. So we are looking out particular the likelihood of have of saying that you got more than two tails and also the likelihood of of declaring that you got the three, the number three because we it turns out that most people say that they got three tails that.

 

Jennifer [00:57:01] They're not so.

 

Danila [00:57:03] Yeah. They don't say four--.

 

Jennifer [00:57:07] That'd be too obvious.

 

Danila [00:57:08] That'd be too obvious but three is the mode in for the contro, but so we can see whether that changes and we find that it does significantly. So these are outcome variables.

 

Jennifer [00:57:18] Okay, awesome. All right. So let's talk more about these results. So what do you find is the effect of your ethics training program on police officers survey responses?

 

Danila [00:57:28] So we find a significant impact on the values index. As I said, that we have five indexes, so not all of them were affected. There will be too good, but also too suspicious, I would say, you know, if everything worked out perfectly, but we do see an impact on the values index and in the Active Resource Citizens Index-- the Relationship Index.

 

Jennifer [00:57:51] Yeah. And remind us how long after the training this end line survey took place.

 

Danila [00:57:57] Right. This was done 20 months after the training and more than two years after the baseline.

 

Jennifer [00:58:04] It's been a while.

 

Danila [00:58:05] Yeah, it's been a while. And also after the award ceremony. So the award ceremony was done in December 2020, no 2019, sorry, the end line was December 2020 after they were ceremony, we disappeared. So we there was no contact anymore with our field team or we research. So that was basically that was it.

 

Jennifer [00:58:28] Did they still have the WhatsApp groups going? Or was that over?

 

Danila [00:58:30] Yeah, the WhatsApp groups are active. I mean, they were active, but we don't I mean, we could look at the data. We are not we are not monitoring those those apps. By design, we wanted them to feel completely unmonitored. So we don't, we are not tracking on them. But so those are still there. So potentially maybe they are still they're still going, but they didn't hear from us in the sense that we didn't contact them or we didn't in any way. You know, none of our fields enumerators showed up anymore.

 

Jennifer [00:59:00] Yeah.

 

Danila [00:59:01] Registries or anything like that. So it's been at least a year since hearing about the program or us or anything related to the training.

 

Jennifer [00:59:11] Yeah. And then what was the effect on the cheating game outcomes?

 

Danila [00:59:15] Yes, the cheating game. So first in the paper, we showed the distributions. And so we see that for the control group and the untrained officers, the distribution of these, you know, coin tosses, this rolls of the coins shows evidence of cheating. So the distribution of kind of, you know, skew to the right and they say the mode is three is not to as it should be according to the binomial. And so there is evidence of cheating. And then we see that with the training, the distribution is not significantly different from the theoretical distribution. So it shifts back to look much more similar to what the theoretical distribution should be. And also in the regression analysis, we looked at the likelihood of, for example, reporting the number three. And so this is, you know, 45% of officers in the control group reported a three. It should be 25% theoretically, and the slightly overreporting at three is reduced by 13 percentage points. So it's kind of a big impact if you look at the baseline of 45%.

 

Jennifer [01:00:20] Yeah. So this is your sort of like real world or like actual behavior outcome in a context where you didn't have like we've had this conversation a few times, like what you really want is, you know, how often are they stopping busses and taking bribes? And so absent that data, this is sort of like--

 

Danila [01:00:40] The closest.

 

Jennifer [01:00:41] The closest, yeah. And it is a really clever and creative way they try to get information on like real behavior. Right. And yeah, it's not, it's not like what you'd ideally want out in the field, but it is way better than just a survey.

 

Danila [01:00:58] Survey.

 

Jennifer [01:00:58] Yeah. Yeah. Okay, so you do a bunch of robustness checks. The one I wanted to highlight and give you a chance to talk about a little bit is looking at the potential spillovers to untrained officers. So why were you interested in this and what did you find?

 

Danila [01:01:13] Yes, we were hoping in a sense that there would be spillovers. Right. And that the trained officers could also somehow influence the behavior of other officers in their districts where coworkers were not trained. We find no evidence of spillover effects, so the untrained officers in the treatment is just completely unaffected by the training for all of the outcome variables. And so this suggests that really, you know, you had to be there, basically. So it's it's not as I said, this was a training that was extremely interactive and there was a lot of brainstorming and activities. And it really acted, we believe, on this individual identity and group identity. And so what we learned from this is that really participation is necessary. Active participation is necessary for officers to internalize whatever the content of the training was.

 

Jennifer [01:02:15] Yeah. And also just strikes me, I mean, my hunch I'd be curious what your hunch is. My my hunch is that these WhatsApp groups are important and like just finding other people who are also interested in sort of like creating change in the same way you are who aren't necessarily working next to you feel super important. And so that's something that those untrained officers just don't have access to, right?

 

Danila [01:02:39] No, I know exactly. And yeah, as I said, I mean, we designed the second day of training the agent of change identity training to be super district.

 

Danila [01:02:49] It was not about it was it was about being an agent of change and being, you know, again, a police officer. So we never emphasize the district element. So it is indeed possible that they are really focused on who was trained, you know, with them. And they're talking to in their in their WhatsApp groups rather than trying to change attitudes or behaviors of their coworkers were not trained.

 

Jennifer [01:03:14] Yeah. Okay. So what are the policy implications of these results which it policymakers and practitioners take away from all this?

 

Danila [01:03:21] I want to believe the policy implication is that change is possible. Even in in environments where it seems impossible. And so, I mean, I, I was very surprised by the results myself, because we as I said, we discussed a lot about how difficult bringing change in this environment would be. But there is also the the survey 20 months later, so along time later about this survey data and they incentivized cheating game data suggests that the training did do something to this officers. And so I think we come back to what we discussing at the beginning. I think there is there needs to be more research done on training. It's unclear, of course, how much of this impact has to do specifically with the content of the training being about the individual identity, the group identity, but all I can say is that this calls for for more research, for more funding. And we say that with this kind of research, because I see potential in the impact of these kind of trainings.

 

Jennifer [01:04:27] What did the Ghanaian police force that you were working with? Have you talked about these results with them? What was their response?

 

Danila [01:04:34] Yeah, we talked and we went and present the results and we are in contact.

 

Danila [01:04:39] We we want to keep a good relationship also because we were trying to get some data from there, from them, understand if the data exists.

 

Jennifer [01:04:46] Mm hmm.

 

Danila [01:04:47] And so I think that we're positive about it. I mean, again, one of the problems with these agencies in developing countries and certainly in Ghana is that there is a lot of turnover at the leadership, like in leadership. So there's been, once again, a leadership change within the police and so whenever there's leadership change, of course, you have to start all over, right.

 

Jennifer [01:05:08] Right.

 

Danila [01:05:08] Or recreate in the relationship and explain to them what the program was about and the importance of the program. So it hasn't been as straightforward in terms of policy implication of or, you know, replicating or anything like that, as I would have imagined, I would have hoped. But we're still talking with them that I'm hopeful that we can keep the relationship going and go back and train the other officers.

 

Jennifer [01:05:32] Yeah. Are there any other papers related to this topic that have come out since you first started working on the study back in 2017?

 

Jennifer [01:05:42] That's interesting, Jen. Just yesterday I was at your vice seminar. Yes. Yes. And so you tell me, I tried to learn this literature, but there is this interesting study of this training program on racial diversity training in Texas that I was listening to as of yesterday. So I'm sure there is more work that is being done. I'm not sure how much of it is in the US, how much of it is in the in the Global South. I'm sure that there are really, you know, smart people doing research on this topic. And so I'm just trying to make my way towards reading as much as possible about what people other people are doing.

 

Jennifer [01:06:26] Yeah, it is a hot area these days for, for good reason. Yeah. That that seminar yesterday was by Matt Ross, who is on his way to Northeastern University next year. He has this cool, this cool paper on culture to cultural diversity training, and he has other work with CarlyWill Sloan on also looking at like field the effective field training. So yeah, there is lots of smart people out there finding ways to get at some of these questions, given how little we know right now.

 

Danila [01:06:53] Yeah. Yes. Yes.

 

Jennifer [01:06:54] And so what's the research frontier? What are the next big questions in this area that you especially and and others will be thinking about going forward?

 

Danila [01:07:02] I think for us, you know, one of the limitations of this study, besides the data limitation that we talked about, one of the limitations is that we cannot really pinpoint the mechanism of the chain. Like we we focused on many different things, during the training and particular individual identity and the creation of a new group identity and acting on beliefs and norms. And so it would be great. I mean, I would love to be able to disentangle the impact of the different components of the training and see what is most responsible for the positive impact on on attitudes and behaviors. With the current data we can. Right. And the same with, you know, you talked about the what's the WhatsApp group and we talked about the other important thing is to signal is visible signal that these officers are different. This pin that we gave them the agent of change pin. And that's another important component potentially that could have constant I mean, the idea was to generate a reminder of the training and making them feel that, you know, whenever they are tempted to do something, kind of be reminded of what they committed to to do when they are committed to be.

 

Danila [01:08:12] And so for me, from a research perspective, but also from a policy perspective, would be really great to be able to disentangle the effectiveness of these different components of the training and. Another thing is just know this. This study was very small scale. We involved a few hundreds officers. And so I would be really interested in seeing how and whether it scales up. And so definitely, you know, trying to do more about scaling up and also understanding mechanisms better.

 

Jennifer [01:08:43] Yeah. So anyone out there wants to implement this training in their police department email Danila. Excellent. Well, my guest today is Danila Serra from Texas A&M University. Danila, thank you so much for talking with me.

 

Danila [01:08:57] Thank you so much Jen for having me.

 

Jennifer [01:09:04] You can find links to all the research we discussed today on our website probablecausation.com. You can also subscribe to the show there or wherever you get your podcasts to make sure you don't miss a single episode. Big thanks to Emergent Ventures for supporting the show and thanks also to our patrons, subscribers and other contributors. Probable causation is produced by Doleac Initiatives, a 501(c)3 nonprofit, so all contributions are tax deductible. If you enjoy the podcast, please consider supporting us via Patreon or with a one time donation on our website. Please also consider leaving us a rating and review on Apple Podcasts. This helps others find the show, which we very much appreciate. Our sound engineer is Jon Kerr with production assistance from Nefertari Elshiekh. Our music is by Werner and our logo was designed by Carrie Throckmorton. Thanks for listening and I'll talk to you in two weeks.

 

Jennifer DoleacRCT, Policing