Probable Causation

View Original

Episode 70: Pedro Souza

Jennifer Doleac

Pedro Souza

Pedro Souza is a Senior Lecturer in Economics and Finance at Queen Mary University of London.

Date: April 26, 2022

A transcript of this episode is available here.

See this content in the original post

Episode Details:

In this episode, we discuss Prof. Souza's work on the effects of police body-worn cameras in Brazil:

“De-escalation Technology: The Impact of Body-worn Cameras on Citizen-police Interactions” by Daniel AC Barbosa, Thiemo Fetzer, Caterina Soto and Pedro CL Souza

OTHER RESEARCH WE DISCUSS IN THIS EPISODE:

transcript of this episode:

Jennifer [00:00:08] Hello and welcome to Probable Causation a show about law, economics and crime. I'm your host, Jennifer Doley at Texas A&M University, where I'm an economics professor and the director of the Justice Tech Lab. My guest this week is Pedro Souza. Pedro is a senior lecturer in economics and finance at Queen Mary, University of London. Pedro, welcome to the show.

Pedro [00:00:28] Hi, Jennifer. Thank you so much for having me. It's a pleasure to be here.

Jennifer [00:00:31] Happy to have you. Today, we're going to talk about your research on body worn cameras for police. But before we get into that, could you tell us about your research expertize and how you became interested in this topic?

Pedro [00:00:43] Yes, of course, Jennifer. So I'm interested in social networks in a kind of in a very wide sense. So I'm interested about how human interactions shape economic outcomes. And in essence, kind of the cameras are not specifically a network paper, but it's also an intervention that shapes how police officers and the citizens interact. So that's how we kind of became interested in how the cameras would affect the police citizen interactions and it happened that I was in the right place at the right time and I had the right implementation partners. And I happened to have this amazing team of coauthors by Daniel Barbosa, Caterina Soto and Theimo Fetzer, and we all put that together. So that's how it came to be, came through an angle of social networks. And through that I became interested in the police activity and it being in the right place, the right team, the right partners to implement that project.

Jennifer [00:01:37] Awesome. So your paper is titled "De-escalation Technology: The Impact of Body Worn Cameras on Citizen Police Interactions." As you mentioned, it's coauthored with Daniel Barbosa, Theimo Fetzer and Caterina Soto.

Jennifer [00:01:49] And in this paper, you run a large randomized trial of body worn cameras with police officers in the state of Santa Catarina, Brazil. So tell us a bit about the context there. What a crime rates like and what are police community relations like in that state?

Pedro [00:02:03] Should I first say that that Santa Catarina is a beautiful place. Jennifer [00:02:07] Yes.

Pedro [00:02:07] It has an amazing, amazing place to visit. And it has you know, basically, it's an amazing nature and it's amazing hikes. So if you're a fan of kind of being outdoors, that's a place to go. But Santa Catarina, it's a relatively wealthy state in Brazil in a middle income country such as Brazil. It's a relatively safe as compared to a relatively unsafe country. And the crime rates in Santa Catarina is still like three times that of the US and five times that of the UK and with a very substantial within state heterogeneity. So different places in Santa Catarina are quite different in terms of kind of policing challenges and the crime rate. So you're going to get some places that are like super, super safe and some places that are not. It's a relatively small state, but in a huge country, so it still has kind of 1 million inhabitants.

Pedro [00:02:57] It's comparable to the whole population of Portugal or Israel. It's a relatively geographically small state, but in in a big country. So if you take the state east to west, it's almost the distance between Edinburgh and London, a bit less than perhaps Paris to England, but it's still quite, quite large. So it's a hugely heterogeneous population as well. And the community police relations as compared to Brazil are generally okay, certainly not as bad as in other states like Rio, but there's, you know, from from other research that I've done, we have to observe that 50% of of the inhabitants, if you're walking around in their municpalities fearing being victims of crime, 40% of our respondents still felt that the police was corrupt and 20% didn't trust the police at all.

Pedro [00:03:48] So I wouldn't characterize as a good relation with the police. I would characterize it as perhaps better than average in Brazil, but with substantial policing challenges that remain.

Jennifer [00:03:58] Okay. So when you and your team first started talking with policymakers about this experiment and talked with them about why they were considering implementing body worn cameras, what were the main goals? On the policy side, were they primarily focused on police use of force, which we typically talk about with respect to body worn cameras in the U.S.? Or was it something else in Brazil?

Pedro [00:04:21] Sure. So I don't think there was a single goal. I think there was a multiple kind of interest that they had in implementing this project. You know, I remember getting there and speaking to the police officers for the first time, you know, the high ranking police officers. And I was telling them, like, look, I've come to understand whether it works or not. We should randomize and then interrupted me and said, No, no, no, we already know we need to randomize to know whether it works or it doesn't. And I kind of I fell in love.

Jennifer [00:04:47] That's amazing.
Pedro [00:04:49] So it's been very interactive in that way, right. Jennifer [00:04:54] Yeah.

Pedro [00:04:54] So that you get the buy in to randomize. So they had a genuine interest in knowing whether cameras worked or it didn't work. In principle, I think they had the right motivations, which was to kind of make efficient use of public money. They want to invest and they want to know whether that investment is worth it. So in terms of kind of the dimensions that they were looking for, first, they were looking at use of force as well, but they often relayed the idea that the cameras were a protective device.

Pedro [00:05:22] And that's also true that being a police officer can be dangerous job. And having a camera is also a protective device to protect the police officers and that was a big part of also what they had in mind when implementing the cameras they had already lined up where they were at the beginning of the discussions. They were planning to deploy the cameras in large scale. So eventually it did happen. So they had 2000 cameras across all the state. So our project was, in a way, testing whether this large scale intervention would have any prospect of actually working and delivering the results that they were expecting. So I guess it was kind of this moment in time where they were genuinely interested in whether it works along with the potential scale up and it actually didn't inform the scale up. So it did happen in the end of the day.

Jennifer [00:06:14] Okay. So some combination of trying to protect citizens and trying to protect the police from violence. And so what are the various ways we might expect, you know, cops wearing a camera to effect those outcomes? What are the mechanisms here that we should have in mind?

Pedro [00:06:30] Sure. So I think when when the camera the camera's there, a number of things can change, but the two main avenues we kind of already touched on violence and use of force. I would rather think it as an equilibrium outcome rather than a one sided outcome. But it's both officers complying with the protocol for use of force. We have to define a little bit what's excessive use of force relative to the protocol, which is yet another dimension to discuss. And there is the other side, which is citizens internalizing that kind of whatever actions that they take are going to more likely have repercussions.

Pedro [00:07:08] These actions can be against or not against the police officers because it's an equilibrium outcome it's not very easy to disentangle whether it's coming from one side or another. We have some indication and to be completely frank, I think the two sides are the two aspects are relevant and I'm pretty sure that the police side of reducing use of force is very, very relevant to the force. But also there are a number of other potential kind of channels and mechanisms through which kind of introducing a camera may change how this interaction unfolds.

Pedro [00:07:37] So for example, the cameras increase the capacity to recover evidence, let it be directly, because of course, the camera records what happened, but also indirectly as the camera forces the police officers to review the forms properly and make sure that whatever documentation, pictures and whatever evidence is gathered in the right way from the scene where the crime happened. It also has the potential to run, in the paper we look at those effects and we find no effects, but it would have the potential for, say, effect efficiency measures can officers respond more quickly because they have the cameras. They cannot just slack off. They have to respond to some call. There could be the potential also to the patrol routes being digitally changed so that, you know, you have a camera. Now, I want to patrol the easy parts of the city. We don't find any any evidence of that, but it could change way more than that just specific interaction. And of course, there is a big question here on whether the the videos that are recorded, the evidence that more generally is recorded from the crime scene, what is the that effect on the criminal justice system afterwards? So there is a number of changes that are happening.

Pedro [00:08:50] We try our best to map them and I think there is quite a bit of research yet to be done, although I do think that our results are quite compelling in descending on what has been documented already from from that experience.

Jennifer [00:09:03] Yeah. So expecting consequences if you are misbehaving either on the police or citizen side requires kind of a couple of steps here, right? So you've got like this objective footage or like an objective third party, like always watching and recording what you do and then as economists we'll sort of the next step we would expect then is that there are consequences if you don't do what you said you did or don't do what you were supposed to do. And so I guess this leads to a question of exactly how these are implemented and used in the police departments you're working with. I know this is a big topic of conversation, like the actual implementation in the US context. Are there people reviewing the footage from body worn cameras in these departments or what kinds of consequences are the police expecting? Do you have a sense of that?

Pedro [00:09:53] So that's that's a very, very important question. So I used to say that the cameras, they are not a monitoring device. They just open a window for someone else at some other point in time, in some other location to monitor that interaction so that the expectation that someone else might observe that interaction and that's the key element here. And whether that person would do something with that recording should it need to be done. Right. And the key a key element and in our case, we knew that the the recordings, they were accessed by the higher instances in the police. So if there was a complaint against a certain police officer, they would be able to map that into an actual dispatch and look whether the recording took place or didn't.

Pedro [00:10:39] And the criminal justice system had access as well to use it as proofs, irrespective of whether the action was perpetrated by the police officer or not or against the police officer, but it could be used as evidence. And finally, it did happen that the police units that had the cameras installed, they were using randomly reviewing some of the videos and using them for training, for internal training, so they would review in data sessions, sometimes a little bit less than they did. But they would review with the police officers a very open spirit with a police officer to say what did go right or didn't go right or should be fixed, and how you engage with the citizens and so on. So there was this training element here that the recordings kind of subsidized quite a bit of the training of the police officers.

Pedro [00:11:26] Of course, this was also specific to some police officers and some commanders that decided to make use of the recordings in that way. It's also a unique kind of way of using the video recordings that I would hope that other police departments could also make use of.

Jennifer [00:11:44] Yeah, that's really interesting. Okay, so before your paper, what had we known about the effects of police body worn cameras?

Pedro [00:11:52] So we knew that the body worn cameras didn't have much effect. Jennifer [00:11:56] Yeah.

Pedro [00:11:57] So there were a number of experiments in the developed world. So we had Rialto in California, Birmingham in the UK. These experiments had not found much effects from a number of conversations. I knew also that these studies were trickling down to justify why cameras should not be adopted. So it was fitting to the policy debate. We were not happy with the status of the researcher due to a number of reasons.

Pedro [00:12:21] One is kind of the context in which these cameras were implemented, and of course, their policing challenges are everywhere, but they're not so comparable with the policing challenges just in the high crime scene like Brazil. And we are also not comfortable with the research design where we thought for a number of reasons the research designs would not be able to capture the true effects of the camera. So, for example, we we thought that the previous studies did not account properly for contamination. So if you were to compare treated and control groups, for example, treated in control officers, you have to take into account that they are in dispatch groups together. So if you and I are in the same dispatch group and you have a camera, I'm a control officer, but of course, I'm going to change my behavior because you have the camera as well. So we need to be able to observe that interaction and who composes the dispatch group in order to tease out what what are the true effects by comparing treatment control

with defining a control group that is not affected even directly by the camera implementation itself?

Jennifer [00:13:26] Yeah. And just to clarify here, these are all RCTs for the most part, which is sort of amazing. So what you're talking about, what sort of confounded effect isn't the standard issue we have where, you know, finding a comparison group is hard. These are these were all randomized trials, but there are these sort of these questions about the extent to which the treatment effect spilled over to the control group and then whether that could bias things towards zero.

Pedro [00:13:51] Exactly. One challenge is exactly, yes, it is very rightly so. It's finding the comparison group. But another issue is comparing what to what we aimed at, comparing the outcomes at the level in which the interaction unfolds that's the dispatch cannot get more granular than that. So while previous papers that were comparing the outcomes, say, at the officer level or at the shift level, of course these can potentially manifest the treatment effects, but these are all indirect observations as opposed to looking at the data that most closely records how officers and citizens interact, which is the dispatch level. By looking at that granularity of data, we can also do a number of things so we can control for, say, time variation. We can control for a number of kind of exogenous factors that make our make our lives a bit easier in a statistical sense, because we get more and more power to identify the treatment effects.

Jennifer [00:14:53] Yeah. So I do want to give all of these other researchers the credit they deserve for running all these impressive RCTs. I mean, you know, this is a rare space in the criminal justice policy landscape where we actually have a bunch of RCTs on a specific type of intervention, which I mean, from a research perspective is just amazing. And as you're pointing out, even a place where we have at least a over a dozen RCTs in various cities, so there really have been a bunch in the U.S. and the U.K. randomizing at different levels, randomizing at across different officers or across different shifts. And each of those randomization levels is trying to avoid certain types of spillovers. And they're looking at citizen complaints and they're looking at use of force rates. And yeah, just across the board is just like null effect, null effect, null effect, null effect, which has been really interesting. And so, you know, as we continue talking about your paper, you're going to find something very different.

Jennifer [00:15:55] And I think you make a compelling case that you're measuring the effects in a cleaner way, but there is also this, you know, we talk about kind of why this is hard. All of those the researchers also knew that there was the potential for spillovers. But I think a lot of times it's difficult to get the level of buy in that it sounds like you all had in Santa Catarina. And maybe just differences in data access. I don't know what are what do you think? Like as you guys were approaching this question and you had all these other RCTs in mind and you're saying, you know, I'm just not quite convinced that this is telling us what the causal effect of body worn cameras are, especially in a context like ours. What were the main challenges that you felt like you had to overcome? Is it mostly about like randomizing at the right level where you really focused on getting better data than the other researchers had? What were the hurdles there?

Pedro [00:16:47] Yeah, no, of course. So I mean, the previous studies, they they were ground breaking and trying to study this question and their multiple constraints randomizing something with the police is no easy task.

Jennifer [00:17:01] Yeah.

Pedro [00:17:01] And getting them to randomize in a way that you can properly map all this these effects even less. We did have a lot of strain, especially at the beginning of this period. Then it changed over time a little bit, but for the implementation of this project, we did have a lot of strain and we had multiple, multiple rounds of conversation. I went there to Santa Catarina, I don't know how many times, but like something between seven or ten times to refine and refine an implementation plan such that it would be feasible, such that we would would identify the treatment effects that we would be able to comply with target or idea analysis and control for spillovers and so on and so forth, given the food constraints that they have in terms of control operations.

Pedro [00:17:46] So I couldn't devise an implementation plan that would of course suit their modus operandi in a way. And then it ended up being the simple interpretation plan where we would randomize across officers and days. Ideally, we would randomize whether certain dispatches are treated or not, but that's not feasible. And the closest that we could get from that is the combination of randomization across offices, and that allows us to get as close as possible to the randomization across across dispatches and to have very good data access that would allow us to map. So for us, the unit of treatment, as I said, is the dispatch, but the we considered dispatches being treated if at least one police officer was wearing a camera, irrespective of who that officer is and then we later look at heterogeneity. For example, we find that if junior officers are wearing the camera, then compliance is higher for a number of reasons that we think, for example, junior officers are more likely to adjust to new technology there are concern, though, all very consistent with that. So there was this two sides. So I think there was the strain from the police, which was very good. There were very dedicated police officers to try to kind of really find out the effects of the camera and willing to fully do what they could do to provide an environment that was a suitable as possible for a kind of a quality research and to open up the doors in terms of data, we do have some very granular data. We don't have any data that identifies any single individual. So we don't observe the videos themselves. We don't observe the names of any of the victims, and we don't observe the names of the police officers.

Pedro [00:19:31] But apart from these very reasonable concerns about individual privacy, we pretty much got data for what we requested initially. So I think it was a combination of two things. It was the treatment design, the buy-in and the quality of the data that allowed us to overcome these hurdles that potentially other papers that faced.

Jennifer [00:19:56] Yeah. And so you worked with several police departments in Santa Catarina to implement this big RCT. So tell us a little bit more of the story here I think it's always fascinating to people exactly how these RCTs come about. So how did the individual police departments get involved? Tell us a little bit more about kind of what those conversations were like and how this all developed.

Pedro [00:20:20] Sure. So the police in Brazil, they are organized at the state level. So we got in touch with the Central Command for the state level police, and that happened through our implementation partner Igarape institute. They were at the time developing a software called Cop Test. It's a very interesting piece of software that would transform mobile phones into body worn cameras. So it's kind of an app that would allow police officers back into headquarters, not just to have the live feed, but to enable two way communication with the police officer. There was a really interesting piece of software, and then there was a dashboard where they could see where the police officers were. So it's an integration of kind of a body worn camera with a management system of the videos and also the real time kind of captured device. So they were in communication with the Ministry

of Santa Catarina. It's a really neat piece of software. It was a fantastic. So there was this conversation going on at the time.

Pedro [00:21:19] I joined this conversation and we said, like, why? Why don't we randomize and why don't we see the effects that that might have? So that spiraled into the project. Eventually, we changed the technology for a can of regular body worn cameras because the mobile phones were not a good, good hardware for the police day to day activities the battery wouldn't last is not sturdy enough. So they had the best software is still there, but the hardware wasn't proper for the police operations. So that's why we eventually shifted full on to the body worn camera. So that's how this conversation happened. So we eventually got funding and we were there, we're there ready to start a lot of work. I mean, I'm saying that it's a lot more complicated than it sounds because --

Jennifer [00:22:06] I'm sure these things are always so complicated, like when you're it's just the logistics of pulling something like this off.

Pedro [00:22:13] Yeah, the logistics and a lot had to be adjusted within the police. So the operating protocols that the police know how to interact with the city, it has to be adjusted. So there was you know, also the training had to be adjusted. So there's the training department. There is all the legality around that and how to deal with the videos, what can and cannot be done. So there was no the sort of law department involved. There's, of course, the I.T. department that needs to handle that amount of cameras with. In our case, we had 73 cameras, six docking stations, but the production of lots of and lots of videos. It's not easy and then we need to have a system in place to make sure that if we need a video, we know where it is and we can pull that video.

Pedro [00:22:58] So I.T. department had to get involved as well. Plus the commanders of the different police units that that implemented the project and, of course, the participating police officers. So at some point, I felt like it was speaking to everyone in the police for different reasons, for different aspects of the implementation. And they were they were very kind. I think in the end of the day, it's not just no interest in making sure that the research works, but also I think that they were generally interested in knowing the effects of the body worn cameras and also using the project as a way to prepare themselves for a much larger scale intervention that followed. So it all came together at the right time.

Jennifer [00:23:44] And had this organization done other like the police commanders you were working with, had they done other RCTs in the past? Is that how they knew going in they would have to randomize in order to note the effects here?

Pedro [00:23:55] They've done another RCT with us on community policing. We found no effects of community policing, so we created community policing groups randomly across the state was a it was a design in which we advertised the community policing program. It was along with other groups. So it became also kind of a meta study on community policing, where other groups in different settings around the world also found no effects of community policing. So I think behind that there was a huge selection of story in the sense that we think that community policing may work very well and the feedback is always so positive, but the selection of participants in two community policing groups is not random to an extent. There are people that are much more likely to interact with the police in the first place, to have an excellent good perception of the police.

Pedro [00:24:48] And that's what know in a sense that drives our perception that police that the community policing is relatively efficacious. But in doing so, the project so we had

done this RCT before, I think they were the own studies had shown them that that randomizing things was a good way of assessing of assessing treatment effects. And so so yeah. So there was this kind of prior knowledge or perhaps, you know, good experiences with randomizing things.

Jennifer [00:25:18] Yeah. And experience within RCT that showed something they thought work didn't work, which presumably helps them understand why research is important if they weren't already fully bought in. Very cool. Okay. Well, tell us more about the specific design of your experiment. So you have all this buy in you're able to randomize at these levels that other researchers aren't able to do randomized. So exactly how did you assign these body worn cameras across officers and across shifts?

Pedro [00:25:45] So we selected one in three officers to assign a camera. So we selected all the front facing officers that were in the police units that were selected into the study. And overall, we had 453 officers, so 150 were assigned to use the camera. We did one in three because we knew we have the police officers they combined themselves into dispatch units. And our goal was to compare dispatch units treated in against control and we wanted to maximize power. So we want to have 50% or as close as possible to 50% of the dispatchers that are treated. And to do so, we need to induce this person treatment at the officer level. Our pre data indicated to us that it was one in three or close to one in three, so that's what we did. In addition to that, we had two in seven, so two days in seven in the week were called the blackout days.

Pedro [00:26:43] So they serve two purposes. So we can compare kind of within officer variation. We can, for example, estimate learning effects whether in an officer that used a camera in the past behaves different even in the absence of the camera, we find some evidence of that. And it also allows there's a very clean setting that can use kind of contamination effects because during those days, no one in the police units were using the camera at all. So we rule out any contamination, even if we were not able to directly observe which we are. So it allows us to do robustness exercises and we don't find different effects.

Jennifer [00:27:17] Okay, so you've got one out of every three officers is assigned to wear a camera all the time except these blackout days. So all the time. And then you figured out that basically that would give you about half of all incidents where officers are showing up. You'd have at least one officer on the scene who's wearing a camera. So the incidents basically treated. And then you've got these blackout days where no one wears a camera for two of the seven days. And I have to say, this is the piece of the paper. Like when you get to the part of the paper, it's just like, how do they pull that off? Like, that's the part of the implementation that seems the hardest to me of actually like getting everyone to remember not to wear their camera on those days. So very cool. So you've got randomization across officers and also randomization across days. Excellent. And then tell us about the data. You have to measure all this.

Pedro [00:28:05] At that point, we had kind of we literally WhatsApp groups that not only we had two and seven days that were the blackout days, but we didn't want to announce them beforehand. We wanted to announce them--

Jennifer [00:28:16] Right.

Pedro [00:28:17] Lastly to avoid any anticipation effects. So we literally had the police officers, the commanders and in in WhatsApp groups. So we were literally sending

messages to them. So tomorrow is a blackout date or no cameras and so that was literally at how it happened.

Jennifer [00:28:32] Amazing. It just feels like there aren't that many organizations, police or otherwise, that would be able to pull something like that off. So it's just remarkable.

Pedro [00:28:40] I agree.
Jennifer [00:28:40] All right. Tell us about the data.

Pedro [00:28:45] So looking back to the data. So it's the special level data. So we observe. So dispatch produces kind of a police unit going somewhere to do something. It can be initiated through 911 calls, or it can be self-initiated by the dispatch units that are patrolling the municipalities, it records, what happens or what type of crimes are suspected or confirmed, who was present, where, how severe the call is, whether the use of force was applied. The use of force can be, in theory, can be anything. Even a command police officer commanding someone to do something can be use of force. Having said that, the use of force that is recorded is that of firearms being shot or non-lethal weapons being being shot, but in theory, anything can be classified as use of force. And the full data, we don't have that data, but the full data would have like textual transcripts of of what happened, descriptions of what happened, photos and so on. We don't have this data for, as I said, to protect the anonymity of the human subjects, but our data would record what happened, when and whether there was any outcome in terms of use of force, whether there was any arrest and so on.

Pedro [00:29:58] And in combination with that, we have the automated logs of the cameras. The cameras are kind of mini computers, so they have kind of a system messages of what happened when they were turned on, when they were turned off, when they were in standby, when the video went and they were recording when they were not, whether it was kind of, let's say, low battery alerts and things like that. So that data we get, we don't have the videos themselves, but we know whether a certain camera that was being used by a certain officer in a certain place at a certain time, it was on or was off.

Jennifer [00:30:31] Okay. And so there are a bunch of things you could measure here. So what are the main outcome measures you're most interested in?

Pedro [00:30:37] So we start with compliance. We find that about a quarter of the dispatches are recorded, which can be consistent with some protocol for use. That's another thing that's very important here. So the protocols used was that the cameras should be turned on whenever there was interaction with a citizen. Not all the calls have interactions. They can, for example, look for something and not find or look for someone and not find, and so they can be roaming and not have a meaningful interaction. So that was the first outcome. The second set of outcomes is whether what happens to the reporting margin, whether police officers, the recorded instance more often, how the recording changes and whether there, for example, more victims in the reports. And we do find evidence that they referred to investigations about 9% more often those reports have about 20% more victims. That's compared to the dispatchers where there was no camera present.

Jennifer [00:31:33] So tell us a little bit more about that. So this is basically just trying to figure out if the cameras are affecting like how diligent the officers are at doing their jobs. Is that the idea?

Pedro [00:31:45] Yeah, we think that this is improving the quality of the reporting. Importantly, we find that the incidents of domestic violence grows by 67.5% of course we don't think that it's the domestic violence that is increasing. That's the reporting of domestic violence that is increasing, that could be reported as anything else but domestic violence. So it increases by quite a substantial amount. And we think that this is just the effect of the cameras either directly providing evidence about domestic violence or indirectly through police officers being more diligent in reporting the facts as they are. And this is incredibly important, I think, because otherwise it would just fall into the blindness of not having the right set of public policy tools, not victims themselves, not being able to be protected, because basically the crimes were not being accurately reported by the police officers. And that's a very significant and meaningful increase as we seen the data.

Jennifer [00:32:49] And then the last outcomes that you look at are the effects of these body worn cameras on citizen officer interactions. So what do you find there?

Pedro [00:32:58] So we find that, as I said, like 25% of the dispatches are recorded, but that's that's, you know, of course, compliance not perfect, but is also some part of that is explained by the fact that the protocol did not mandate that there was a recording at the beginning of the day, that whenever there was no interactions between the citizen and the police officer, we find that compliance is higher when junior ranked officers wear the camera and those who had no previous investigation or offenses.

Pedro [00:33:28] So that highlights to us that the camera also interacts potentially with career concerns, but at least this is consistent with career concerns of police officers in that those who have a longer future timespan with the police force, they're more willing to adopt the technology and essentially turn on the camera when they need to be turned on. So that's a very important margin of originality that we find here. We find that the reduced the reduced use of force by 61% compared to against control dispatches is a very significant reduction of use of force. We find that it reduces charges of contempt and disobedience, which are types of crimes that are often associated with when there is some negative interaction and speaking negative interaction between the police officer and the citizen, they will often be recorded as as charges of contempt or disobedience and in a way to justify use of force by the police side.

Pedro [00:34:30] And we also find that there is a reduction in handcuffs and arrests. We combine the whole thing in the form of the negative interaction index when any of those things happened. And we find that the negative interaction index is reduced by 44%, which is again, a meaningful reduction of negative interactions between officers and citizens.

Jennifer [00:34:51] And remind us again what is included in that use of force category. Pedro [00:34:53] Its use of weapons, which can be lethal or non-lethal weapons, but it's

one of the two.

Jennifer [00:35:01] Okay, great. Yeah, those are big effects. The effects and very different from those null results we were talking about earlier in the other studies. So you talked a little bit about how these effects vary with who's wearing the camera. Is there any other variation across different types of incidents or are there other dimensions of heterogeneity that were worth mentioning?

Pedro [00:35:22] Yes, one important margin is among the severity of the incident and to be quite frank we expected the body worn cameras to affect incidents that were more violent in a way, and we found the opposite. So we found that incidents that are ex-ante classified is important because before attending a call, the police classifies the severity of the incident. And it's important that it's before because the severity of the incident is not itself contaminated by the presence of the camera, but it specifies it has low risk or high risk. They do so by assessing four questions regarding whether the perpetrators at the scene, whether they have arms or they don't, and some kind of basic questions to elicit the sort of the propensity for violence. And we find that the effects are concentrated along the low risk.

Pedro [00:36:12] So the interpretation here is that the camera, instead of just de- escalating a situation that's already dangerous and that there is already the use of force. What it does, it's actually preventing the escalation of the violence and preventing that use of force becomes necessary in the first place. So it's not about that, you know, incursion in a drug dealer that's already going to be violent and that's already going to result in an arrest having a camera is not going to affect the use of force in that setting. It's already going to in some way or another, unfortunately, but there's always there will be the use of force anyways. The margin that a camera works is by preventing that otherwise quote unquote simple incidents tend not to be violent ones like a traffic crash or domestic incident as well. So whether that sense in that sense, that camera will prevent it from being an incident that becomes violent as opposed to an incident that is violent into something that is not.

Jennifer [00:37:17] Okay. And then you also make use of those blackout days to see whether there were learning effects. So what do you have in mind there and what do you find if the cameras work?

Pedro [00:37:30] I would expect that there is some sort of internalizing of the effects by the police officers, meaning that by the time that you create this awareness that there is someone recording, someone observing your interaction, it's possible that that sort of internalization of the consequences of your own action will still linger, even if if you're not wearing the camera for some point in time. We do find some evidence of that. We do find evidence that the reporting effects, even for police officers who are not wearing a camera if they had used in the past. So what I mean is that blackout days treated police officers, they will still have a very similar reporting effect as compared to the treatment to the control group in the blackout days, as if they were wearing the cameras, which we associate to a learning effect, learning about using the camera instead of internalizing that camera eye that sits with you, even though the camera is not there.

Jennifer [00:38:28] Yeah, it's almost like you kind of get in the habit of, like, writing down these details or having these conversations or whatever it was that led them to record more of the victims or more of the domestic violence incidents that you'd been talking about before, but you don't find the same effects on use of force. Was that right?

Pedro [00:38:46] That's correct. So we don't find the same effect on use of force. It's possible that you get into a habit of, for example, collecting data, gathering the evidence from the field, figuring out that perhaps is not that much more costly to report in in the correct way, and that has benefits for the victims.

Pedro [00:39:05] So that kind of bonding goes on. So that's kind of a more and more subtle change of behavior as opposed to use of force. Use of force might be might depend

on also the whole dispatch unit observing, rather observing that the cameras present as opposed to some internal element and internalizing that that particular police officer has used in the past, which is not observable to the rest of the dispatch group. So it could be that is operating from that perspective. And in that case it would be in a way understandable if it changes one margin of reporting that depends on a single police officer as opposed to use of force, which depends on the whole composition of the dispatch group.

Jennifer [00:39:44] Yeah. Or the citizen's right or the citizens they're interacting with. Pedro [00:39:48] Of course. Yes. Yes. And so I'm going beyond the research. I'm just

trying to interpret what that result goes. But in fact, it's very difficult to say.

Jennifer [00:39:57] Yeah, but it is really interesting, I think, because there's a, you know, to the extent that it's the fact that you're being watched and there are potential consequences for misbehaving or for bad behavior, then you would expect you'd need to be wearing the camera all the time to get those kinds of effects. And so that seems to be the case, maybe with use of force, but not with the like recording effects of like actually writing down in a more comprehensive way what happened. But yeah, then there's it did make me think about like this could be the place where the citizen police interaction is really relevant because of course the citizens were not treated before, even though the officers were, but super interesting to try to think through that.

Pedro [00:40:40] Yeah, sure.

Pedro [00:40:41] I think to get to the bottom of that question so we could think about that in sense, for example, in implementing the randomization that's visible to the citizens so we can get to that side of the interaction as well, irrespective of the recording. So there are other designs that could speak to disentangle those effects and those those mechanisms. I think, you know, there is a quite a bit of potential interesting mechanisms to decompose and I think you alluded to one of them that's for sure.

Jennifer [00:41:07] Yeah, very cool. And then finally in the paper, you spend some time comparing your findings with all those previous null results from the earlier RCTs in the U.S. and U.K. And we've already discussed a bit. You argue that the unit of randomization in those earlier studies may have biased their estimates toward zero. So tell us a little bit more about what you do here and kind of what the punch line is in terms of how in your you and your coauthors are interpreting the differences across these studies.

Pedro [00:41:33] Sure. So there are two main differences between our design, our research and the previous research. One is the context. The second one is the design. The research design itself. By design I mean the unit of randomization, the unit, the data and so on. It's not that we didn't have a clear view on whether it was one or the other, but what our analysis suggests is that the design makes a very big difference in what we estimate as the treatment effects. This is not to say that I'm not ruling out that context is important. I'm just saying that we can very confidently say that the research design is important. We do so because we can map the research design and implement the previous research designs in our own data.

Pedro [00:42:16] And either by looking at some aspect of the randomization, for example, just across officers or by collecting the data to more aggregate data, for example, corruption at the officer level or the data level or the shift level. That is not something that

we can do. So in that way we can mimic the research design that has been used in the literature. And when we do so, we find very similar results that the previous literature does. By very similar I mean, it's within the same ballpark it's within the same in the confidence intervals. So we could have gotten the similar result had we implemented both design as also in Santa Catarina. So this speaks to the importance of the research design.

Pedro [00:42:58] It's possible that the context also matters, but we don't have a way of of mapping what would have happened with the research, with the treatment effects in the other designs, had it been done in a different way, of course. So in a way, in a way, we spoke a little bit about why we think that happens, but one of the issues that is very important here is the issue of contamination. Again, the comparison between the treated and control finding the appropriate control group that is not affected by the camera usages is very, very important.

Pedro [00:43:29] Another issue here is the unit of the data that we use. It's very granular, allows us to introduce a number of fixed effects. We have just a sheer size of observation is much larger and it's really where we would expect the interaction to unfold. So I think it's the kind of the two things coming together, research, design that is finer, that controls for spillover effects, for contamination effects in various ways and forms that that could take place that is allowed by a granular data design.

Pedro [00:44:02] That's what allows us to kind of put this apart from the literature I would stick my bets on that. So more than even more than the context that we do, what really matters here is the research design and that that's what the difference between the treatment effects.

Jennifer [00:44:18] Yeah, it'll be interesting to see if we now get a bunch of our RCTs in the US and UK again that just give cameras to fewer officers essentially to kind of get closer to what you guys were doing. Your design is a really nice model, I think for what we would like these kind of our cities to look like going forward. Okay. So what are the policy implications of the results from your paper and the other work in this area? What should policymakers and practitioners be taking away from all this?

Pedro [00:44:44] Our research shows that cameras can work to reduce police use of force and improve reporting, reduce the fatality, improve potentially improve relations with society, and also to protect officers from abuse themselves. So nowadays, if you had the costs reduced a lot of these cameras and this continues to decrease and the technology to improve. So even if the cost benefit was largely given the effect sizes, it was largely favorable in our original implementation. Nowadays, the scales have tipped even further in favor of camera use. But I would highlight here that cameras again cameras don't warn anyone. It's just the institutional settings and the set of incentives that the police forces have that will make a difference with whether cameras will have an effect or they don't. So whether the videos are analyzed, whether the videos are checked, or whether they are accessible by the criminal justice system and internal investigation. So crucial factors in creating a reasonable expectation by the police officers that the videos will be seen, that actions will have consequences.

Pedro [00:45:53] And so there's all these actors within the police and outside of the police, they substantially matter. So in a sense, it's not a blind implementation of cameras, but also an intimidation of cameras where all these institutional elements have to be taken into account and have to be thought such that the cameras have seen the effects.

Jennifer [00:46:13] And then how did the officials in Santa Catarina specifically react to these results?

Pedro [00:46:19] Very positively, I think so. The paper made the press there and so that also triggered other interest from other police forces in Brazil who also adopted cameras. They were pleased that the large scale implementation, this has kind of a scientific grounding to it. And that was, I think, quite positive in the end of the day. So yeah, the reception was was very, very good.

Jennifer [00:46:45] So are there any other papers related to this topic, body worn cameras or something similar that have come out since you all first started working on this study?

Pedro [00:46:54] Many of the papers is specifically in Brazil, for example, they were teams studying the effects of body cameras in Sao Paulo, which is a very large police force. Initial results are very promising. They show that the lethality falls very much in line with the estimates that we have in our paper. There's a recent paper also looking at the experience in implementing body camera in Rio, they document a number of facts, but the compliance there was was tricky, but Rio police forces, let's say it's a bit more difficult to manage. And so the newest kind of wave of research is coming along with treatment effect sizes that are consistent with evidence in different settings.

Jennifer [00:47:39] Very interesting. And are they all using a similar kind of implementation as your design?

Pedro [00:47:45] They have similar with differences some of them are not randomized. They're are just observational with the robustness, the observational data, and they know pre-trends and [00:47:56] [0.0s]

Jennifer [00:47:56] Sure.
Pedro [00:47:56] Validate the pre-trends and the whole thing, which is great and it's it's it's

very solid evidence as well.

Jennifer [00:48:02] Interesting. Yeah. And that is, of course, a different way to kind of deal with these spillover effects. We love RCTs but like sometimes if these interventions, if you'd expect that treating, you know, a subset of officers is going to have like community wide effects. Sometimes an RCT, within that community is just not going to work like there's just no way to deal with the spillovers. And so looking at using more like observational data across places could be really useful.

Pedro [00:48:26] It's super revealing and super robust as well so. Jennifer [00:48:28] Yeah.

Pedro [00:48:30] As I said, like certain questions cannot be answered by or can be very difficult.

Jennifer [00:48:34] Right. Right. Yeah, fantastic. Okay. And so what's the research frontier? What are the next big questions in this area that you and others are going to be thinking about going forward?

[00:48:44] I think there are a number of a number of of questions and open questions. One which I alluded to before is the institutional setting. So what are the right set of incentives and monitoring of the camera itself that moderate the camera use. Now, that is a little bit unknown. So we can think about as various police forces in the world. And so I hope we can get better data about the types of implementations and monitoring that police forces themselves implement so we can get an answer to that question. The second, I think area that would be interesting to clarify is given that can violence is an equilibrium outcome, it can be officer side or citizen side.

Pedro [00:49:29] I suspect that both are relevant and I'm sure the police's excessive use of force is a place here, but I'm sure that there is some relevance also to the other side. It would be very interesting to be able to distinguish between one side and the other side of the right. So if we could randomize, for example, whether citizens know that they are being recorded as opposed to the officers knowing that their recorded what the officers say about to the citizens about them be recorded into Santa Catarina that they were actually saying. I always say that they were being recorded and that by itself it's a compliance device. If we could make progress in disentangling these two sides, I think would be important and very interesting.

Pedro [00:50:12] Also, the effects about preventing the de-escalation kind of shows that there is something a bit more complex than the headline explanation. So about the situational dynamics of how the dispatch unfolds, then just the average of the the comparison between treatment and control. And finally, I think one area of research that could be very interesting to look at, it's about the perceptions of both. So we cannot that that's perhaps one area where randomization will be difficult to conduct. But whether decisions they change their perceptions of things because of the accountability and the monitoring that the camera allows. We cannot get to that question by just randomizing at the levels that we are randomized, and we need an implementation that's consistent over space and time in a way that the citizens can notice that that there is a there is a consistent and permanent a strong indication that interactions will be always recorded. So it's that kind of this continuum of over and over interactions and interactions that come with recording that perhaps can change citizen perception of the police, and that can only be achieved by providing a direct co-variation in the camera use.

Pedro [00:51:34] And over time as well, perhaps observational studies can do more about this, but also it's also a data collection challenge because when we gather that from admin data, we need to be able to kind of survey individuals to get to that question. So I think these are these are kind of the three main areas in which the research frontier could explore in the future. I would add to that sorry before I forget the effect on the criminal justice system, I think it's also quite important. Some recent research has shown some effects on that direction, but there's a lot more to be done along that to showcase how the videos are used and whether it affects conviction rates and to some extent follow up on. Your effects, right. So if the police officer not reporting what happens on the ground with a lot of detail and accuracy, we cannot expect that incident to become a criminal justice picked up by the criminal justice system, but conditional on them being accurately reported what we don't know is what happens yet with when it exhausts kind of the police boundaries and goes to the justice system. So that would be very interesting.

Pedro [00:52:42] Again, we need more time to be able to answer that question, but this is a kind of a series of outcomes that would be very interesting to follow up as we gather more evidence across police departments and as there is more time after the cameras

have been implemented, not just in Santa Catarina, but in other police forces in Brazil and worldwide.

Jennifer [00:53:02] Yeah. On that last point, I will plug Katie Bollman, who's a job market candidate this year, has a neat paper. Her job market paper is basically looking at what happens to the court cases and how how cases proceed or the prosecution and so on if there's body worn camera footage available and the results are sort of mixed, it's complicated, it turns out. So yeah, I think definitely a lot more interesting questions here to answer that are all suddenly much more interesting now that we have a sense that cameras work. They were less interesting when we thought they didn't work.

Pedro [00:53:35] Exactly. Read the job market paper. That's very interesting. Jennifer [00:53:38] Excellent. All right.

Jennifer [00:53:40] My guest today has been Pedro Souza from Queen Mary, University of London. Pedro, thank you so much for talking with me.

Pedro [00:53:45] Thank you, Jennifer. It's a pleasure talking to you. Thank you so much for the space. It's been a pleasure.

Jennifer [00:53:55] You can find links to all the research we discussed today on our website probablecausation.com. You can also subscribe to the show there or wherever you get your podcasts to make sure you don't miss a single episode. Big thanks to Emergent Ventures for supporting the show and thanks also to our patrons, subscribers and other contributors. Probable Causation is produced by Doleac Initiatives, a 501(c)3 nonprofit, so all contributions are tax deductible. If you enjoy the podcast, please consider supporting us via Patreon or with a one time donation on our website. Please also consider leaving us a rating and review on Apple Podcasts. This helps others find the show, which we very much appreciate. Our sound engineer is Jon Keur with production assistance from Nefertari Elshiekh. Our music is by Werner and our logo was designed by Carrie Throckmorton. Thanks for listening and I'll talk to you in two weeks.