Episode 90: Matthew Ross

 

Matthew ross

Matthew Ross is an Associate Professor of Public Policy and Economics at Northeastern University.

Date: March 28, 2023

A transcript of this episode is available here.

Episode Details:

In this episode, we discuss Prof. Ross's work on police officer field training:

“The Effect of Field Training Officers on Police Use of Force” by Chandon Adger, Matthew Ross, and CarlyWill Sloan.

OTHER RESEARCH WE DISCUSS IN THIS EPISODE:

 

transcript of this episode:

Jennifer [00:00:08] Hello and welcome to Probable Causation, a show about law, economics and crime. I'm your host, Jennifer Doleac of Texas A&M University, where I'm an economics professor and the director of the Justice Tech Lab. My guest this week is Matthew Ross. Matt is an associate professor of public policy and economics at Northeastern University. Matt, welcome to the show.

Matt [00:00:27] Hi, Jen. Thanks for having me.

Jennifer [00:00:28] Today, we're going to talk about your research on police officers, a field training, but before we get into that, could you tell us about your research expertise and how you became interested in this topic?

Matt [00:00:38] Sure. I think many of us in this field that study economics, crime and policing, you know, I started off really as more of a traditional labor economist. And the way that I got introduced into the policing side of things is through a number of different agency projects that I worked on, looking at discrimination in traffic stops. And, you know, one of the things that kept coming up over and over is, you know, we would go and engage with these public sector agencies and, you know, identify that there are these longstanding problems going on and they would sort of ask us, well, what's next? You know, what? What can we do about this? And, you know, that really is what sort of sparked my interest in police training and trying to figure out, you know, how we can implement interventions to reduce some of these longstanding disparities and some of these other issues in the criminal justice space.

Jennifer [00:01:28] So your paper is titled "The Effective Field Training Officers on Police Use of Force." It's coauthored with Chandon Adger and CarlyWill Sloan and in this paper, you're focused on a particular phase of training required for new police officers in Dallas, Texas. So let's start with some background on the overall timeline to become a police officer. How does this process work from the moment someone decides to apply for the job?

Matt [00:01:53] Sure. I think, you know, Dallas PD is pretty representative of police departments around the country. And so the way that it works in Dallas is that typically a police officer comes in, you know, a new recruit comes in, they get hired by the department and the first thing that happens is they get sent to the academy. So the academy in Dallas lasts about 36 weeks. And typically, you know, most police departments across the country, you know, they operate their academies in a similar fashion to Dallas and it consists largely of simulation and classroom based training and in Dallas, it lasts about 36 weeks. After a recruit graduates from the academy and at least in Dallas, you know, 99.9% do we hardly saw any attrition from between the academy and then sort of progressing to the next stage.

Matt [00:02:43] So after a recruit graduates from the academy, they move on to something that's called field training. And the way you can think about this, economists in your audience is that it's sort of an apprenticeship style model. So it's like what you know a lot of times a lot of other occupations used to do, you know, in the sixties and seventies, you know, basically where a junior police officer is paired up with a senior police officer and effectively they're observing what they're doing on the job. And through that mentoring relationship, they're effectively learning what it is to become a police officer. So after a

recruit finishes field training in Dallas, they go into this probationary period that effectively lasts a year.

Matt [00:03:30] And there's sort of a nickname for this in Dallas, as they call it, Little T, so little training. And basically the only real requirement in Dallas, which I think is probably a little bit different than other police departments in this sense, but basically the only requirement is that they have a partner and that the partner has been with the police force and, you know, they stressed this when we had interviews with them is that, you know, the partner has to be at least, you know, a month to several weeks to, you know, more than that, but it has to be basically a more experienced police officer, but the margin for more experience is actually quite small. So they basically have to be partnered up for approximately a year and after that probationary period is over, they graduate to becoming full fledged police officers.

Matt [00:04:13] Typically, you know, the career trajectory of a Dallas police officer is that they typically remain at that rank for about three years unless they decide to go and seek employment with another agency and after three years, they're promoted to a senior corporal. And typically then they sort of take on different responsibilities and their in their job changes a lot. So, you know, if we're thinking about, you know, the traditional patrol police officer, basically they're you know, that is years 1 to 3 of a police officers career path.

Jennifer [00:04:49] Okay. And then let's talk more about that field training phase since it's going to be the main focus of your paper. So it sounds like within that phase, there are actually multiple phases. So say a little bit more about how that process works.

Matt [00:05:02] Sure. In general, the recruit is paired up with a more senior police officer. So in Dallas, it has to be a police officer with the rank of senior corporal or higher. So senior corporal is what basically you have to have three years experience or more. The way that it's structured in Dallas is actually very similar to how other agencies do it across the country and in fact, in the criminology world this is, you know, this sort of style of field training is called the San Jose model, so it's been around since the 1960s and it's it's it's been adopted virtually by almost every policing agency across the country. And there's variations, but largely the overall structure is quite similar. So basically what happens is that in Dallas, the recruit gets paired up for their phase one field training officer, and that phase one field training officer is, you know, one of the most important relationships that the recruit is going to have because basically, you know, they're effectively shadowing this senior corporal and the senior corporals basically, you know, showing them the ropes in terms of what they're supposed to be doing when they're on patrol.

Matt [00:06:14] The phase one, they're paired with the phase one FTO for exactly seven weeks after phase one, they move on to phase two and phase three. And the idea there is that they're supposed to be rotating around two different field training officers or FTOs so that they can get different perspectives on policing, but the key component is that, you know, after they're done with phases one, two and three, which all last seven weeks, they then go on to phase four, which is the last part of their field training, and they actually return to their phase one field training officer. So whoever they happen to get in phase one, they come back to and phase four for the last three weeks. And basically what happens here is that the phase one/four FTO is looking at how much progress that they've made.

Matt [00:07:01] So, you know, when they first started off, they're shadowing their FTO. The FTO is is leading the charge on most of the calls and by phase four, they you know, the recruits should be taking over most of the responsibilities on the call. And the FTO is effectively evaluating them at the end of this to decide whether they should then progress to being a probationary police officer. The other thing that's happening sort of all along the way at this and you know, there is a formal structure for this in Dallas, but it really is, so they do formally record these things, but, you know, it's somewhat of a of a more loose evaluation structure.

Matt [00:07:40] So what I mean by that is that every day, basically the field training officer is required to give an evaluation to the recruit and they do this in writing and the records are kept in city hall. But effectively what came out of interviews with them is that they don't actually look at these records, so they're not used for anything, but, you know, there is this sort of continual evaluation process going on throughout field training. So the recruit is constantly getting feedback from their field training officer on things they've done right and things they've done wrong.

Jennifer [00:08:10] Okay. And then what had we previously known before you all first started working on this paper about the effects of police training of any kind on various police outcomes?

Matt [00:08:21] Yes, I think the literature, you know, I think our paper sort of touches on a bit of a broad literature here. So, you know, we know a fair amount that police discretionary decision making matters a lot in the law enforcement context. So, you know, we have the Goncalves and Mello paper for 2021 and their more recent working paper you know, my coauthor, CarlyWill, has a paper with Mark Hoekstra in 2020 looking at use of force in, you know, different neighborhoods based on the police officers race and ethnicity. You know, I've also done work on discretion in terms of, you know, police traffic stop enforcement behavior. So we know that discretion matters a lot.

Matt [00:09:03] In terms of police training you know, I think the literature there is a bit more mixed. So for a long time, there really hadn't been any rigorous, you know, causal studies of, you know, short and long term effects of police training and that that's begun to change a lot recently. So, you know, in terms of police training, you know, Kyle McLean et al., have a study using an RCT looking at various, you know, short term classroom based training initiatives. I also have another paper looking at the effect of cultural diversity training on the decision by highway patrol of the distribution of the race and ethnicity of the traffic stops they make. And then, you know, Emily Owens and there's others that have studied procedural justice training. And for the most part, you know, and I'm going to paint with a little bit of a broad brush, but you know that literature largely finds that classroom based training interventions have, you know, relatively small and short term effect on actual enforcement behavior.

Matt [00:10:08] So, you know, there's some variation in there. And what these studies are find in the types of enforcement that they've looked at in the types of training initiatives, but generally speaking, I think the literature says that, you know, classroom based training initiatives have relatively short term effects and either null or quite small results on enforcement outcomes. The other I think, relevant literature that sort of motivated us in this study is this sort of separate literature on the effect of peers in experience in policing. So, you know, Jeremy West has this paper on, you know, looking at, you know, search behavior over time and sort of an officer's experience at getting better and better at making searches, you know, over the course of their career. Roman Rivera also has a paper

looking at, you know, the effect of peers in the policing, in policing. And Holz et al., you know, has a paper that looks at the effect of appears injury on subsequent enforcement behavior.

Matt [00:11:06] So the way I would sort of think about that is like, you know, when we were getting ready to write this paper, we're sort of looking at this literature and thinking, well, you know, it sure seems like classroom based training initiatives don't have that much of an effect. But these other types of experience or peer effect, you know, either interventions or natural experiments sure seem like they do have a pretty big effect. And so, you know, I think to position our paper, I think we're the first really to look at field training and sort of this causal, you know, applied microframework where we can really get these clean estimates on the effect of field training on actual enforcement outcomes.

Jennifer [00:11:48] Okay, great. I think in some ways you're being generous to the existing literature. There really is just like remarkably little evidence here. I mean, it's just a perception important for such an important topic. It's like, how do we not know more about what types of police training work and especially on these real world outcomes, right? Not just like surveying people after they took the class or something, but actually looking at their behavior. So very helpful to see this study. So in that context, then why don't we know more than we do what makes this topic so difficult to study?

Matt [00:12:21] So I think the big barrier to entry in this space in what makes studying police training so difficult is that, you know, for a long time there was just simply not good data available on police training. And it really hasn't been, you know, until, you know, the last few years when there's been this big push to reform police training and, you know, really try to figure out which types of training are effective, that, you know, good data on police training initiatives has been available for researchers. I mean, I think the other piece of this that has proved especially challenging is the fact that, you know, in most states in Dallas in particular, there's legislative mandates that require, you know, specific components of training.

Matt [00:13:11] And so, you know, trying to think about implementing in an RCT in that context is is is really challenging. The other piece of it is that, you know, some of these things, you know, in some sense, you know, it's it's sort of the ethics of it are questionable right, so there's core components of police safety, you know, of police training that impact the officer safety or the citizens safety. You know, the thinking about sort of randomizing training on those margins, you know, or not providing officers with certain aspects of that training becomes really difficult. So, you know, I think all the all of that, you know, all of what I said is true for both classroom based training initiatives or field training like this paper studies.

Jennifer [00:13:54] Yeah. And so that leaves us in a situation where we want to find natural experiments like phased roll out. So you can't train everybody at once then you know, already you're going to have to, you know, train 30 at a time or something and so maybe you can do that in a way that gives you some traction on this problem. So, yeah, it's a place where researchers are just beginning to have these conversations with law enforcement and have them work together to sort this out. So in this paper, you're going to measure the causal effect of the field training officers those FTOs for short, and in particular their propensity to use force in their interactions with civilians on their trainees future enforcement behavior. So to do this, you're going to use the way that trainees are assigned to FTOs in Dallas, Texas, as a natural experiment, as elsewhere it's hard to do in

an RCT here, but you do have a nice natural experiment. So how does this assignment process work?

Matt [00:14:48] Yeah, so the impetus for this project really came out of these discussions that we had with Dallas PD, and, you know, we were sort of asking them all types of questions about their training process, both field training and the classroom based training that they go through. And one of the things that they kept stressing to us over and over again was just the importance of field training. And so when we probed a little bit deeper on how sort of the assignment works, you know, we were surprised to find out that, you know, effectively what they were telling us is that they were just randomly pairing FTOs or field training officers to recruits. So basically, you know, just the 30,000 foot version of how it works in Dallas is that basically there's seven divisions. And the command staff in Dallas is taking each academy class where they have a bunch of recruits and they're making decisions about where they're filling gaps, basically because of retirements or because of promotions. They simply don't have enough patrol officers in in a subset of these divisions.

Matt [00:15:47] And so some of that the assignments that are happening at the divisional level, you know, may be nonrandom. So, for instance, you know, if a particular division is down, you know, somebody that speaks Spanish, for instance, they might send an officer that speaks Spanish to that particular division they also do that with female officers, right so they have a female officer available to do a female, you know, search of a female civilian. So some of that, you know, is happening in sort of a nonrandom fashion, but then once the recruits folder makes it to the division level effectively, you know, because of the nature of these sort of phase one, phase two, phase three, you know, field training regime that they have in Dallas, they're effectively just randomly assigning the first FTO and, you know, the in the FTO rotation to these recruits. And so basically there's you know, there's effectively, you know, the command staff at the particular division basically doesn't really know very much about each recruit other than their name.

Matt [00:16:54] So, you know, as I think every applied social scientist should be, we were fairly skeptical when when they told us that all of this was happening, you know, in a quasi random fashion. And so, of course, you know, I think the first thing that we did when when we heard this, we got excited, but of course, the first thing we did was we went to the data and actually tested for whether it seems like this is true and to our, you know, our our surprise and delight, the data supported the institutional claim that this was happening effectively, randomly.

Jennifer [00:17:26] Great. Yeah. So you don't have the trainees choosing their FTO or the FTO choosing their trainees or they're being matched in some sort of way. And I think you mentioned in the paper that the process is very different for that "little T" training you mentioned before. So the kind of probationary period, after all the field training where it sounds like the trainees actually do choose their FTO or their their supervisor. And so that would be very difficult to study here am I remembering that correctly.

Matt [00:17:53] Yeah, No, that's right. You know, and I think that's actually one of the comments we get a lot, you know, about this paper is, you know, people want to know more about like, you know, well what's if the FTO matters, you know, what about the partner? What about the person they partner with in "little T"? And I think the challenge there is that there's just so much selection going on.

Jennifer [00:18:10] Yeah.

Matt [00:18:10] There that fortunately we can't really look at that in the same way. We can't with field training officers.

Jennifer [00:18:14] Right. Probably important, but you don't have a research design. Matt [00:18:18] Exactly.

Jennifer [00:18:19] Yeah. Okay. So. So you've got this nice natural experiment. It seems like the trainees are assigned FTOs in a in a essentially at random. So how do you use that natural experiment to measure the causal effects of the FTOs on their trainees future behavior?

Matt [00:18:37] Sure. So the first thing to think about in terms of, you know, the identification in this paper is that, you know, we go in and we measure the field training officers past propensity to use force basically in the period before they get assigned a particular recruit. So, you know, the way you can think about this is that, you know, we have data on the field training officers interactions on a variety of 911 calls we see whether the 911 call you know ends in an outcome like force also things like arrests and, you know, different types of arrest. And what we're able to do is basically we consider that an exogenous period because basically the field training officer is operating independently of the recruit because they haven't been assigned the recruit yet.

Matt [00:19:25] And so, you know, we take that, you know, exogenous measure of the field training officers propensity to use force and we look at the recruits, subsequent force use propensity in the period following field training. So basically we're constructing the FTO measure in the period before the recruit is assigned to the FTO, and we're constructing all of the outcome measures for the recruit in the period after they're done with field training. So, you know, for all intents purposes, you know, the FTO and the recruit are not in either period. And so, you know, the identification should be nice and clean as long as we have random assignment between the field training officer and the recruit.

Jennifer [00:20:09] Yeah. So the underlying idea here, I guess, is that, you know, everyone just has different preferences about how much force to use. And assuming we have enough variation across these police officers, these FTOs in those preferences, which we'll talk about in a moment, then being randomized across FTOs essentially means you get randomized to different levels of force or different preferences about force from your supervisor. And we have now lots of different studies that take advantage of this aspect of human behavior that we all just vary in our preferences over things whenever we have discretion. And so, you know, randomizing people across courtrooms with different judges means you get randomized to different judge preferences about incarceration and all that kind of stuff. And so it might seem very unfair, but as you mentioned before, there's lots of evidence that police do have lots of discretion and they vary a lot in their use of that discretion. And so that's basically the experiment that people essentially randomize to different levels of force kind of from their supervisor. Is that right?

Matt [00:21:18] That's right. I mean, I think there's a couple ways to think about it. So, you know, the way that I think about it is that the way that we're measuring field training officer propensity to use force, we're effectively trying to control for sort of all these differences in the call for service that might lead to force. So obviously there's going to be some circumstances that are so dangerous or the citizens acting in such a way that it just necessitates force. So what we're trying to do is control for as much as that as possible

and identify, you know, this variation, as you said, and in the individual's propensity to use force rates, that there's this discretionary aspect of force use.

Jennifer [00:21:59] Yeah. So some people might just be assigned to more dangerous beats or something. So you want to compare officers that are all assigned to that same beats you're comparing apples and apples. But yeah, once you control for all that stuff, you're just sort of left with just differences in preferences across people.

Matt [00:22:15] I think that's right. I mean, the other thing I think to think about too is, you know, I think there are a lot of possible mechanisms to sort of underlie, you know, some of the things we're finding in this paper, but I think our preferred mechanism and I think we're going to get into this in a little bit more detail, but I think our preferred interpretation for this is basically the field training officer is teaching the recruit about when it is and is not appropriate at the margin to apply that discretionary aspect of force. And so that is, you know, through the the randomization, what we hope that we're identifying.

Jennifer [00:22:53] That's the treatment. Okay, so what data are using for all of this?

Matt [00:22:58] So some of this data is actually came from Emily Weisburst that she used her 2022 JHR paper things like calls for service, but we also CarlyWill and I and Chandon paired the calls for service data that Emily used in her paper with a bunch of additional public information requests that we made on things like CAD notes and RMS incident reports. We also got a we have a large set of force incident reports as well as arrest records and then we have human resource information about the individual officers. And so basically what we did was we took all those desegregated datasets and linked them all together to create a really large linked dataset. So basically in our dataset, you can track an individual calls for service, you can look at the specific notes associated with that particular incident. You can see whether or not that call for service or 911 call resulted in force or an arrest and you can also tell all of the officers that arrived in the timing of their arrival to that particular incident.

Matt [00:24:09] So in total, our dataset is about, you know, it's several million 911 calls in most of our estimates, we're going to use a slightly smaller sample focusing on the post field training period of the individual recruits. And you know, most of the time our samples are going to be in the 1 million calls for service range. Also worth noting that, you know, our sample of recruits that are coming in over this period is 411 so, you know, we're looking at a period from from 2013 to 2019, basically.

Jennifer [00:24:43] Okay. And you mentioned this a little bit before, but tell us a little bit more about how exactly you're calculating an FTO's propensity to use force. This sounds easy when you actually think about it and it becomes a little bit complicated. So what do you all do?

Matt [00:24:58] So, you know, as we were sort of alluding to before, right, there's a lot of different circumstances that, you know, are very clearly going to necessitate a police officer using force. You know, so what if you know, the intuition behind what we're trying to do is we basically want to try to control for as much of that stuff as possible and then obtain an estimate of a particular officer's propensity to use force above and beyond, you know, the average across all officers for these things. So basically what we're doing is we're running a regression where the left hand side is, you know, a binary variable for, you know, doesn't, you know, call does not result in force or one call does result in force. And on the right hand side, we have a bunch of control variables like, you know, with things like

number of officers that arrive on the scene of the particular call, the beat so the geographic location of the call as well as the type of call so that the, you know, what crime was going on that motivated the call in the priority type given to it by the dispatcher. Then we're also controlling for some temporal stuff like the calendar month and the day of the week and whether the call was at night. Basically, we were trying to throw the kitchen sink at it in terms of controlling for as much as we possibly can to soak up these differences you know, aside from an individual officer's propensity to use force sort of above and beyond these observable factors.

Jennifer [00:26:25] Okay. And then so once you soak up all that other stuff, how much variation is left in that propensity to use force? And then is that variation correlated with other officer characteristics like age or race or stuff like that?

Matt [00:26:40] Yeah, you know, we were actually surprised by this we you know, I mean, this is one of the first things we did was, you know, plot out these the field training officers, you know, force use rates in the pre period before they're assigned recruits. And of course, you know, we plotted these density plots and looked at how much force it varied across these field training officers. And I think one of the things that we were sort of surprised by right off the bat is just how much variation that there was.

Jennifer [00:27:07] Mm hmm.

Matt [00:27:07] You know, in fact, some of these field training officers, you know, don't use force very often at all relative to their peers. And there's other, you know, field training officers at the extreme that are using force quite a lot. So there's a significant amount of variation both on the recruit side and on the field training officer side in terms of their force use propensity. We don't I don't think we formally show these correlations in the current draft of the paper, but, you know, I'll say that the field training officers propensity to use force, it is correlated with some of the observable characteristics. But, you know, we do a set of runs where we actually control for the field training officer characteristics and what we find is that even when we control for the field training on officer characteristics, actually the result for force propensity gets much larger.

Matt [00:27:57] So, you know, if it's correlated with the field training officers characteristics in ways that you might expect. So, you know, the field training officers that are male tend to use force, field training officers that are younger tend to use force more and it's and it's a similar thing for the individual recruits as well.

Jennifer [00:28:13] Yeah. And that stuff that as you were saying earlier, previous papers have shown this sort of these differences across different types of officers and arrest propensities and force propensities and stuff like that. Can you give us a sense of like what's a typical amount of force? What are we talking here? What's kind of the the average? And then what's a lot for those folks that are on the kind of extreme end?

Matt [00:28:36] Yeah. So, you know, in general, the force is a really rare event. I think, you know, that's one of the challenges that we spend a lot of time in the paper and thankfully you're not you haven't asked me a lot about during this. We spent a lot of time having to deal with the fact that force is just a really rare outcome for a 911 call and so it's about one in a thousand. You know, as far as the variation, you know, one standard deviation variation in the amount of force is actually equal to like 138% increase above the mean. So it's like it varies pretty substantially across FTOs.

Jennifer [00:29:12] Okay. But with yeah, it's still it's not like every other incident or something. They're using force it's a it's a pretty low rate still.

Matt [00:29:19] No it's quite small. I mean even the even papers that have looked at arrest as a, as an outcome of sort of struggle with the fact that that's a rare event and that's an order of magnitude larger than force. Yeah, it's a really particularly relevant.

Jennifer [00:29:31] Yeah. Okay. Despite the impression we might have from the news, it's still you know, it's important because we our concern is that it is it's happening too often, but the challenge for researcher is that it is not really happening all that often, which makes it tough to to find variation in.

Matt [00:29:48] I mean, I think the other thing on that that's worth noting is in our in our setting in Dallas, the requirements for an officer to report force are actually pretty low, despite the fact that this is an extremely rare event. It's actually true that police officers deaths are reporting even one might consider a really low level of force, such as like, you know, manhandling somebody when you're putting them in handcuffs or or, you know, stuff that we wouldn't necessarily think of that rises to the level of force that we often hear about in the news. That said, the standard that Dallas uses to report force is one that has, you know, is sort of considered to be a gold standard for reporting force, you know, by policing agencies and one that's been adopted by a lot of agencies across the country. So there's other agencies that only report much higher levels of force, but there's many that that have the same standard as Dallas.

Jennifer [00:30:50] Okay. Yeah, that's a really good flag that when we're talking about force here, we're talking about that whole range for everything from, you know, putting your hands on somebody up to using a firearm. And so when you're talking about propensity to use force, you mean propensity to use any kind of force, right?

Matt [00:31:05] That's right. In the paper, we do some runs where we restricted the more severe uses of force, but for the most part, we're talking about anything that Dallas P.D. defines as force.

Jennifer [00:31:15] Okay.
Matt [00:31:15] Which is a relatively low bar.

Jennifer [00:31:18] Okay, great. And then what outcomes are you most interested in here?

Matt [00:31:23] So I think the outcomes that we were initially interested in and the outcomes that the paper focuses on are a little bit different. This is I think this is one of those things where the paper, you know, you don't want to write a paper that explains exactly how you made the sausage, but you know.

Jennifer [00:31:42] But you can tell us here.

Matt [00:31:43] Exactly. So, you know, I think when we started off this project, I think we were just interested in looking at field training broadly. And so I think we had a lot of different outcomes that we were interested in. And for the most part, what we found was that force was really one of the only behaviors that we were able to find a really strong sort of effect being passed down from the field training officer to the recruit. So, you know, we

looked at a lot of other stuff that I think would have been quite interesting if we had found, you know, particularly strong effects. We looked at things like response times and times on call. We also looked across it at different measures of of proactive policing and what one might call making of discretionary arrest or what I would call like ticky tacky type arrests. And for the most part, we do find some of these things are correlated with force behavior, but we didn't find that any of these other types of enforcement behavior were were being passed off from the field training officer to the recruit in quite the same way.

Jennifer [00:32:44] Interesting. Yeah. And so, you know, in some ways, like thinking about the effect, if you've got the treatment here is propensity use force seems like the most likely outcome the outcome you'd be interested in here is do the people, do the recruits later use force? And so that's a very direct thing to look at, but it sounds like you what you're saying is that you considered other propensities, like propensity to make arrests, propensity to like show up quickly to the scene, things like that and none of that there either wasn't the same kind of variation in FTOs or that stuff just like didn't matter.

Matt [00:33:21] Yeah, I think that's right. I mean, the other thing to consider, though, with I mean, the one caveat I'll put on that is that Dallas is a little bit of a unique police department in terms of like what their police officers actually do. You know, Dallas has been facing this challenge where they have had a large rash of quits after this pension reform that they had, you know, prior to the sample period so prior to our sample period in going forward, they've been really short staffed. And so in Dallas, basically there's this huge log of 911 calls and all, most of their police officers are doing are answering 911 calls. So, you know, you could imagine in a different setting where the police are sort of doing a broader range of activities, like, for instance, making traffic stops or engaging in motor vehicle searches or things like that. You might be able to look at, you know, field training officers propensity to do things like that and the recruits propensity to subsequently do things like that, but in our setting, there's really sort of a limited number of tasks that they're doing, you know, and across that range of tasks, it really look like, you know, propensity to use force was one of the only things that was really transmitting between the field training officer and the recruit.

Jennifer [00:34:35] Interesting. It also makes I mean, in some ways the story cleaner, like if the you know, the field hiring training officers were also varying a lot on these other dimensions that were all then correlated with use of force going forward, it becomes more complicated to think about, you know, what the mechanisms are. And here it's just like it seems like a fairly straightforward story here that they're learning about when it's appropriate to use force from their training period, which is.

Matt [00:35:03] I think that's right.
Jennifer [00:35:03] A story you were telling before. Yeah. Okay so what was the effect of

FTOs propensity to use force on their trainees future use of force?

Matt [00:35:12] So what we found was that if an FTO's propensity to use force increased by one standard deviation and, you know, as I mentioned before, that's actually a really large increase so a one standard deviation increase in the FTO's propensity to use forces is 138% increase relative to the mean. And so we found that that one standard deviation increase in an FTO's propensity to use force was associated with about a 12% increase in recruits, subsequent likelihood to use force on a given call for service. And another way to think about this is to put it into an elasticity and I think this is somewhat useful, you know, for thinking about how to compare the effects that were finding relative to what the

literature is also found in pure effects in a variety of other settings what we actually find is that one standard deviation to 12% increase is about 0.1 elasticity. It's a bit under that, but that's relatively comparable to what others have found in the pure effects literature in general, which I think gives us a bit of external validity to what we're finding.

Jennifer [00:36:22] Got it. So it matters, but like the fact that you saw somebody use force doesn't mean you are automatically going to use force. It's not like a 1 to 1 kind of change, which makes sense. We wouldn't expect people to be affected quite that much.

Matt [00:36:36] I think that's right. One note on what you just said, Jen, is that, you know, one of the other things that we got asked a lot when we initially started sharing this paper was whether, you know, whether it was the field training officer, teaching the recruits something about when it is or is not appropriate to use force versus maybe something like the recruit is witnessing the field training officer use force. And so we actually do, you know, a number of tests in the paper to try to see, you know, which of these stories is what's going on.

Matt [00:37:10] And so what we find is that it actually seems to be more about the field training officer teaching the recruit about using force versus them actually witnessing the FTO use force. So it doesn't seem to be like a potentially like a trauma story or the you know, the recruit is actually involved in a force incident. So then they use force later. It's something about teaching them about that you know, about that discretionary decision.

Jennifer [00:37:34] Interesting which makes the results even more striking. Matt [00:37:37] That's right.

Jennifer [00:37:37] That that matters. Did this effect vary across different types of officers at all?

Matt [00:37:43] You know, so I think that's a very important question, especially when we think about this particular paper sort of sitting in this broader literature about peer effects. So, you know, if we think about our result, talk a little bit more about why we think this particular mechanism is the correct one, but if we think about this result as being about transferring information about when it is and is not appropriate to use force between the FTO and the recruit, you know, it might matter a lot, you know, the characteristics of the FTO and it might also matter a lot, you know, what are the characteristics of the recruit and, you know, almost more importantly from like a peer effects kind of standpoint, it might actually matter a lot, the pairing of the recruit and the FTO. So are they the same gender or the same race or are they around the same age and what we find is that, you know, we do find some variation in our results across different types of officers. So in particular, we find that the results are slightly higher with younger officers. We also find that they're slightly higher with male officers, but for the most part, their results are relatively consistent across all these different subgroups.

Matt [00:38:56] And, you know, so so like I said, some of these results are slightly higher, but the confidence intervals are overlapping. And so we can't, you know, statistically distinguish between any of these heterogeneous results. And so for the most part, it actually looks like everyone is responding to force use in a similar way, regardless of their characteristics.

Jennifer [00:39:16] Okay. And then how long does this effect appear to last?

Matt [00:39:20] So I think we can say with a fair amount of confidence that it looks like the effects lasts at least two years. And the reason I say that is, you know, we varied the time period that we looked at in the post period, so when the recruit is done with their field training, but there's two reasons basically why why, you know, things get a little bit more tenuous after two years. One is that, you know, our data simply doesn't support us going out much beyond three years. The other is that, you know, at that two, three year point is when officers get assigned to different duties and oftentimes they get promoted or they become field training officers themselves. So, you know, we can say with confidence that it looks like the results last at least two years at the two year mark. Our results tend to attenuate and sort of fade out at that point for so between 2.5 and three years, they basically, you know, fall to zero. You know, and I don't think that we can confidently say that the results are actually going away or whether, you know, the officers in our sample so we know they're not necessarily atritting out of our sample, but I don't think we could say that they're actually engaged in the same duties that they were before. And so that, you know, you know, it's definitely that their propensity to use force is actually going down after that.

Jennifer [00:40:43] Okay. And as you mentioned before, you try to dig into the mechanisms here a bit to understand why FTOs are having such an important and long lasting impact. So what are you able to look at here and what's the main takeaway.

Matt [00:40:56] You know the way that I think about this part of the paper is we really, you know, tried to look at all these different dimensions of the mechanism. So through presenting this paper, I know CarlyWill did your seminar series in the spring and, you know, sort of shopped this around in a number of places and we got a lot of feedback about alternative potential mechanisms and a lot of them, you know, made sense to us as an alternative potential mechanism. You know, some of the examples of some of the things that, you know, we thought about were, you know, you might imagine that, you know, even when a field training officer and a recruit are done with field training, Right. So they're no longer paired together. It might be that, you know, and this is particularly true in Dallas, where police officers have discretion over the particular calls that they answer it might be that, you know, a recruit, even after they're done with field training, tends to show up on calls where their FTO is.

Matt [00:42:01] So you imagine that the recruit in the photo become buddies through the field training process. You know, you might want to show up to calls where your FTO is so you guys can, you know, chew the fat after the calls done. So that's one possibility. So, you know, we explored that in the paper found that not to be you know, that was not explaining our results.

Matt [00:42:23] Another thing is that we wondered whether the partner selection that we talked about before that goes on during this "little T" period. So this probationary period that partners selection might be part of what's driving their result. You can imagine the story where, you know, the, you know, a particular FTO trained five recruits, you know, at varying time periods and some of them end up partnering together and it's really not about the field officer. It's really about this partnering with another high force officer. You know, we explored aspects of that, you know, controlling for the particular partner's force propensity, and we weren't able to explain the results that way.

Matt [00:43:01] Other aspect to this that we looked into is proactive policing. So you might imagine because, you know, we're in this setting with Dallas where the officers have some

discretion over which name one calls they respond to that. You know, maybe, you know, what we're picking up is not force propensity. It's, you know, heroism or something about a police officer selecting in to the most dangerous calls. Right. And you know, what we find when we control for like a wide variety of the call characteristics is, well, that that doesn't seem like it's true either and it's not related to other aspects of policing that we might think are related to proactive policing. So it does it doesn't have to do with the types of calls that the recruits going to. And it doesn't look like they're doing other stuff that we would think of as proactive policing.

Matt [00:43:51] You know, and I think the last one that, you know, I was I think I was I in particular was very concerned about was, you know, people were asking us whether we thought maybe this was less about a story of, you know, force used propensity, maybe it was a story about force reporting behavior. So basically, you know, if you think about our data as being a function, you know, the force reports that we get is being a function of whether the officer actually decides to report the particular force incident. You could imagine that, you know, the mechanism behind our results might be that, you know, it's it's the FTO teaching the recruit about when it is is not appropriate to report force rights. It's not about force use it's about force reporting. And I can say with about as much a degree of confidence as one can have without sort of reviewing all of the body cam footage of Dallas P.D the incentive structure of reporting, it doesn't really make sense for an officer to not be filling out a force report.

Matt [00:44:56] So, you know, the way that you can think about this is that, you know, in Dallas, for the most part, during our sample period, they're wearing body cams and there's oftentimes when there's a force incident, there's more than one officer on the scene. And when an officer goes into the reporting system, which in this case is called Blue Teams or Blue Shield, when they go into that software to report a force incident basically what they're doing is they're explaining to their supervisor why they had to use force. So the incentives are such that if an officer inappropriately used force, they would definitely want to report it, because it's basically allowing them to craft a narrative about why the force was justified. And in fact, the database is not called a use of force database it's called a response to resistance database. So we think it's just from the institutional perspective, very unlikely that this is about reporting behavior, because we think that reporting compliance is is extremely good in Dallas.

Matt [00:45:56] The other piece of it is, you know, we did as best we could to try to get at other aspects of reporting. So we did a number of runs where we looked at, you know, propensity to write longer reports or omit information from incident reports. You know, we thought maybe that's sort of a proxy of, you know, just willingness to report about what happened on a call for service. And what we found is that there doesn't seem to be any differences that are driving our results in terms of reporting behavior. So, you know, that's sort of a a roundabout way to say that, you know, we sort of in the paper, we rule out a lot of different mechanisms that might explain our result and the only one that really, you know, we feel that we're left with is this idea that it's really about the field training officer, teaching the recruit about when it is and is not appropriate to use more force.

Jennifer [00:46:48] Yeah, I thought the the piece about the body cam footage and reporting was really interesting and useful and another way that body cams are going to be very helpful to research. Right. And I think I think you mention in the paper that there's regular auditing of that footage. And if you're found to have not reported, there are like pretty severe penalties. And so, again, just like lots of incentives pushing toward reporting here, which is super helpful for the type of incident we I agree we would normally be

worried about being underreported. So very cool. Okay. So what are the policy implications of these results? What should policymakers and practitioners who are listening take away from all this?

Matt [00:47:28] You know, I think I think that's twofold. So, you know, on the one hand, I think that, you know, from a policy perspective, you know what our paper shows that field training is really important. And in the particular setting, you know, which I mentioned before, is sort of unique to Dallas in that, you know, most of the activity that these officers are doing on the job is responding to calls for service. So in that setting, we find these really strong effects, you know, where, you know, field training officers are having this large impact on a recruit, subsequent force use behavior. You know, so on the one hand, it's telling us that full training is really important for this particular dimension of recruit, subsequent enforcement behavior, but you might think in that settings outside of Dallas, where, you know, you think about municipal policing agencies, where police are doing, you know, a lot of traffic enforcement, that in those settings they're very well might be other aspects of policing that were field training is having a large impact. So from the from you know from I think on the one hand it's really emphasizing how important a field training is.

Matt [00:48:37] On the other hand, you know, it's also it's giving us potentially a cost free intervention for reducing force in policing. So, you know, if you know, as a society, you know, we sort of look at these news stories that are coming out, we think that the amount of force that police officers are using is is inappropriate. You know, in our in our paper as sort of not taking a stance necessarily at that, but, you know, from a policy perspective, if we think that, you know, police in a particular agency are using too much force, you know, this really gives us, you know, effectively a cost free potentially intervention for reducing force. So if you simply, you know, do a better job at screening field training officers or potentially, you know, you put more resources into, you know, overtime for the field training officers that use less force effectively.

Matt [00:49:31] You can reduce force for the whole policing agency by simply reshuffling people potentially. I think there's two sides of it one is that I think there's more research needed in in field training in general. And then, you know, in particular with force use I think it potentially gives policymakers a real a really powerful intervention for reducing, you know, a particular, you know, outcome of policing that I think we as a society have decided is happening too frequently.

Jennifer [00:50:00] Yeah. It also highlights the costs here of tolerating unnecessary use of force, or it has these spillovers to peers, to trainees in ways that are that go beyond just the individual so so to the extent that would you know, we had all this conversation about whether it's a just a few bad apples or the whole orchard and you know, those few bad apples affect the behavior of people around them and this is one way that they do it. And so making sure that they have less influence over their peers can be very helpful.

Matt [00:50:33] You know, I think the criminology literature Jen does a much better job at stressing those type of peer effects than, you know, sort of we in the social sciences maybe have done up to this point. You know, they talk a lot about, you know, in the in the 21st century policing task force, you know, the document that came out of that they talk a lot about the importance of policing culture and sort of this warrior versus guardian mentality. And I think, you know, this paper really emphasizes, you know, as you said, these, you know, how important these peer effects are. You know, and I think the other thing to think about, too, is, you know, we find this relationship, which is, you know, one

standard deviation increase in FTO's propensity to use force leads to a 12% increase in recruits, propensity to use force.

Matt [00:51:25] The other thing to think about is that like potentially, if you're if you're thinking about an intervention where you're you know, where you're rearranging the or selecting less forceful field training officers, you should really be thinking about this as sort of a dynamic effect because, you know, the recruits eventually when they at least in Dallas, when they reach their senior corporal level after three years, they basically can either become a detective or a field training officer. So, you know, not only are we reducing force for that first group of recruits, but then there's these dynamic effects that potentially are going to exponentially reduce force over time.

Jennifer [00:52:01] Have any other papers related to this topic come out since you all first started working on the study?

Matt [00:52:06] That's a good question. So I know that I would be remiss if I didn't acknowledge that our colleagues in criminology are constantly working on topics like this. So, you know, I'm sure that there are things in that literature that, you know, I've missed, but, you know, I would say as far as economics and, you know, the applied social sciences go, you know, I don't think that there has been a lot done looking at force and peer effects in a field training setting. Although I would say that, you know, Roman Rivera has this recent paper looking at peer effects sort of in different outcomes, but peer effects that I think is related but not necessarily related field training, but peer effects in policing. So Oendrila Dube has this interesting RCT that I saw at the Summer Institute looking at the effect of cognitive behavioral therapy on police officers. I don't think that there's a working paper out just yet, but, you know, I thought that was a, you know, a particularly interesting intervention, you know, not necessarily aimed at training or field training, but, you know, potentially really powerful intervention at reducing things like force and policing.

Jennifer [00:53:21] Awesome. Yeah, I've been looking forward to seeing that paper. I'll also mention my colleague Danila Serra has a paper on police training in Ghana, and so I'll put a link to that paper and podcast on in the show notes. And I remember, when she was on, she was noting she's new to the policing and crime space and was noting her shock that there was not just a ton of evidence on what trainings work, at least in the U.S. And so, yeah, it is a it is a thin space that I agree people are beginning to pay attention to. And so it's a hot area right now. And so every new paper that comes out, I'm super excited to see. So on that note, what's the research frontier here? What are the next big questions in this area that you and others are going to be thinking about in the years ahead?

Matt [00:54:04] You know, I think in general, I think the policing space is pretty wide open in that there's a lot of really important questions that I think we as a society have that applied social scientists haven't really answered in a particularly convincing way yet. So I really think that that is sort of an open field with respect to field training in particular know, I think that there are a lot of questions. You know, I sort of mentioned this before that, you know, if you're operating in a different setting other than Dallas, where, you know, police officers are making a lot of traffic stops, making a lot of searches, you know, you might think that field training might have different effects. You could imagine, you know, I've done a lot of work on, you know, racial profiling and traffic stops. And, you know, I would love to be able to find a setting where I could write a paper looking at, you know, the likelihood to stop minority drivers and whether, you know, that sort of thing is passed down from a field training officer to a recruit.

Matt [00:55:03] So I think, you know, there's a lot of additional questions we have that, you know, I think are really important that we could just answer about field training, let alone sort of the policing space as a whole.

Jennifer [00:55:14] Yeah. And I think like big picture, again, the guiding question here is like, how do we maintain the benefits that we know that police can have in terms of crime reduction while reducing the social costs involved, which is this use of unnecessary use of force, unnecessary escalation of incidents to arrests and so that to me feels like the million dollar question that is kind of guiding all of this research. And we are just at the very beginnings of figuring out how to improve policing in that way. So, yeah, exciting time to be a researcher. And thankfully we've got, you know, 18,000 different police departments across the country that are all.

Matt [00:55:52] That's right.
Jennifer [00:55:53] Doing the right stuff.

Matt [00:55:53] Wildly different things.

Jennifer [00:55:55] Yeah, lots of opportunities for cool, natural experiments. My guest today has been Matt Ross from Northeastern University. Matt, thank you so much for talking with me.

Matt [00:56:05] Thanks for having me, Jen.

Jennifer [00:56:11] You can find links to all the research we discussed today on our website probablecausation.com. You can also subscribe to the show there or wherever you get your podcasts to make sure you don't miss a single episode. Big thanks to Emergent Ventures for supporting the show and thanks also to our Patreon subscribers and other contributors. Probable causation is produced by Doleac Initiatives, a 501(c)3 nonprofit, so all contributions are tax deductible. If you enjoy the podcast, please consider supporting us via Patreon or with a one time donation on our website. Please also consider leaving us a rating and review on Apple Podcasts. This helps others find the show, which we very much appreciate. Our sound engineer is Jon Keur with production assistance from Nefertari Elshiekh. Our music is by Werner and our logo was designed by Carrie Throckmorton. Thanks for listening and I'll talk to you in two weeks.

Jennifer DoleacPolicing