Probable Causation

View Original

Episode 19: Jeremy West

Jennifer Doleac

Jeremy West

Jeremy West is an Assistant Professor of Economics at the University of California at Santa Cruz.

Date: December 24, 2019

Bonus segment on Professor West’s career path and life as a researcher.

A transcript of this episode is available here.

See this content in the original post

Episode Details:

In this episode, we discuss Professor West's work on racial bias in policing:

"Racial Bias in Police Investigations" by Jeremy West.

OTHER RESEARCH WE DISCUSS IN THIS EPISODE:

Transcript of this episode:

 

Jennifer [00:00:00] Hello, Probable Causation listeners. Before we get started today, I want to encourage you to support the show on Patreon. For just five or ten dollars per month, you get access to exclusive bonus content, such as interviews with book authors hosted by David Eil and bonus segments with the scholars I interview on the show, talking about their background and life as a researcher. Plus, you'll know that your contributions help keep the show running, something for which the entire team is grateful. To subscribe, go to Patreon.com/probablecausation. There's also a link on our website. Thank you in advance for your support. Now on to today's show.

 

Jennifer [00:00:43] Hello and welcome to Probable Causation, a show about law, economics, and crime. I'm your host, Jennifer Doleac of Texas A&M University, where I'm an Economics Professor and the Director of the Justice Tech Lab.

 

Jennifer [00:00:52] My guest this week is Jeremy West. Jeremy is an Assistant Professor of Economics at the University of California at Santa Cruz. Jeremy, welcome to the show.

 

Jeremy [00:01:02] Thanks for hosting me, Jen.

 

Jennifer [00:01:04] We're going to talk today about your work on racial bias in policing. To set the stage for us could you tell us about your research expertise and how you became interested in this topic?

 

Jeremy [00:01:15] Absolutely. So I'm interested more broadly in questions of how we as societies can design and implement public policies that can basically more effectively improve the lives of people around the world. And actually, most of my research is studying environmental economics and public policy questions in that arena. And I became interested in this great topic of racial bias in policing in a kind of completely indirect way. So just kind of how I get started on this. I was collecting data from multiple states on automobile crashes for a project related to fuel economy and actually realized the setting is nearly perfect when you examine some questions related to police officer behavior. And I've actually learned quite a lot myself about this fascinating area of research along the way.

 

Jennifer [00:01:59] Great, yes. So your paper is titled "Racial Bias in Police Investigations." This is a super important and policy relevant topic, obviously, so I'm excited you're here to tell us about your research. There's lots of public conversation about whether police treat black civilians differently than they do white civilians, for instance. And this conversation has become more urgent in the wake of events like Ferguson and the posting of videos of African-American men who were killed by police, apparently unnecessarily. But quantifying the effect of race is hard. So racial disparities in outcomes like arrests or use of force are hard enough to show that depends on being able to get the necessary data, which aren't always available. But those disparities aren't necessarily driven by what economists would refer to as bias or discrimination. So talk us through this distinction a bit. What do economists mean when they're trying to measure bias?

 

Jeremy [00:02:48] Yeah, so let me just first be clear about what I mean by race, since that's also a pretty loaded term. And just to be upfront that I'm using this purely in the societal sense, not a biological or genetic or predatory characteristic, per se. And actually the data that we'll discuss later, actually, just classifies individuals already as being white, black or Hispanic/LatinX. So that's how I'm very loosely using the word race.

 

Jeremy [00:03:12] To your question on bias. So to economists, bias is really about decision makers, which would be police officers in this case, intentionally choosing to treat some people differently. And this could be favorably; this could be unfavorably, but intentionally choosing to treat people differently beyond what's directly supported by the evidence. So we could contrast this with something like treating people differently because there's a strong evidence base for doing so. So the more formal economic speak is we call these mechanisms either statistical discrimination or some preference or taste based discrimination on the part of the decision maker for treating individuals differently. So one analogy might be political bias. Say a television viewer is less inclined to believe the claims of a particular candidate because that candidate has a history of documented falsehoods. That's probably a reasonable type of statistical discrimination based on prior evidence. In contrast, if the viewer just doesn't believe that candidate because of their political party affiliation that would be preference based discrimination. So for police officers, treating different people differently is almost certainly a good thing in some cases, like if someone is pulled over and they smell very strongly of drugs or alcohol, we'd all reasonably encourage that police officer to test that driver more often for a DUI. But in contrast, if somebody is pulled over and they just happen to be black, that alone is probably not something that most people would encourage, changing the likelihood of testing for a DUI. And the challenge, of course, is an individual's personal characteristics and their likelihood of guilt could be correlated. For example, a young male driver is probably more likely to be carrying contraband than, say, a middle aged woman and separating out what's bias on the part of the police officer, the decision maker here, from what's just simply good old fashioned policing is really the empirical challenge.

 

Jennifer [00:04:52] Yes. So let's talk more about the empirical challenges to studying this issue. So as you've very nicely described, tests for racial bias can be thought of as testing for the causal effect of a civilian's race on the outcomes of some police interaction. That is, to what extent outcomes differ solely due to their race rather than their behavior or some other factor police might be considering. So this puts us firmly in the land of causal inference, familiar territory on the show. So what are the primary hurdles to testing for racial bias in this setting?

 

Jeremy [00:05:23] There's really one primary hurdle here – one flavor of hurdling, kind of multiple flavors, I guess  of the same hurdle – which is that we as researchers only observe the decisions that were actually made by police officers for encounters that actually happened. And so nearly all the academic literature examining police bias is using data from either traffic stops or stop and frisk or some setting in which officers have at some margin determined which individuals end up in the data set to put it that way.

 

Jeremy [00:05:51] So, for example, if an officer decides not to stop a driver or pedestrian, then this possible "encounter" is never observed. It doesn't exist. And so that makes test for racial bias kind of complicated because of this potentially important selection of which individuals, which encounters end up in the data and some prior empirical work has tried to overcome these selection bias concerns by making either statistical assumptions about the distribution of unobserved encounters or kind of relying on some kind of clever instruments in certain contexts. But ultimately, this means that the subsequent analysis is only going to be as credible as the assumption. And these modeling assumptions are ultimately difficult or impossible to test about kind of which encounters did or did not end up in the data set. And this could substantially effect then our ability as researchers to accurately quantify what is actually policing bias.

 

Jennifer [00:06:43] So before the study, what had we known about the presence and magnitudes of racial bias in policing?

 

Jeremy [00:06:48] Oh, boy. So this is a very extensive literature. I mean, as you mentioned earlier, kind of a very important and socially relevant and pressing topic and so kind of broad literature with really prior conclusions kind of all over the map as far as what people have found in this area of work. So one of the pioneering papers in this literature is by John Knowles, Nicola Persico, and Petra Todd in 2001, looking at traffic stops and kind of modeling this objective of police officers. And they actually find no bias against racial minorities in traffic stops. Later work by Shamena Anwar and Hanming Fang in 2006 and Kate Antonovics and Brian Knight in 2009 kind of brings more mixed evidence to this question. But as you mention, there have been many, many related studies kind of all over the place.

 

Jeremy [00:07:38] There's also an interesting branch of this literature. So this kind of selection bias of who's in the data set being a very well acknowledged concern. There's this interesting branch of the literature that's using this clever natural experiment that's been called the "veil of darkness." So the basic idea here is that officers can determine a driver's race just before the sun sets, but we still have a much harder time determining a driver's race through the windshield or whatever after the the sun is set. So any difference in stop frequency's by driver race just before versus just after this veil of darkness closes, can provide some nice identification for bias. And I can mention a couple of leading papers using this technique, one by Jeffrey Grogger and Greg Ridgeway in 2006 and then some more recent work by Bill Horrace and Shawn Rohlin in 2016. But even this technique has been shown to have some rather concerning shortcomings. There's a paper by Jesse Kalinowski, Stephen Ross and Matt Ross in 2017 that kind of shows some concerns where drivers might change their behavior in a way that makes this veil of darkness test perhaps less credible. And so, as you might also expect now, these studies also find mixed evidence as far as the actual conclusion. And kind of collectively as a body this literature provides really no clear overall conclusions about police officer intentional treatment of individuals differently.

 

Jennifer [00:08:58] And, of course, there's a much broader literature on racial bias in the criminal justice system. And I didn't ask you about that because that gets us into a whole other territory where there is plenty of work here just in the policing space. So, yes, you are in this paper, you're using an extremely clever strategy to identify the causal effect of drivers race on the outcomes or to measure racial bias in the setting. So tell us about the natural experiment you've found that you're able to exploit in this paper.

 

Jeremy [00:09:25] Sure, so as I alluded to earlier, I was kind of using automobile crash data and kind of collecting data from a number of states for a question unrelated to police behavior and kind of realize this is actually a really nice natural setting to test for how police officers might be treating individuals differently. So the really nice feature from a causal inference perspective of automobile crash investigations is this dispatch process. So whenever a driver or drivers get into a collision, somebody calls the police or calls 911, and then a police officer is dispatched to investigate the crash.

 

Jeremy [00:10:01] So the important thing here is that the dispatch process is actually an assignment of an officer to an encounter in very stark contrast to the officer selecting for which encounters happen. And that's kind of what's plagued the prior literature, is this concern of selection of which encounters end up in the data set. Here are the encounters certainly are not random overall. Drivers getting to crashes happens for nonrandom reasons, but conditional on a crash happening, which police officers dispatched is actually something I can show is basically random with respect to, importantly, the drivers race. So more specifically, the race or the races of the drivers who happen to be in the automobile crash has no influence whatsoever on the dispatcher's choice of which officer to send. And this is actually pretty intuitive if we think about it, because we should believe that it's going to be the closest officer or maybe the closest, most experienced officer or some other kind of characteristic about who's going to be a capable crash investigator and is available to do so in short notice. That's ultimately assigned to handle the crash investigation.

 

Jeremy [00:11:06] And this is my understanding as well, from talking to dispatchers and people involved with the dispatch process that their decision of how to respond to that 911 call of dealing of an automobile crash is kind of a quick decision about trying to make sure that the situation is addressed as quickly as possible by sending a nearby officer to handle that investigation. And so it could be a black driver, it could be a white driver or both involved in the crash. But regardless, it has no bearing on which officer is ultimately going to be making the decisions about that crash investigation.

 

Jennifer [00:11:39] Right. So unlike kind of stops out on the street in the real world or something like that, unrelated to crashes, the police officer doesn't have a chance to sort of look at the vehicle or the person and decide whether or not they want to be the person who goes to the scene – it's made completely independent of them. So talk a bit more about how you use that natural experiment to measure racial bias here. This is essentially a difference in difference design. So walk us through that difference in difference.

 

Jeremy [00:12:07] Right, so we have this dispatch of the officer, as you mentioned, kind of the nice feature is that the officers' decisions are ones that are made after the encounter is initiated. The officer has been dispatched; they're there at the crash investigation. They had no decision as an officer over the existence of that encounter. But they certainly can make decisions about how that encounter then gets handled subsequent to the dispatch. So we have this basically as good as random dispatch of which officers handling which crash or crashes. So for a crash in a given area, it might be a white officer that's dispatched, it might be a black officer or might be a LatinX officer, et cetera. And so this can show us whether officers of different races behave differently. We've got this as good as random assignment, this independent assignment of officers by the officer's race to crashes with drivers of different races, but ultimately looking at whether officers of different races behave differently doesn't directly get a racial bias. And this could be just because, say, for example, black officers might be more lenient to all drivers. So if you find that kind of independently assign black officers less likely to issue traffic citations, that could be true across all drivers, not specifically any kind of racial bias on the part of police officers.

 

Jeremy [00:13:27] So, as you mentioned, I use this difference in differences technique to get racial bias more directly. And so this is a technique I should mention has been used in prior literature, both for police officers and in many other contexts, looking at racial bias in and all kinds of settings in the economics literature. And so the basic idea here is that we can look at what happens when a driver gets dispatched an officer of his or her own race compared to one of a different race while controlling for any kind of broader or general differences by driver race and propensity to be guilty or by officer race and the propensity to be more lenient or harsh in writing citations. So, for example, as I mentioned earlier, if black officers are just more lenient to all drivers we can control for that in this difference in difference idea and see whether black officers are more lenient specifically to black drivers, even more so than they are to all drivers is kind of the idea here.

 

Jennifer [00:14:20] And you have data from a large anonymous state patrol agency which you merge with a few other data sets. So tell us about this full, very detailed data set that you are able to use in this paper.

 

Jeremy [00:14:34] Yeah, so my data agreement, unfortunately, won't allow me to name the state, but as you note, it's a large state police department. So you can think of these as like highway patrol they're called in some states or state troopers in other states. So this is mainly highway and more rural automobile crashes, which we could think might differ in some ways from, say, local police. But there's also an important aspect where unlike local police where, and this certainly varies by jurisdiction, there's often multiple officers involved, in investigations for state police, it's usually, it's almost exclusively single officer patrol. So even if the crash investigation has multiple officers there, there's going to be a primary officer who handles the investigation, is making all the decisions, and interviewing drivers and so on. And then my understanding is that any other officers at the scene are just there basically for traffic control and and kind of managing the broader scene while the main officer is doing the investigation.

 

Jeremy [00:15:31] So the data that I used this paper is this very detailed data on all automobile crashes handled over a number of years by this large state police department. And this is kind of everything you could imagine might potentially be recorded about an automobile crash - the weather conditions, the road where it happened, all the things about the crash itself, all the things about the individuals involved in the crash that might be relevant, like their cars, their individual characteristics, like the race and age and gender. And so this is where I'm getting my measure. Race is something that's already recorded in the data. And then also any decisions that the – any official decisions that the police officer made in particular, like writing different types of citations potentially of different drivers. So the data, the crash data include each investigating officer's name and badge number, but of course not the officer's race, since that would not be a really relevant thing to record for a crash investigation. So, as you mentioned, I kind merge different data sets together to obtain this. So what I use for the officer's race is a Freedom of Information Act request for state police personnel records, which do include the officer's race, and then I can merge this by the officer's names in order to determine which race officer is at, which crash, with which race drivers.

 

Jennifer [00:16:53] And then you also have information on the cars, am I remembering that right?

 

Jeremy [00:16:56] Yes, yes. So the crash data include – well the important thing that I use for the cars actually include the vehicle identification numbers. This is the kind of unique number that's stamped on your windshield and everywhere else and in an automobile. And so I use a personal licensed vin decoder software that allows me to then take that number and convert it to all kinds of interesting things about the car, like how old it is and what its value would be, which I'll use for some of my analyses we can talk about later.

 

Jennifer [00:17:27] Great. And then so what are the outcome measures that you focus on in this paper?

 

Jeremy [00:17:33] So the main decision that officers are making at a crash investigation is what happened. And then ultimately what this means officially is which driver, if any, to write a citation for and which offense or offenses that driver or drivers might be guilty of. So in many cases, there's actually no citation written. So it's, maybe it's a misconception – I've at least heard people voice before when I've presented this work that, you know, some driver has to be cited in the crash. And that's certainly not the case. There are a large portion of the crashes in these data where there's no citation written, regardless of how many drivers are involved. And police officers similarly are not actually deciding who's kind of "at fault" for the collision. So that's an insurance term. And certainly the police officer's actions, like writing citations, will affect how insurance companies treat the "at fault" status for the for the crash. But the main thing officers are doing is determining who broke the law. If somebody did break the law and trying to make sure that people are appropriately cited for breaking that law.

 

Jeremy [00:18:34] And a driver could actually be cited for multiple offenses. For example, you could be cited for running a stop sign and driving without vehicle registration or with expired registration. And so the main outcomes that I look at in these data are citations that the officers wrote to different drivers involved in crashes. And I can look at all citations. So, for example, did a driver get any citation in the crash, and then I can look also at different types of citations. So there are actually a number of very specific citations, like running a stop sign would be one of them. But I primarily categorize citations into three kind of well-established types of violations that people can commit in an automobile. The first category is moving violations. So this is like running a stop sign or failing to yield right away in these types of violations that actually involve moving the automobile. The second type of citation is for non-moving violations. And this actually gives me a lot of traction as we'll discuss later. But these are things like driving with an expired license or an expired vehicle registration and things that the driver clearly had made a decision to do prior to getting to that crash, not renewing the vehicle registration, for example. And then the third type of citation I examined is felony offenses, which is a pretty small minority of total offenses and citations in the crash data. But this would be stuff like vehicular manslaughter. And this is ultimately something I use basically as a falsification test as we can discuss later as kind of these are the types of citations that you think officers probably should have very little discretion over. If there's vehicular manslaughter involved in the crash, it's going to get much more oversight formally from higher up in the administration.

 

Jennifer [00:20:21] Right. So it doesn't matter what the race of the driver is, they're going to get – they're going to be arrested for manslaughter. Great. So tell us about the main results. What do you find is the effect of interacting with an other race officer on each of those outcome measures?

 

Jeremy [00:20:34] So overall, I find significant racial bias in traffic citations as this difference in differences strategy yields estimates that indicate police officers are much more likely to issue citations to drivers whose race differs from their own. And to give you some more quantitative magnitude that's about a 3 percentage point increase in the likelihood of citing a driver of a different race  than the officer's own race relative to a kind of baseline average of about 45% of the time a driver gets a citation. So that's about a 6% increase in citation likelihood if the officer's race just happens to – the dispatched officer's race just happens to differ from the drivers. And so this findings really robust to including very detailed controls in this crash data specific to the crash and the driver, kind of everything you can imagine in that crash data set - the type of road, weather conditions, drivers, age, and so on - just tossed into this regression. It doesn't change things at all. Similarly, these results are robust to choosing within officer variation. So this is formally having a fixed effect for the officer. But really what this means is just following the same officer over time. Sometimes this officer is dispatched to crashes of the black driver, sometimes with a white driver, sometimes with a Hispanic or LatinX driver. And it kind of robustly shows the same pattern of citation behavior.

 

Jeremy [00:21:56] And then finally, I really push the data far by actually using within crash variation and having a fixed effect for the crash. So this can only be identified now from drivers of differing race crashing with each other. So say a black driver and a white driver get into a crash of each other. It turns out that it really matters for both those drivers, which race officer is dispatched as far as who is who, if anyone is going to get a citation. And I do find this bias is present as far as the categories of citations for nearly all citation types, both moving and non-moving violations. But as I noted earlier, not for felony violations. So there is some interpretation here – might be that the officers are certainly making a discretionary choice with a non-moving violation, since the only way you can be guilty or not – the only way you could not be cited for a non-moving violation you were guilty of is if the officer chooses not to cite the driver, which is de facto appears to happen quite a bit. And then the interpretation for felony violations could just be because these types of situations are very serious. So maybe officers are not willing to exhibit any bias when it is a very serious situation or could it just be because felony violations like vehicular manslaughter get very formal investigations from higher up in the policing agency, and so there's certainly a lot of oversight for those types of conditions.

 

Jennifer [00:23:18] So how should we think about these other race interactions? Are these in practice usually black or Hispanic drivers stopped by a white officer, or is it a mix of other pairings? And I guess, to get more to the point, can you can you say anything about the extent to which certain driver officer pairs are driving these effects?

 

Jeremy [00:23:36] So one thing I kind of glossed over when we were talking about the strategy of difference in differences is that I'm examining all these crashes within very small local areas. So specifically, what I have is a census block group fixed effects for the area of the crash. So it's controlling for anything that's kind of heterogeneous or varies across different communities within the state that the state police agency I'm examining patrols, but it's a pretty tricky question to answer how different race interactions might be driving the results. And ultimately, the difference in differences strategy doesn't really facilitate saying, "Okay, is it white officers who are disproportionately driving this racial disparity or is it black officers who are disproportionately driving this disparity?" The results clearly show that having an own race officer dispatched to the crash scene is favorable to a driver relative to having an officer whose race differs from the drivers, but making claims about which races of officers drive this pattern it's just not something the method can test for. I guess I can say that black officers are the most lenient across the boar;  white officers are the most harsh across the board; and Hispanic/LatinX officers are somewhere in between. And kind of in line with this broader pattern of behavior. A nonwhite driver with a white officer does have the highest likelihood of a citation across kind of all possible combinations I examined. But the difference in differences method just really isn't well positioned or can't really facilitate a test for kind of absolute degrees of bias. This is only a measure of relative bias in crash investigations.

 

Jennifer [00:25:15] Yeah, and that's a good point. I guess so – it sounds like there's a lot of variation here. You do have a good number of black officers. You have a good number of Hispanic officers. So it's not just white officers in your data.

 

Jeremy [00:25:26] Yes, yes. So one of the things I think it made into the appendix is using this kind of borrowing technique from the Dutch organization literature is showing basically the market share for lack of a better term of officer race in different areas. And there's quite a lot of variation across the state. It's not the case that, say, some communities are exclusively black officers and other committees are exclusively white officers. Those would be the extreme cases, but there's lots of variation of kind of mixture between. 

 

Jennifer [00:25:56] Great. Okay, so the primary challenge in papers like this as we've sort of alluded to already is to convince yourself and readers that you've isolated the effect of race on the outcomes of interest, so that the effects you're finding aren't driven by other factors that might be correlated with race, things like wealth or how drivers respond to the officers. So tell us about the various robustness checks you run in the paper and how you can rule out some of those other stories that could be driving racial disparities.

 

Jeremy [00:26:23] Absolutely. So, as you mentioned, one of the major challenges in claiming evidence of racial bias is that people's race is often correlated with other characteristics like their economic means. So what could appear to just be bias by police officers on the part of individuals' race might actually just be attributable to treating, say, poor individuals differently than wealthier individuals. So one of the cool aspects of the data that we briefly discussed is that I actually observed the vehicle identification number for each car. And then that allows me through this merge to the vin decoder data to have this nice proxy for individuals' economic means by using the value of their automobile.

 

Jeremy [00:27:04] Obviously, this is not a perfect proxy and it would be more ideal to actually observe individuals' wealth or their income or something like that. But at least this automobile value or automobile age can serve as a nice, tractable proxy I can use in the data. And so what this lets me do is then run this same test for racial bias. This difference in differences estimate across drivers with vehicles of very different values. So, for example, what's the estimated racial bias for drivers with really, really low value cars that are virtually worthless compared to drivers of very expensive new cars? And what I find is actually, interestingly, that the estimated racial bias is basically completely invariant to vehicle age or value. So certainly drivers with older cars get citations more often. The new cars, drivers with high value vehicles are cited much less often than drivers of low value vehicles. But the racial bias doesn't vary at all with the the value or the age of drivers' vehicles. So that's a nice way of addressing this potential confounding variable of race being correlated with income or wealth or these other measures that might ultimately – or apparently do also – factor into citation propensities. And then I do some other tests as well, looking at things like how racial bias varies with drivers' age or drivers' gender. And again,  examining how this varies of other driver characteristics also shows that the racial bias doesn't seem to depend on these other factors, like driver age or gender. So to summarize, I basically find that officers cite other race drivers more frequently, regardless of their age, their gender, their vehicle value or the characteristics of the local community of the crash.

 

Jennifer [00:28:38] And you make the argument in the paper that I found pretty compelling, that, you know, one possible story here is that maybe drivers interact with other race officers differently. Maybe they're, you know, they push back more or more aggressive or something. And that could be what's leading to the officer to treat them differently. And you argue that if that were the case, we would expect to see at least some differences across age or gender or something. Is not everyone – surely not everyone is behaving exactly the same way toward the officer. Am I getting that right?

 

Jeremy [00:29:05] Yeah, exactly. So ultimately, it's the officer making a decision about whether to write a citation or not. But that decision, of course, could be influenced by how a driver treats the officer. For example, a driver is more inclined to argue with an officer whose race differs from their own. Then this increase in that propensity for a citation in that interaction could be attributable to the driver just mouthing off to the officer more often in those cases or being more aggressive or something like that. So the extent that that happens, it's pretty hard to make a claim that it would be kind of uniformly happening across driver, you know, old women drivers versus young male drivers. We probably wouldn't expect that they'd be equally likely to mouth off to the officer based on how that racial interaction looks like. So the fact that I find this racial bias estimate is invariant to  other driver demographic characteristics, to me at least, is pretty convincing evidence that while driver behavior certainly could be part of the story, it's certainly not the major factor that's driving this this disparity in citations.

 

Jennifer [00:30:09] You note that your results are in line with the hypothesis that officers' racial bias reflects leniency in favor of drivers of their own race rather than unwarranted harshness against drivers of other races. So what do you mean by that?

 

Jeremy [00:30:23] So this is one of the places where having these different categories of citations brings the most value to the study. And in particular, the non-moving violations are a really giving me a lot of traction here. So a moving violation in a crash investigation is something that potentially could be endogenous for the encounter. So, for example, say a driver, and it happens a lot, gets cited for speeding. Well, this isn't like a speed trap where the officer was sitting there with a radar gun and documented clearly that the driver was speeding. This is a crash investigation. The officer shows up after the fact when he or she is dispatched to the crash scene. And so giving a driver a citation for speeding is really kind of a he said she said kind of situation where the officer has to determine or maybe a driver admits that they were speeding and that's what led to the crash. So in that case, it might be that, for example, maybe Hispanic drivers just have a harder time communicating with white officers or something like that, and so we wouldn't want – or at least moving violations don't provide as clear evidence that necessarily this is either harshness or leniency in officer behavior.

 

Jeremy [00:31:35] We can contrast that with non-moving violations, where I think I get a lot of traction in that respect. And that's because nonmoving violations are things that are very salient and objective from the officer's perspective. So whether a vehicle registration is expired is something that the officer can see just by looking at the outside of a vehicle. It's something that the officers required to record the registration details as a routine part of any crash investigation. So this is information the officer is clearly exposed to and then might use in making their decision. And so the fact that I find a disparity in non-moving violations or this racial bias in non-moving violations  is pretty clear evidence of leniency because there's only one direction that that difference in citations can go. Officers can't write people a citation for expired vehicle registration if their vehicle registration is actually current, because obviously anybody would take that to the court and that officer would face repercussions from that. So the disparity, this gap in citations for non-moving violations can only be attributable to leniency or it can only operate through officer leniency.

 

Jennifer [00:32:40] You also argue that the racial bias you're documenting is due to statistical discrimination rather than taste based discrimination. And I know you defined these before but one more time for the listeners who aren't familiar with this terminology. Statistical discrimination is using race as a proxy for some unobservable characteristics you care about, for instance, whether the person actually committed a crime while taste based discrimination is focused on race itself. So talk us through your reasoning here. Why do you think your estimates represent taste based discrimination?

 

Jeremy [00:33:09] So again, here I'm going to rely pretty heavily on these non-moving violations. And so I should be clear that I'm not in any way trying to claim there's no statistical discrimination in this setting, just that I think that it's pretty convincing to me that the evidence is supporting that at least some of the bias I'm documenting, some of this difference in treatment of individuals with respect to the racial interaction, is through some taste based mechanism or some preference of the officers. And so the way the non-moving violations facilitate this is, as I mentioned a few minutes ago, the only way an officer can treat individuals differently for non-moving violations like expired vehicle registration is through leniency. And so if officers are being more lenient to some individuals than others then it's hard to justify that as being statistical discrimination. This isn't like speeding or something else where the officer might say, you know, maybe a younger driver would have been in general more likely to be speeding than an older driver. And so even though the officer didn't directly observe speeding, it could still be some statistical discrimination, some reasonable belief on the part of the officer about about guilt for speeding that led to the crash. Contrast that with expired vehicle registration. The officer clearly knows what the truth is. And so if they're choosing to be lenient and in some cases with some individuals more often than others then that's a pretty clear evidence that the officer has a preference or a choice in doing so.

 

Jennifer [00:34:37] Yes, I think the statistical discrimination story for like the non-moving violation would have to be something like I'm not only trying to tell if you've actually violated the law by not updating registration, but I'm also trying to tell you know how serious you are about planning to go update that tomorrow. Right. Or like you were actually on your way to the DMV right now, and I'm trying to decide whether how likely that is to be true. But that's sort of a separate issue from whether you actually deserve the citation. So, yeah, I think I am on board. I think it becomes a more convoluted story to say that this is statistical discrimination.

 

Jeremy [00:35:13] Especially when you think of it as relative racial bias. So if this is the case that maybe there is evidence, I don't know, but maybe say hypothetically, white drivers are more likely to actually follow up on taking care of that registration tomorrow then minority drivers are it's hard to see how that difference in treatment that would be based on data about our historic propensities for following up on the vehicle registration would actually vary by officer race. Why would it matter whether it was a black officer or white officer if it's this channel of 'oh some individuals are more credible when they say that they're going to get this taken care of tomorrow.'

 

Jennifer [00:35:51] Yeah, that's a great point. So you released this research as a working paper a couple of years ago, but this is, of course, an important area where lots of researchers are focusing their energy. So let's talk about what else – what other papers are out there? What other research has come out since you first wrote this paper that helps shed light on these issues?

 

Jeremy [00:36:10] So, as you mentioned, the criminal justice system overall has a lot of work looking at racial bias, kind of all different parts of that process or set of processes. One area in particular that I guess I could focus on is that there's this small but growing recent body of research examining how police officers change their decision making over time as they gain experience. And I'm going to focus in particular on this through some self-interested reason that I've contributed to this literature. But I also think this is a particularly interesting area to explore and has a lot of policy relevance as we think about kind of moving from documenting problems like racial bias and disparities in police officer treatment to transitioning towards solutions. Okay, what might actually work at trying to mitigate these societal problems? So a couple of papers in this area, Greg DeAngelo and Emily Owens have a paper in 2017 showing that police officers, as they gain experience and familiarity with traffic patrol change how they write citations and kind of which laws they enforce or the intensity to which they enforce different laws.

 

Jeremy [00:37:22] Bill Horrace, along with some coauthors, has some work looking at how police officers change the degree of racial bias they exhibit or how that varies with their patrolling experience. And then I have a recent paper myself. It's looking at the idea of learning by doing by police officers. So does the act of policing itself as a form of experience influence how officers make choices? And in that paper, I'm actually looking at this setting we talked about is this scary endogenous world [00:37:54] [0.0s] of traffic stops. So there certainly is this selection there of which drivers get stopped. But I actually find in that paper that officers have the exact same search rate of drivers  as the officer accrues experience. The higher levels of experience don't impact how often the driver searches, sorry, an officer searches stopped drivers. But it does affect their hit rate for finding contraband in a traffic stop quite substantially. And in particular, more experienced officers are much more likely to be successful in a traffic search. And to kind of dig into the mechanisms of then tie it back into racial bias, I look at how that improvement that correlates with officer experience might operate through various channels. And two channels in particular are the officer could just be statistically discriminating more often as they gain experience and I actually find no evidence of that. The very minimal evidence that there's any reallocation of searches across drivers of different demographic groups and instead I'm attributing that improvement that's from officer experiences operating through this channel of just kind of cognitive or non-cognitive ability at kind of determining which people are likely to be guilty. And kind of maybe an analogy here would be like a poker player as they gain experience is better at reading other poker players tells; police officers as they gain experience, get better reading which drivers are truly guilty and carrying contraband versus those that aren't. So I think kind of a natural avenue for ongoing research, as well as kind of looking at what channels might we lean on or what points of leverage might we have is influencing policies that might actually improve the equity and efficiency of policing.

 

Jennifer [00:39:34] So putting it all together and the results of this study and the other studies you mentioned, what are the policy implications of this work?

 

Jeremy [00:39:43] So one thing that I think is interesting, and this is leaning a bit on the lack of any racial bias for these felony violations, is that oversight is probably very important. And there's a lot of literature elsewhere, less that I'm aware of in the policing setting, but certainly in other contexts where economists have looked at differences in oversight, like auditing the behavior of government agents, for example, or monitoring their behavior in various ways, as being a way that can reduce the use of taste based discrimination or create more equitable and efficient treatment of different individuals.

 

Jeremy [00:40:26] Certainly the issue we mentioned we can get experience learning could facilitate interventions towards trying to make sure that officers are better trained. Or maybe these two officers patrol idea. If you can spare an experienced officer with a rookie than that can make that rookie kind of move up this experience learning curve faster as they can learn from the more experienced partner that they're working with, can be one policy implication. And then I think reduce discretion is a big one as well. So I have a paper with Justin Marion that we just finished up looking at environmental hazardous waste cleanup. And we find in that setting that discretion on the part of the agents of the government turns out to be a really big factor in racial disparities there and how the quality of the clean up results. So there's other work as well showing that agent discretion is an important factor. And in the case of police officers, they have lots and lots and lots of discretion in making their decisions as they do their job. So perhaps there's an avenue for technological improvements or advantages there that would kind of take some of the discretion away from decision makers in cases where we're concerned about bias.

 

Jennifer [00:41:37] And just to elaborate on that a little bit, there is definitely evidence from a variety of settings, including the criminal justice context, that the more you allow human discretion to enter into the equation, the more you're going to get disparities on things like race and gender. So Crystal Yang has a nice paper looking at this in the sentencing context. But yeah, obviously individual human beings don't like giving up their discretion. So it becomes somewhat of a political process to move us in that direction. But if we want to close racial disparities, removing the human piece from the decision making can be helpful.

 

Jeremy [00:42:13] It's worth noting that algorithms have also been shown to be racially biased as well. So there's probably no silver bullet.

 

Jennifer [00:42:19] Sure. Yes, this is going to be very complicated. Yes, indeed.

 

Jennifer [00:42:23] Yeah. So you have already alluded to this a little bit, but what do you see as the research frontier here? What are the big open questions in the space that you and others will be thinking about in the years ahead?

 

Jeremy [00:42:35] So in my view, we have basically a ton of evidence now of all degrees of credibility that police officers, not always, but in some cases do intentionally choose to treat individuals differently based on demographic factors like the individual's race. One way of saying that is we've really kind of documented this problem quite conclusively. I know the existing literature is kind of all over the place, but there's enough evidence on the scale in favor of there being a problem in some cases that is kind of justifying transition towards what is the solution. And obviously, that's almost always the harder question. Economists are really good at pointing out problems. It's much harder to say here's a really effective solution and show concrete evidence of that in practice.

 

Jeremy [00:43:20] And certainly I should say, not all police officers are biased and certainly not in all cases, but there's enough of a pattern of behavior that this warrants some intervention or some societal policy changes, especially if we as society want equitable and efficient policing. We need to figure out how do we change this behavior. We talked a little bit about this with maybe removing discretion and things like that. But I think kind of broadly speaking, the big open questions are how do we fix this. And two ways could be changing how police officers make the decisions, like removing some of the decision making discretion from the officers themselves, by using technologies and algorithms and that sort of thing. But the other ways are training and education in ways to try to change how the decision makers, the police in this case, make their decisions without removing the decision making from them directly. But there's no silver bullet here. Some of the approaches that have been used to try to diminish bias, like police officer body cameras, for example, which do provide increased accountability and review, have quite mixed results as well in the research literature as far as their efficacy in actually influencing outcomes of concern. And other solutions, like two officer patrol have been mentioned in many cases to have some importance. But these are expensive to implement. And so it's a difficult budgetary justification to make to policy policymakers. And I guess I'll just say overall, I think the research frontier for these topics is really moving from studies that document the problem to studies that showcases solutions. And I look forward to seeing this research and ultimately the improvements in the world that come about from it.

 

Jennifer [00:44:57] Same here. I completely agree that's a really important research frontier and one I'm watching closely. My guest today has been Jeremy West from UC Santa Cruz. Jeremy, thanks so much for doing this.

 

Jeremy [00:45:08] Thanks again.

 

Jennifer [00:45:14] You can find links to all the research we discussed today on our website, probablecausation.com. You can also subscribe to the show there or wherever you get your podcasts to make sure you don't miss a single episode. Big thanks to Emergent Ventures for supporting the show. And thanks also to our Patreon subscribers. This show is listener supported, so if you enjoy the podcast, then please consider contributing via Patreon. You can find a link on our website. Our sound engineer is Caroline Hockenbury with production assistance from Elizabeth Pancotti. Our music is by Werner, and our logo is designed by Carrie Throckmorton. Thanks for listening and I'll talk to you in two weeks.