Probable Causation

View Original

Episode 21: Aurelie Ouss

Jennifer Doleac

Aurelie Ouss

Aurelie Ouss is an Assistant Professor of Criminology at the University of Pennsylvania.

Date: January 21, 2020

Bonus segment on Professor Ouss’s career path and life as a researcher.

A transcript of this episode is available here.

See this content in the original post

Episode Details:

In this episode, we discuss Professor Ouss's work on nudges and failure-to-appear rates:

"Nudging Crime Policy: Reducing Failures to Appear for Court" by Alissa Fishbane, Aurelie Ouss, and Anuj K. Shah.

Related policy paper: "Using Behavioral Science to Improve Criminal Justice Outcomes: Preventing Failures to Appear in Court" by Brice Cook, Binta Zahra Diop, Alissa Fishbane, Jonathan Hayes, Aurelie Ouss, and Anuj Shah.

OTHER RESEARCH WE DISCUSS IN THIS EPISODE:

Transcript of this episode:

 

Jennifer [00:00:00] Hello, Probable Causation listeners. Before we get started today, I want to encourage you to support the show on Patreon. For just five or ten dollars per month, you get access to exclusive bonus content, such as interviews with book authors hosted by David Eil and bonus segments with the scholars I interview on the show, talking about their background and life as a researcher. Plus, you'll know that your contributions help keep the show running, something for which the entire team is grateful. To subscribe, go to patreon.com/probablecausation. There's also a link on our website. Thank you in advance for your support. Now on to today's show.

 

Jennifer [00:00:42] Hello and welcome to Probable Causation, a show about law, economics, and crime. I'm your host, Jennifer Doleac of Texas A&M University, where I'm an Economics Professor and the Director of the Justice Tech Lab.

 

Jennifer [00:00:53] My guest this week is Aurelie Ouss. Aurelie is an Assistant Professor of Criminology at the University of Pennsylvania. Aurelie, welcome to the show.

 

Aurelie [00:01:01] Thank you. Thanks for having me.

 

Jennifer [00:01:03] Today, we're going to talk about a field experiment that you ran in New York City using behavioral science and nudges, including text messages to reduce failure to appear rates among those who received a court summons. Before we do that, though, could you tell us about your research expertise and how you became interested in this topic?

 

Aurelie [00:01:22] Absolutely. So I have been conducting research on the economics of crime since before I started my PhD. That was the topic that I was most interested in. I wanted to learn how economics tools could help us figure out what crime reduction policies worked, how we could design an effective and a fair criminal justice system. And then more recently, I got interested in research at the intersection of behavioral science and crime policy. Specifically, I was interested in learning more about when the traditional incentives that we think about work to reduce crime and when criminal justice actors thinking about judges, juries, police officers actually think about incentives when they're making their own decisions versus when are people making mistakes or deviating from this cost benefit analysis, which is one of the guiding tools we have when in economic theory. So I, you know, there had been some work in behavioral science, a growing body of work, showing success across different fields—health, personal finance, environment, education—lots of really encouraging work showing that when we are more cognizant about mistakes that people make, we can develop policies that either take these into account or remedy them or leverage them to design better policy. But that's not something we were doing in crime policy. So I got interested in figuring out whether these approaches also worked for criminal justice policy.

 

Jennifer [00:03:08] Your paper is titled "Nudging Crime Policy: Reducing Failures to Appear for Court," and it's coauthored with Alissa Fishbane and Anuj Shah. And you're focused on people who get summonses in New York City. So tell us about the summons process there. What's the set of offenses you might get a summons for? And what is the process that a person needs to go through in order to deal with a summons in New York?

 

Aurelie [00:03:31] So this differs across jurisdictions. I feel like we often talk about the criminal justice system, but there's actually tons of local criminal justice systems. And so it really varies from place to place. In New York City, which is where our study took place, summonses are the lowest level offense that one can be- can have interactions with the criminal justice for. And in other places they may be called things like citations. The kinds of offenses that get you a summons are quality of life offenses. So the most frequent one would be open containers. About a third of summons recipients got- were ticketed for an open container. Other frequent summonses are things like park trespassing or cannabis. And that's actually interesting because cannabis used to be one level up and it recently became a summonsable offense as New York City was moving towards less criminalization of cannabis. So a person, when they get a summons, what happens is they are given by a police officer a ticket. And their next step is that they have to show up to court for this offense. And that's basically it. So they are just given this ticket, they have to show up to court, and this is the next step for this kind of offense.

 

Jennifer [00:05:01] And then once they do show up in court, is it usually a fine or a fee or is it something someone might go to jail for?

 

Aurelie [00:05:08] It is not something somebody may go to jail for. Typically, they may get a fine or a fee or their case may be dismissed or no no jail time in, you know, the vast majority of cases, just for summonsable offense, one would not face jail time.

 

Jennifer [00:05:24] OK, and the problem you're trying to help the city solve is that a lot of people who get a summons don't show up to court. So talk about the problem of failures to appear, FTAs for short. How big a problem is this nationwide and in New York City in particular, what do we know about the people who aren't showing up to court each year? And what happens if you miss your court date?

 

Aurelie [00:05:44] So, again, this is a great question and the answer would really differ jurisdiction by jurisdiction. So nationally, the failure to appear rate for felony defendants is around 20 percent. And I think these numbers are for 2016. So the failure to appear rate is is is quite large, even for felony offenses, which are the most serious types of offenses. In the context that we're looking about, summonsable offenses, the failure to appear rate was even higher. 41 percent of people, before we started working on this project, were not showing up to court for their for for their court date. They were missing court. And so, you know, the the consequence for not showing up to court is actually that a new offense is added to you. So an arrest warrant is open. And that means that the next time you interact with law enforcement, you could be arrested for having this open warrant. So even though, and this is the case also for summonsable offenses. Right. So even if the initial thing that I did may not appear to be a big deal, if I don't show up to court, then I have an open warrant. And so the police officer can arrest me for having this open warrant.

 

Aurelie [00:07:10] And I think another consequence is that if I'm arrested for a more serious offense, then a judge may be figuring out whether I should be in jail, pretrial or not. And in many jurisdictions, including New York, the one thing that a judge will take into account when making this pretrial decision, the thing that they're supposed to put their weight on is whether I am likely to show up to court or not. And if they think that I'm likely to not show up to court, then I may be accruing some points or the judge may think that I'm at higher risk. So either with a formal point system or more informally, that's what their perception is. And in turn, this means that having missed a court date, the best predictor for missing a future court date is having missed the past court date. And so this failure to appear even for for, again, a small summonsable offense, quality of life offense, can result in me being more likely to spend time in jail pretrial.

 

Aurelie [00:08:15] And then lastly, there can be some consequences of having an open warrant. This, again, varies by jurisdiction, but there can be some consequences in terms of access to some social welfare. There can be some immigration consequences for people who are not U.S. citizens. So having an open warrant itself can just trickle in a lot of different kinds of problems.

 

Jennifer [00:08:38] So before this study, what had we known about what drives FTAs and how to reduce them?

 

Aurelie [00:08:44] So that's a great question, and I think, you know, there's a twofold answer to this question. One, there is some descriptive work interviewing people and trying to understand when they have missed court dates or when they had shown up to court what was driving their decision. And I can tell you a little bit about work that others have done. But also this project was done with a with a team of people, so you mentioned my coauthors Anuj Shah and Alissa Fishbane. I should say this was a randomized controlled trial, and as with these kinds of projects lots of people were involved, particularly we had some terrific research assistants: B. Diop, Bryce Cooke and John Hayes and others in the- in our team as well. And ah, the- Alissa Fishbane is affiliated with the lab, ideas42, and their team conducting some- conducted some interviews to ask defendants why they had shown up to court or why they hadn't shown up to court. And it's really interesting to see the range of responses that people would say. Some people would bring up things like fears of not being treated fairly by the criminal justice or would bring up motives like, you know, feeling that the kind of offense for which they were asked to show up to court was not legitimate. Some mention that they didn't really know what the consequences of not showing up to court was. They didn't think it was a big deal. So some just seemed like they didn't have the right information. Some people also mentioned some logistical barriers that were holding them back from showing up to court, things like how do I- how do I arrange to not miss work? How do I figure out child care? How do I concretely show up to the court? So it's a mix of mundane reasons and of more profound reasons or barriers that were preventing people from showing up to court.

 

Aurelie [00:11:02] What's interesting to me is when we look at our current criminal justice policy, what are the tools that we have for people to show up to court. Things that- the most frequent kinds of policies that we have are first pretrial detention that mechanically brings failures to appear down to zero because the person is in jail pretrial awaiting for their trial. And about 20 percent of people who are behind bars in the United States have actually not had their trial yet, their pretrial defendants. And then the second very frequent tool is monetary bail. Right. And the idea behind monetary bail is that if people have some financial collateral, then they will have skin in the game. They will be more likely to want to comply. And in a separate project, that's a topic that I've studied with Megan Stevenson. So if we look at our policies, our policies really have this flavor of either mechanically and at very high cost to society, up to the defendants, making sure that people come up to court or trying to change people's incentives for compliance by adding some form of either supervision or financial collateral for them to fail. So while both the interviews that we conducted and past research- you know, there's one study in particular in Nebraska—which tested whether sending people different kinds of postcards to show up to court was effective in reducing failures to appear—pointed at more mundane barriers or crime policies tend to insist more or to leverage more changing incentives for people to comply.

 

Jennifer [00:12:57] So you frame the study as a test of whether people who don't show up to court are actually rational decision makers who are responding to incentives or whether they're simply making mistakes, as you just described. So you and your coauthors discuss a few possible reasons that people might fail to appear in court other than the standard economist's explanation that it's a rational choice. So what reasons do you have in mind there?

 

Aurelie [00:13:20] Absolutely. And I should say, again, like I think sometimes the rational choice agent is a little bit of a straw man. I think in the context of criminal justice policy, even if people aren't explicitly using the rational agent model in terms of how they think about why people commit offenses, de facto, the kinds of policies that we have would work if a person were thinking about the costs and benefits of their actions. So I gave examples from showing up to court, but think about other policies, such as more policing, longer sentences. These also work on these same levers. We want to increase deterrence and deterrence works if people are actually paying attention to the costs and benefits of their actions. And then on the flip side, if you think about policies such as job training or longer education, then that seems to be improving the outside option to committing crime. So, again, trying to change people's calculus. And I don't want to give the impression that I don't think these policies work. First, there's an extensive literature showing that these policies do tend, with with differing varies- with varying degrees of success, to reduce crime. But our hypothesis is that as a society, we haven't really thought about other things that can be driving some forms of offending. And so we're not developing policies that could help address that side of the problem.

 

Aurelie [00:15:01] And so the one mechanism that we focus on in this project is to think about inattention. So I mentioned previously other reasons why people may be not showing up to court, such as a financial or logistical barriers towards showing up to court or a lack of trust in the courts. This is not something that we test directly. What we test is whether people may simply not be paying attention to their court date. Right. They don't take the extra minute to figure out, OK, concretely, what am I supposed to do to show up to court? Or they forget about their court date and nothing reminds them that they actually have to show up to court. So these are the two- so this inattention barrier is the main one that we're focusing on in this project.

 

Jennifer [00:15:53] So you consider the effects of two interventions in the study. One is redesigning the summons form to make the important details more salient to people. And the second is sending text message reminders to people who gave you cell phone numbers. So let's talk about each of these. First, tell us about the form that you get when you get a summons and how you and your research team changed it.

 

Aurelie [00:16:16] Absolutely. So the form that people- that defendants used to get when they weren't- when the police officer handed out to them, it's informally called a pink ticket. And think about any administrative form that you got in the past. It had a lot of information. So the way it was organized, the first I want to say third of the form described the offense and provided information about the person who was receiving the ticket—so things like their address and their date of birth, things like that. And it wasn't until the very bottom of the form that their next steps, which is to say that they had to show up to court and where they were supposed to show up appeared. And then it wasn't until the back of the form—so a defendant would have to flip the form and look at the other side to see—that if they didn't show up to court, they would get an arrest warrant.

 

Aurelie [00:17:18] So the forum was organized in a way where the really important information, namely that they had to show up to court on this day, at this time, and if they didn't show up to court, they would get an arrest warrant was buried within the form. And so we worked to make several changes to the form. But in particular, the way the form is currently organized is that the first thing that shows up is- tells a person this is a criminal appearance ticket. You must show up to court on this date at this place and in order to avoid a warrant for your arrest, you must show up to court. Right, and so all of this information is just made very clear at the very top of the form. So the idea behind this intervention was to help reduce the amount of attention necessary to get the relevant information. So once again, all of the information was already there in the old form. It just took, you know, maybe if a person wanted to look for this information, it may take them something like 30 seconds to a minute to find it. Whereas on the new form, it just jumped out at you from the very start.

 

Jennifer [00:18:32] Yeah. And I've seen you present this paper so you can actually show images of the two forms side by side up on the screen, and it is extremely striking. Like the old form is just a convoluted mess of boxes to fill in that you really have to sit down and really focus on to figure out what's in there. And the new form, it really is highlighted like this is your court date, you will get an arrest warrant if you don't show up. So you can see very clearly how that could make a difference. OK, so then next, you used the cell phone numbers that people gave you, or gave the police, to test an additional intervention, text message reminders, and you actually tested a few different types of messages. So tell us about each of those.

 

Aurelie [00:19:12] Yes, so, you know, I think we were interested in two big picture questions. First, we wanted to know whether receiving any text message could help reduce failures to appear in court. You know, I think nowadays everybody who goes to the dentist and possibly also to the doctor, we get text message reminders about this thing that we may not really want to do. However, in the criminal justice, we- this doesn't really exist. So a first order question was, will reminders reduce failures to appear in court? I think, you know, to start out with, you could see how this would help if people were forgetting. But on the other hand, you know, I think the consequences seem so big of not showing up to court, to have an arrest warrant, that it's not trivial whether it's actually the case that people are forgetting and therefore an intervention that helps you remember would be effective.

 

Aurelie [00:20:06] The second level question that we were interested in was figuring out what kinds of text message was most effective. So we tested two main approaches. One was to be very explicit and to remind people that if they did not show up to court, they would get an arrest warrant. And so the kinds of messages that they would say would say something like "Helpful reminder, go to court on Monday, June 3rd at 9:30. We'll text you to help remember. Show up to avoid an arrest warrant." So it really wasn't a mystery, it was just written out there. And then conversely, we had some simple reminder plan making messages which started the same way, "Helpful reminder, go to court on Monday, June 3rd at 9:30a.m—" But instead of bringing up the warrant, we'd say something like— "Mark the date on your calendar and set an alarm on your phone." So defendants would all get three text messages a week before, two days before- three days before sorry, and the day before their court appointment. But either they would be reminded that they would get an arrest warrant if they didn't show up to court or they- this was not brought up at all. And we were interested in figuring out to what extent people were just forgetting or not getting accurate information or not recalling accurate information about a forthcoming- about the fact that they would get an arrest warrant if they did not show up to court.

 

Jennifer [00:21:40] And in order to measure the causal effects at each of these interventions, the new form and the text messages, you had to implement them in a way that gave you both a treatment group and a control group. And the way you do this is a bit different across the two interventions. So how were the new forms distributed and how did this allow you to measure the causal effects of the change?

 

Aurelie [00:22:01] Yes, so both interventions, kind of the gold standard for and to have a causal estimate of whether these interventions would reduce failures to appear, would be to do a randomized controlled trial where we flip a coin to determine whether you get a text message or not or whether you get a new form or not. And you know the kind of information that you're given. This was actually possible to do for the text messages. And I'll go over this in a little bit. But it was not possible to do for the new form intervention. So concretely, what happens is these are tickets that are given out by NYPD officers, and so they walk around with a pad of summons forms. And the idea that they would, you know, randomly give some defendants an old form or a new form or that some officers would be randomized to get old forms and new forms, this was just not possible to do in the field. The logistical constraints would be much too complicated.

 

Aurelie [00:23:06] And so instead, we came up with another strategy to evaluate the effect of these forms. So each police officer, each NYPD police officer has to walk around with a lot of stuff and they could only walk with one pad of forms. Right. So they could only have a pad of the old forms or of the new forms. And so each form has a serial number and we can tell when the serial numbers reflect old forms or new forms. And so what we can do is figure out for each individual police officer, when was the first new form that they gave and when was the last old form that they gave? At some point they- when they ran out of old forms or at one point NYPD decided to just retire the old forms and all officers would start using new forms. And so each officer would change at a different date when they started to use the new forms. And so what we do is compare summons recipients who happen to be either among the last who got an old form for a given officer who gave them the form versus one of the first to get the new forms. And so by focusing on this narrow time window, just before and just after a police officer changed the kind of summons forms that they were working- that they were giving out to summons recipients, we're able to get the causal effect of summonses on court compliance.

 

Jennifer [00:24:40] Right, and so the intuition here is that it's essentially random that you happen to get the old form versus the new form because you just- you got the summons the day before, the hour before, something like that. It wasn't a situation where, like, the police officer was deciding to give you the simpler form because you seemed like you paid less attention to what was going on or something like that.

 

Aurelie [00:25:02] Yeah. You pay less attention or you seem like a nice guy. Let me give you the new form. Right. And the nice thing is there's a lot of seasonality in crime, in offending, also in court compliance, actually. And so the nice thing is, since there was a staggered rollout, each police officer, you know, there were a few a few months during which police officers were changing forms, we're able to control for this seasonal time trends.

 

Jennifer [00:25:26] That's great, yes. So some officers had some- one form and some officers had the other. Great. And then you said the text messages are in RCT, so so that's a little bit simpler. But tell us tell us how the text messages were implemented and how that allowed you to measure the causal effects there.

 

Aurelie [00:25:45] Yeah. So the text messages were sent to defendants who had provided a phone number on their form and the text messages were sent directly by the Office of Court Administration. So it wasn't sent by us. It was sent directly by the Office of Court Administrations. There's a couple of reasons why this is important. First, it's an official thing, like it's not coming from a team of researchers. It's actually coming from the courts themselves. And second, the nice thing of working from the start with our amazing partners at the Office of Court Administration is that the idea was, if this works, we want to be able to continue sending what text messages and especially the most effective text messages, and so to be in a situation where we're able to continue this work. And so they built the infrastructure from the start of the study with the idea that if they already built the infrastructure, they could continue doing this even when the research was done. And so the nice thing is that this is all on the back end. We could just flip a coin to determine whether a defendant would get no message at all, whether they would get warrants messages—where we brought up the fact that they may get an arrest warrant if they didn't show up to court—or the simple reminder plan making messages. And the idea was in order to learn whether these work or whether they're a waste of people's time, of court resources, we were going to conduct this experiment with the idea that this would be a short run thing and that if we found a winner, then all defendants would be able to benefit from these text messages.

 

Jennifer [00:27:28] And as you mentioned, you were working directly with New York City to run these experiments, which presumably made data access easier. So what data were you able to get to analyze the effects of these two interventions?

 

Aurelie [00:27:40] So we got data from the Office of Court Administration and we were using administrative data, which told us for each defendant why they had gotten a ticket and whether they showed up to court or not, as well as additional information such as there- some some social demographic information, though not a lot, and the exact date at which they had received their ticket, whether they had gotten summonses in the past, whether they had failed to appear to court in the past and the place where they received their ticket.

 

Jennifer [00:28:17] OK, so let's talk about the results. What do you find is the effect of the new summons form on the likelihood of showing up to court?

 

Aurelie [00:28:25] So the both- we find that both interventions actually work really well to reduce failures to appear. So let me start with the summons form. We find that getting a new summons reduces likelihood of failing to appear in court by 13 percent. So this is a 6 percentage point decrease in rates of failures to appear. And I think to me, this was the most surprising result from this study. So I'll- before I contrast both of these results together, let me also tell you what our results are for text messages. We find that text messages overall reduce failures to appear as well. And actually the magnitude is even bigger than for the new forms. So we find that getting any text message reduces failures to appear by 20 percent and that this is driven both by messages which mentioned warrants. This is where we find the biggest effect. We find a 25 percent reduction in failures to appear for defendants who got a message telling them, you know, if you don't show up to court, you'll get an arrest warrant. But we also find a 16 percent decrease in failures to appear for those who got the simple reminders that did not mention the consequences of not showing up to court. So let's take all of these results together.

 

Aurelie [00:29:56] First, the fact that the warrant messages are the most effective messages shows that actually I started by saying that deterrence is one of the guiding principles that we have as we as we develop our crime reduction policies. And this difference between the warrants messages versus the simple plan making reminder messages suggests that actually reminding people of consequences of not showing up to court, bringing up this potential punishment is effective. So this is this speaks in favor of deterrence. However, the fact that the simple plan making messages also work suggests that this is not the only thing that is driving compliance. It is not just because people are afraid of the consequences of their actions, that they behave in a certain way. Actually, some people seem to just have been forgetting the fact that they had a court date coming up.

 

Aurelie [00:31:00] And then I started presenting my results by saying that the new form results were most surprising to me. And the reason for that is that on the old form, again, all information was already there. It was- yes, it was more complicated to find, but it was all there. And here, all we did was move things around. And that also was effective in reducing failures to appear, suggesting that people just weren't taking the extra 30 seconds to one minute that it took to figure out what they were supposed to do to be in compliance, suggesting that inattention may have been playing a role even at the get go when they were getting this information. So that it was really a striking result for me.

 

Jennifer [00:31:44] Yeah, I agree. The effects here are just huge across the board. And then you also had these post court date text messages for people who missed their appointments. What were the effects of those?

 

Aurelie [00:31:55] Yeah, and I wanted to clarify one thing. This is a really interesting project because it was at the intersection of big research questions. I told you, my research agenda is trying to figure out whether in the criminal justice, people are also making mistakes as we- as past literature had shown that people have been making mistakes in other domains as well. Our- we preregistered our hypotheses that we were testing to write the research paper and we wanted to limit the number of tests that we were running to make sure that we wouldn't have any false positives. However, our partners, our policy partners had a different set of questions that they were interested in. So we wanted to help them answer questions that may not be of the same research relevance, but that would be extremely important for them to design the kinds of policies they were interested about—questions about what is the right number of text message that we that we wish- that we should send. Another question was whether robo calls- whether there was a difference between robo calls and text messages. And we really, for the research purpose, focused on text messages.

 

Aurelie [00:33:15] One of the questions they were interested in was whether text messages after one's court date telling people about the the their- telling them that they had missed a court date and either informing them of the consequences of not having shown up to court or telling them that most people had actually shown up to court, using kind of a social norms messaging—which again, has been effective in other domains as well—was also helpful. So that's something that we study in a policy memo that we wrote and that we don't include in our in our in our research paper. And we find that these text messages actually help increase the effects. There's kind of a plateau in how much you can help people comply. But still, these additional messages seem to be helpful. If you just compare, if a person were to get a text message before their court date or a single text message after they had already missed their court date, the messages before one's court date are more effective than a single message after one's court date. But so- but overall, that digital message after one's court date may help as well.

 

Jennifer [00:34:33] So what you just said, about there might be a limit to how much you could help people comply, makes me wonder if they're- if it's possible for you to tell from the data if there's certain kinds of people who- for whom these kinds of reminders or nudges might work. Like, are there any differences across these different interventions in terms of, you know, people who are given summonses for certain crimes respond to this, but not other crimes, or it's young people but not old people or something like that?

 

Aurelie [00:34:59] You know, that's something that we haven't really looked at yet. It may be something that we look at in a future project.

 

Jennifer [00:35:09] Yeah. OK, so another aspect of this that that you guys are very upfront about in the paper is that only a subset of the people who got a summons gave the police a cell phone number. So I think the number was around 10 percent of the total number of summonses that were given out. So talk a little bit about how representative this group is of the broader population. And I guess the kind of the question in the background here is how likely is it that the police might be able to get cell phone numbers from more people now that you've shown this is a really effective intervention and we might want to scale it up?

 

Aurelie [00:35:45] That's a great question. Yeah, so 10 percent of people provided phone numbers and they do differ from people who did not provide phone numbers. So let me give you a few dimensions along which they differ. First, they- the people who provided a phone number tend to be younger. There are a few differences in terms of what kinds of offenses they got their ticket for. So they're a little bit less likely to have gotten a ticket for alcohol, a little more likely to have gotten a ticket for cannabis consumption. And there are some variations across places. In Manhattan, they were more likely to provide cell phones compared to a borough like Queens. However, they were similar in some dimensions as well. For example, they were just about as likely to have gotten a summons in the past or to have failed to appear in court in the past. And another interesting comparison is to look at failure to rate for people who did not provide a phone number versus people who provided a phone number but were randomized to the control group and so did not get any text messages. So 41 percent of people who did not provide a phone number failed to appear in court. By comparison, those who provided a phone number but didn't get a text message, 38 percent of these didn't show up to court. So, yes, there is a difference, this is a 3 percentage point difference. However, 38 percent is still a big number. So it's not like people who are super compliers were giving phone numbers.

 

Aurelie [00:37:23] One challenge with this study to your question about scaling is that police officers during the study couldn't really tell defendants, give me your phone number and you will for sure get a text message reminder. They had to say some language like you may get a text message reminder because again, while the study was underway, some defendants were randomized into getting no text message at all. And so this may have been one barrier towards collecting more phone numbers. And I think, you know, that while in our particular context we worked- police officers were collecting phone numbers and also we worked with summonses. I think that based on those results, it doesn't have to be a police officer who collects this phone number. You could think that somebody else, like a defendant's attorney, could be sending these kinds of reminders. So there are other people in the criminal justice who could be collecting these phone numbers if a defendant is a little bit shy or not- or wary about giving their phone number to a police officer, maybe there are other people who could be collecting these phone numbers and sending these reminders.

 

Jennifer [00:38:38] Yeah that last point is really interesting. I've heard some folks more on the kind of defense attorney side worry about these sorts of text message interventions as basically creating a paper trail that could get their clients into trouble if the text message's coming from the court rather than their lawyer. Is that something you thought about here?

 

Aurelie [00:38:56] Absolutely. And that's another reason why I think that the form redesign intervention is is really interesting. Actually, we conducted some estimations to measure how many warrants were avoided. And so I told you that the most effective text message reduces failure to appear by about a quarter and that the new form reduces failure to appear by 13 percent. But everybody got a new form. Right. And so actually, when we translate that into warrants avoided, we think that more arrest warrants were avoided thanks to the new form than thanks to the text messages. And the nice thing is that these concerns of perhaps it's not ideal for defendants to be providing their phone numbers don't apply in this case. They're getting a piece of paper. Everybody in the criminal justice, we have lots of paper. And I think that part of these results are saying that this is a touchpoint that we have. Let's try to design these forms in ways that will provide the relevant information in a way that accounts for inattention that is happening in a way that is simple, so to increase compliance, kind of sidetracking these issues that you bring up.

 

Jennifer [00:40:10] OK, the last thing you do in—I think it was the paper, not the policy brief, maybe both—was that you conducted some online experiments to try to understand what drives the public's perception of why people miss their court dates and why the public isn't more supportive of applying behavioral science in this context, rather than just increasing punishments for FTAs as you discussed in the beginning. So talk about what you find there.

 

Aurelie [00:40:34] So this work uses online experiments using Amazon, Mechanical Turk. And so just to be clear about this, we're not running these studies with criminal justice actors. Right. The idea that we're trying to get at here is how do lay people think about what is driving failures in the criminal justice context as opposed to failures in other contexts. So what we did is that we presented people with different vignettes describing a person who had to do something unpleasant and then did not go through with this something unpleasant that they had to do. So, for example, this could be a person has to show up to court in 30 days. They don't want to show up to court in 30 days, and then they do not show up to court. Why do you think this happens? Then we contrast this to a very similar question, which would say a person has to pay a bill in 30 days. They don't want to pay this bill in 30 days. Why do you think this person did not comply? So all that we're changing here is the type of action that a person did not do.

 

Aurelie [00:41:47] And then we asked a few questions. We- first we ask them whether they thought that this failure, this noncompliance was due to an intentional action or just simply forgetting. And then second, we asked them whether they were supportive of nudges—so in this case, we limit ourselves to reminders—whether they thought that reminders would be a good idea in terms of a policy to curb that behavior. And so we compare people's attitudes towards courts, towards forgetting to show up to court, to other domains. So not paying a bill, in the education domain not signing up for classes, not showing up to a doctor's appointment, or not taking care of a reduction- of emissions reductions intervention.

 

Aurelie [00:42:39] And what we find is that people are much more likely to think that failing in the criminal justice domain is due to intention and that it is not due to forgetting. And they're less likely to support nudges in the criminal justice context compared to those other failures. So people somehow are thinking differently of the court context than they are for other reasons that people are not following through on things that they are supposed to be doing. And so what's interesting to us here is that perhaps the kinds of technologies that I talked about—sending text messages and even more so putting some work into designing a form or designing information- presenting information in a way that is clear to defendants—these are not difficult to do. These are also not costly to do. The- one of the reasons why these things may not be more frequent is that people, when they think about failures in the criminal justice context, may not be thinking about the mistaken dimension, the fact that people may simply be not paying attention or forgetting about what they're supposed to do in the criminal justice context and the way that they think that this could be driving other behaviors.

 

Jennifer [00:44:01] So they might be opposed to this or not supportive of it because they just don't think it will work. Is that that the takeaway?

 

Aurelie [00:44:07] So I think a bigger takeaway is they may be- they may not even think about developing these kinds of policies. Then when you bring it up and you're like, look, this may work. I think the way I'm thinking about this isn't so much active, you know, opposition to these policies, but more not thinking about this realm of drivers to people's behaviors and not thinking about policies that could address this part of the problem.

 

Jennifer [00:44:35] So this paper is pretty new, but you've been working on it for a little while. So what else have we learned since you first ran these experiments about either what drives FTAs or about the effectiveness of nudges in the criminal justice space?

 

Aurelie [00:44:49] So this is a really interesting moment to be working on pretrial policy. I think that one of the- this is one of the most active areas of research. And since we have been working on this work, other studies have looked at both- about- have looked both at nudges for failure to appear in court. I'll speak a little bit about not just more generally in the criminal justice, but also about drivers to compliance. So one active question is whether other tools that we have to increase court compliance, what are these effects on on on people's lives? So this workbook takes the shape of several papers looking at the effect of pretrial detention on longer on longer outcomes. And then another project that I've been working on with Megan Stevenson is looking, you know, by contrast to these really largely effective policies to reduce failures to appear in court. How does monetary bail work? Does monetary bill help with court compliance? And the short answer is, at least in the context that we're looking at, we do not find that monetary bail and pretrial supervision are reducing failures to appear in court. So this is- now this is a very active policy space where I think people are starting to realize both the disparities created by the use of monetary bail and by pretrial detention. And so trying to figure out whether we can move away from these while also not jeopardizing public safety and not increasing failures to appear. And I mentioned that this is high not just for summonses, but also for other kinds of offenses.

 

Aurelie [00:46:43] And there's been some really interesting work in testing other kinds of reminders to reduce failures to appear in court. In particular, Natalia Emanuel and Helen Ho, who are graduate students at Harvard, have done a large study of behavioral biases and legal compliance. And their study differs from ours in that they're trying to test whether people are lacking kind of the know how to navigate the court system. So that's really interesting. They're not just looking-. So we were interested in the inattention mechanism and they were interested in more the the knowledge of the court system and difficulty navigating the court system dimension. Another innovation that their that their great work brings is that they're able to look at longer term effects of avoiding these failures to appear and in particular in terms of future arrests. So that's really exciting work happening. So both in the policy sphere and the kinds of discussions we have around failure to appear and active research, there's a lot going on these days.

 

Jennifer [00:47:51] That's great. Yeah, the paper that you mentioned that you're working on with Megan Stevenson, she was on the show a few episodes back talking about that paper, so we will put a link to that episode in the show notes as well, if people are interested in hearing more about that. So putting it all together, the results of this study and the other studies we've talked about, what are the policy implications of this work? What have you told New York City and what would you tell policymakers in other places?

 

Aurelie [00:48:15] So I mentioned that this work was really done in close collaborations with the NYPD, the Office of Court Administration and the Mayor's Office of Criminal Justice. And the nice thing with working directly with policymakers, I should say that this work I did when I was part of the University of Chicago Crime Lab and these- this joint partnership resulted in being able to scale this study right away in New York City. So now everybody who is a summons recipient gets this new form. And at the end of our study, everybody is getting- who- every defendant who provide their phone number is currently getting these text messages. So these are really kind of the first order, immediate policy implications that our work has had.

 

Aurelie [00:49:08] I think that moving forward, the- we published very early on—as soon as we had our initial results—we published a policy memo that we made public. And the reason why we wanted to make this public early on is because we wanted people to be able to concretely see what were the text messages that we sent, how was the form redesigned. So all of this is online so that other jurisdictions can see what we did, how it worked and replicate it without having to reinvent the wheel. And that's something that was, I think, really important to us. So that policymakers- I mentioned that the that these interventions are not expensive and they're pretty easy to do. And we wanted to reduce the extra barrier of having to invent text messages and having to invent how to redesign a form, while also recognizing that there are local specificities in terms of what exactly the consequences are, how people would- and what people want include what they can or cannot include in in their forms and their text messages. So I think thinking in the realm of failures to appear in court, we hope that we provided tools for policymakers who want to replicate this.

 

Aurelie [00:50:25] I think that more generally, you know, why did we start working with summonses, summonsable offenses? I think this was- when we started working on this, one of the first attempts to test whether text messages were effective in the criminal justice sphere. And so we wanted to start with a low stakes kind of offense in order to make sure that if something went wrong, it wasn't going wrong for in the context that could have led to really negative downsides. Based on this work, I think that policymakers may be more comfortable testing this in higher stakes situations. Right. And so in particular, I mentioned that some jurisdictions are moving away from monetary bail and so they can't use money as a collateral for people to show up to court. Some jurisdictions are trying to reduce their pretrial population, but they're also cognizant that they don't want to increase their failure to appear rate. And so I think that our results could suggest that when judges are making decisions about who to release from jail, pretrial or not, I think we think more of this as a static thing. A person is high risk or low risk, again, either implicitly or explicitly through risk scores, our results suggest that actually this is not static and this can be changed. Right. And so imagine a judge is worried about a person not showing up to court. What we're showing is that there's a tool which could reduce—there are two tools—that can significantly reduce a person's failure to appear risk. And so perhaps that they would be more likely to be willing to reduce a person from jail pretrial if they had assurance that these mechanisms could help with greater compliance.

 

Aurelie [00:52:18] And then moving away from the issue of court compliance, which again, I do not want to minimize, since it is one of the big drivers of pretrial detention, I think that there are other ways in which- other behaviors for which people may be making mistakes and that our policies are not currently thinking of as making mistakes. Most directly, one could think about probation violations or parole violations as places in which remedying inattention could also help with these policy outcomes. Just to give you an example, in 2016, I think that about a third, a little under a third, of state and federal prisoners were detained for some form of violation of their parole or probation conditions. So this, again, is a place where if inattention is also driving these kinds of behaviors, this could be a tool that we want to leverage as well.

 

Aurelie [00:53:14] And then more generally, circling back to my general research agenda around behavioral science and crime policy, here this project is really focusing on one way in which people can make mistakes, which is they're not paying attention to the consequences of their actions. But I think that there can be other crimes as well where people may not be acting as deliberately as our model suggests. And I think one paper that illustrates this in particular, one of my coauthors Anuj Shah worked on it, "Thinking, Fast and Slow?" looking at cognitive behavioral therapy and how this can reduce involvement in violent crimes. This is one example of recognizing that some decisions may be automatic, can help us develop some policy tools which would address this automaticity and reduce other kinds of crimes as well as long as we start to put our fingers on other other drivers for crime and in particular, mistaken slash non deliberate decisions that can be leading to crime and offending that can help us broaden the kinds of policies that we consider.

 

Jennifer [00:54:26] You've already you've already alluded to this a little bit, but what is the research frontier here? What are the big open questions in this space that you and others will be thinking about in the years ahead?

 

Aurelie [00:54:37] So I think, again, like just working on developing other other interventions that both address these barriers that I mentioned. I think we're not done figuring on places in which inattention can be playing a role in which automaticity maybe be playing a role among and driving some offending. So trying to understand what shape it takes could help us develop policies to reduce crime other than just the usual deterrent deterrence policies. And then more broadly, thinking about mistakes that actors in the criminal justice—judges, jurors, prosecutors, public defenders, police officers—may be may be making or things that they may not be taking into consideration could help improve their decision making process as well.

 

Jennifer [00:55:31] My guest today is been Aurelie Ouss from the University of Pennsylvania. Aurelie, thanks so much for talking with me.

 

Aurelie [00:55:35] Thank you.

 

Jennifer [00:55:42] You can find links to all the research we discussed today on our website, probablecausation.com. You can also subscribe to the show there or wherever you get your podcasts to make sure you don't miss a single episode. Big thanks to Emergent Ventures for supporting the show. And thanks also to our Patreon subscribers. This show is listener supported, so if you enjoy the podcast, then please consider contributing via Patreon. You can find a link on our website. Our sound engineer is Caroline Hockenbury with production assistance from Elizabeth Pancotti. Our music is by Werner and our logo is designed by Carrie Throckmorton. Thanks for listening and I'll talk to you in two weeks.