Probable Causation

View Original

Episode 5: Kevin Schnepel

Jennifer Doleac

Kevin Schnepel

Kevin Schnepel is an Assistant Professor of Economics at Simon Fraser University.

Date: June 11, 2019

A transcript of this episode is available here.

See this content in the original post

Episode Details:

In this episode, we discuss Professor Schnepel’s work on diversion programs:

“Diversion in the Criminal Justice System” by Michael Mueller-Smith and Kevin T. Schnepel

OTHER RESEARCH WE DISCUSS IN THIS EPISODE:

Transcript of this episode:

 

Jennifer [00:00:07] Hello and welcome to Probable Causation, a show about law, economics and crime. I'm your host, Jennifer Doleac of Texas A&M University, where I'm an Economics Professor and the Director of the Justice Tech Lab.

 

Jennifer [00:00:18] My guest this week is Kevin Schnepel. Kevin is an Assistant Professor of Economics at Simon Fraser University. Kevin, welcome to the show.

 

Kevin [00:00:26] Thanks, Jen. It's great to be here.

 

Jennifer [00:00:28] So we're going to talk today about your study with Mike Mueller-Smith on diversion from incarceration. But let's start with a bit of background on you. Could you tell us about your research expertise and how you became interested in this topic?

 

Kevin [00:00:42] Sure. So I'm a assistant professor here in Simon Fraser University. When I was doing my PhD in University of California, Santa Barbara, I was focusing on a project that was evaluating the causal determinants of recidivism in terms of labor market opportunities. And so I became- that kind of opened the door to a lot of other criminal justice focused projects. And Mike and I met as graduate students and we started to collaborate on a separate project looking at how bans from public assistance programs for prior drug felony convictions were impacting offenders. That project's been put on hold. But through the course of that project and looking at the rates of felony convictions over time in the context of our setting, which is going to be in Harris County, Texas, we actually noticed two very large shifts in felony convictions, the number of felony convictions or rates, one in 1994, one in 2007. And then started investigating what was going on with those changes and realized we had a really great opportunity to evaluate or work on a project that was evaluating the causal impact of a diversion program that was kind of behind these big shifts we saw on felony convictions in Harris County.

 

Jennifer [00:02:08] Great. Yeah, so your study with Mike is titled "Diversion in the Criminal Justice System." Very straightforward. So in it, you're considering the effects of diverting low level, nonviolent felony offenders from incarceration, so that means putting them on probation instead or giving them a deferred adjudication, and we'll talk more in a moment about what all that means and the particular policy experiments that you're studying here. But tell us a little bit more about the concept of diversion in general and when it's used in the criminal justice system.

 

Kevin [00:02:35] Sure. And one thing that I want to make clear up front, and as you've said diversion from incarceration, that's part of kind of how we think about, you know, potential diversion. That's not really what our paper is about. It's really about diversion away from a felony conviction. And I'll talk in a lot more detail about that. And I think that's what makes this study so unique and important, given what we know and what's been out there. But just to take a step back and talk about diversion. And this is, you know, helps to highlight the fact that diversion can mean a lot of things. So it could mean a program. It could be, you know, someone, a juvenile that's arrested for a drug offense being sent to a treatment program following an arrest rather than getting charged and convicted in a court. There's a lot of different types of diversion programs out there at different levels of the criminal justice system. And so it can mean a lot of things in our context.

 

Kevin [00:03:35] What diversion is is going to be a second chance in the sense that individuals who are charged with a felony conviction are going to have an opportunity to clear that felony conviction conditional on successful completion of a probationary supervision period. So it's it's going to be you know, it's in the in the context we're looking at in Texas, it's known as deferred adjudication of guilt. Sometimes you hear people describe this as withholding adjudication. They're going to- withholding judgment until the end of some sort of super supervisory period. And so that's the type of diversion we're going to be talking about in this paper. But, you know, there's a lot of other types of diversion out there. This is probably one of the more common types out there, given, you know, a lot of I think a lot of the reason we've seen these programs grow is both in, you know, people recognizing the potential benefits to a second chance and the potential collateral consequences of a conviction. But it's also been growing quite rapidly, I think, because of budgetary reasons and concerns. And just trying to manage a rapidly expanding criminal caseload in these county court systems has, you know, caused some I think changes in the way in which defendants are being treated and the opportunity to participate in some of these programs for defendants. So I think, you know, hopefully that- it might make it I guess, less clear what diversion is. But I want to be clear that diversion is- can mean a lot of different things and what type of diversion we're focusing on in this paper.

 

Jennifer [00:05:24] Yeah, that's super helpful. And good clarification that in this case, actually, you're not actually you're not seeing much of an impact on incarceration, which I think is very interesting and we could talk more about. So I think you're right that this diversion in general is a really hot topic right now in criminal justice reform circles. But, you know, one of the reasons I am really- I love this paper is that it is part of a relatively small literature, I think. So what had we previously known about the effects of diversion before you and Mike started working on this? Had there been much previous work on this topic?

 

Kevin [00:05:54] So there's there's emerging work in the economics literature - as I'm sure you're aware of, and some of your listeners - on the diversion away from incarceration and, you know, having the impact of incarceration versus not incarceration. From our perspective, there is a lot less known and a lot less work being done on diversion kind of before that stage. And so whether or not, you know, giving someone a second chance to avoid a felony conviction is- how that might impact both their behavior and how they're treated later on in the criminal justice system as well as the labor market. There is a lot of work out there that, you know, some great work that's comparing, you know, they were reoffending rates for individuals who participated in a diversion program versus those that didn't. And, you know, attributing- finding large differences in reoffending rates. But the concern, and I'm sure we'll talk about this a lot more later in the podcast, but the concern there would be there is a lot of other admitted factors that might be affecting the behavior and outcomes for those individuals that is not due to whether or not they're diverted at that stage. And so there is you know, there is a paper in- a great paper in 2007 in the criminology literature by Ted Chiricos. Hopefully I'm saying that right. And coauthors where they're comparing- they have a very large dataset in Florida of over 90,000 defendants and they're comparing outcomes and controlling for a lot of factors and trying to compare outcomes for those who had, you know, a judgment which mean they got a conviction and probation versus those who had the judgment withheld. And this is going to be similar in our setting where we're going to have these deferred adjudication or judgment withheld for a set of defendants. And they find large, you know, a large difference in terms of lower reoffending rates for those who had that judgment withheld. But but in that study, I think there's still some concerns that exists with other potential omitted factors that might be correlated with that treatment.

 

Kevin [00:08:15] One, you know, literature that I think is very important to mention here would be the work started by and done by Devah Pager and many coauthors over the years looking at the impact- so using, you know, clever design and sending applicants job applicants to interviews or having individuals apply for jobs and experimentally varying whether or not they had prior criminal histories and whether they had prior convictions or prior incarceration experiences and, you know, found convincing evidence that - and not surprising, I think, to a lot of people - that those prior criminal histories were associated with much lower callback rates and success in the labor market and the job search efforts of individuals that, you know, happen to have those records versus not and varying that visibility in whether or not individuals, the fictitious applicants, had those types of marks on their on their background or previous history. So I think that's a pretty important literature that helps to motivate this question that we're getting at and what we're- you know, I think what one of the things we're able to say something about is what is the impact- the causal impact of having a conviction versus not a conviction, having already committed or being caught for a specific type of crime? And in doing it more and trying to tease out the causal impact by doing it in a real setting where, you know, a defendant is treated one way or the other. And then we can follow- for reasons uncorrelated with a lot of the omitted factors that we might be concerned with- and then we can follow these individuals over a very long period of time. And so I think, you know, the the audit study literature by Devah Pager's is very important.

 

Kevin [00:10:06] There's some recent work, I think, and some emerging work, and we'll talk about this later I'm sure too, with with the frontier and what's what's being done and what's going to be done, I think, in the next few years. But some researchers are now starting to look at the impact of expungement programs and record clearing programs. And so whether or not someone with a criminal history, if they get their record cleared or expunged, how that's impacting their success and labor markets and reoffending behavior. And so I think that's going to be another important literature. There's a recent paper by Jeff Selbin and Justin McCrary and Josh Epstein published in 2017, where they're they don't have, you know, random assignment to this program, but they're trying to control for omitted factors and a selection bias by comparing participants in a program that helps individuals clear their criminal records and seeing what's happening toward to their employment and earnings following that expungement. And they do find, you know, the improvements in labor market outcomes. But there are some important caveats, especially the fact that, you know, it's not randomly assigned in this setting. And it's also the case that individuals who participate in these programs are doing so at a time with depressed kind of labor market outcomes. So those improvements are kind of relative to a period of less activity for those individuals, I guess. So that's there's you know, and there's I think those would be the major areas of of work that's been done where we're starting to learn more about the impact of these different types of treatment and specifically the impact or the expected impact of a diversion program where defendants are being either put on probation with a conviction or put on probation with an opportunity to clear their record from that conviction conditional on the successful completion of that probation, which is what we're going to be doing in our paper.

 

Jennifer [00:12:14] Yeah, it's interesting to think about all the different approaches to measuring programs that are related to this, and of course, the real key here is finding an actual experiment or natural experiment that isolates that one margin. So, like, at what point could we intervene to hope to send someone off onto a different trajectory? Because we can't just sort of-

 

Kevin [00:12:33] Right. Yeah.

 

Jennifer [00:12:34] Can't just like magically turn someone who is committing crime into someone who didn't, which is sort of the the more direct correlational comparison you might make. But it is yeah, it's it's interesting to think about all these these studies that take very different approaches and intervene at different points. And is that the main challenge to studying this- is finding the experiment that you need to kind of isolate this one factor? Is there is there more to it that makes this a really tough question to answer in your mind?

 

Kevin [00:13:01] Yeah, I think that is, you know, that is the main challenge, I think, is is finding that exogenous or that variation in, you know, whether or not someone's getting treated with a conviction versus deferral and what might be causing or whether the underlying determinants of that, the differential treatment. And if, you know, if we can isolate a situation where if we have the same defendant in one case versus the other, had two observe- equivalent defendants, are we able to randomize, you know, who is going to get a diversion and who is not? That would be obviously the ideal experiment to be able to uncover the causal impact of these interventions at this stage in someone's criminal career at that at that level of their involvement in the criminal justice system. But, you know, I think in this setting, these types of randomizing, you know, punishment or judgments or case dispositions is not it's likely not feasible. But so the challenge then is to try to discover or find a natural experiment, as you mentioned, where we do have that same experimental variation and some setting that's going to approximate a randomized controlled trial. And that's, you know, that's what we focus on here in this paper. And, you know, there's a lot of different ways where we can show and reassure our readers and our audience that, you know, it does look like treatment in this case in terms of getting a diversion, getting a court deferral or deferred adjudication as plausibly random, relative to a lot of these both observed and unobserved characteristics of these individuals.

 

Jennifer [00:14:57] Yeah, so let's talk about those experiments. So you have not one but two policy experiments in this paper. You're clearly overachievers here. So tell us about these two policy interventions that you're using to study diversion in this context.

 

Kevin [00:15:11] Yeah, and I think that having two, you know, has created some challenges in interesting ways, and that's it- but it also provides, you know, reassure- reassurance to us as researchers that, and I'll talk more about the results, but we have these two policy experiments on diversion in Harris County, and they go in different directions in terms of whether more or less defendants are being diverted. And having the opportunity to evaluate these two separate changes and seeing what the outcomes are and seeing the consistency in outcomes, I think helps provide us a lot of reassurance that what we're detecting or uncovering is some sort of important causal parameter and or causal mechanism by which diversion is impacting individuals here in Texas. And so to get, you know, to provide a little bit more detail, I'm gonna have to, you know, limit my scope- because you could talk about- these changes as as your listeners will hear- you know, kind of quirky in the sense that they're happening for different reasons. And there's a lot of details that we could discuss about these changes. But I'll try-

 

Jennifer [00:16:18] That's what makes them so fun.

 

Kevin [00:16:21] I don't want a whole podcast on the intr- in intricacies of Harris County situation and these two changes. So I'll try to be brief, but can refer the readers to our paper for a lot more detail. And also welcoming, you know, comments and feedback, especially if any of the listeners were involved in Texas' justice system or Harris County at this time and have some more insight into these changes would be would be very welcome. And so the first one we're evaluating is a change that happens in 1994. And so in Texas, there was an overhaul of the penal code in around this time and in particular, and what's what's most relevant for our study, is that there was a certain class of felony offenses and these were all I think, property and drug type felony offenses that were reclassified into state jail felony offenses. So this was a new type of felony offense that was created by- through this penal code reform. And what happened through this reclassification is that the deferral- so before this change was implemented for this class of offenses, prosecutors and judges in Harris County were using, or judges I should say or or the courts, were using these deferred adjudication judgments for about- for these relevant offenses about 70 percent of the time. And then following for these offenses, the deferral rates or the diversion rates went down to about - let me just confirm this - but I think it was around 35, 40, 40 percent of the time. So there was a very large decrease in the use of these diversion, these deferred adjudication or diversion agreements.

 

Kevin [00:18:17] And what happened was that the way in which the offenses were reclassified made made it such it made it much less attractive for the prosecutors and judges to use the deferral agreements because it limited their ability or it made it impossible for them or difficult for the these agreements to have any sort of overhanging incarceration, punishment associated with the violation of these deferral agreements. And so from the prosecutors perspective- and the reason we know a lot about this is there is there were some memos and there was a kind of a practice implementation of these law changes. And so it was anticipated that this would happen, that the prosecutors would feel the deferral changed from something that they thought could be effective to something that they thought would not be effective because it wouldn't have this overhanging punishment. And the reason it couldn't have the overhanging punishment is because the new class of offenses for these offenses, it was required that the first conviction have a probated incarceration sentence. And so what that meant for the new- you know, going into this new regime where these state jail felony offenses- if you were charged with a state jail felony and you got a deferred adjudication and you violated that deferral agreement, so you you committed another offense while on probation, then the only only option for the prosecutors was then to give you a conviction for the first deferral and then put you on another probation with a probated incarceration sentence. So it was almost as if they would have to be forced to kind of give someone a two probations where they would only want to give them one. And so that, you know, from a lot of digging and a lot of reading of what was going on in Harris County at that time for these types of offenses, that's the reason we isolated and discovered for this very large decrease in the use of the deferral adjud- to the deferred adjudication agreements for these drug and property offenders in Texas.

 

Kevin [00:20:29] And so that was the first change. So we have known that. And I hope some of the listeners go look at our paper because we have a lot of nice figures that show these changes very clearly. But if you're looking at a defendant coming through the Harris County courtrooms the week prior to September, so the change was effective September 1st, 1994, if they're coming through the week prior, they have, you know, twice- they're more than twice as likely to get one of these deferred adjudication judgments than the group coming through the courts the week after. So following September 1st, 1994, so about a 30 percentage point drop in deferrals in that case. And again, due to the penal code reform and how that limited the prosecutorial and and judge's ability to what they thought to give a kind of an effective deferral based on what would be triggered if on the violation of the deferral. So that was 1994. And that's, you know, a really big change, an interesting change for a lot of reasons. And for those individuals, we have the opportunity to track individuals over, you know, up until 2015. And so we can see the outcomes for a 20 year follow up period for that treatment, comparing individuals just on either side of that change and the rate of diversion.

 

Kevin [00:21:56] For 2007, the change was in the opposite direction, and it's not that dissimilar in magnitude where we see about, I think it's going from the - let me just refer to the figure here - but I think we're going from about 50, 45, 50 percent, around 50 percent of the caseload getting deferred adjudication agreements for a low risk felony caseload. And in 2007, we do have some violent offenses included in that category. But it would mostly be property and and drug offenses. And so going from about half of the defendants being diverted with these deferred adjudication agreements to around two thirds. So 50 percent to around 65 or so percent. And this change was a little bit more, I guess, obscure. I don't know if that's the right word, but we saw this in the administrative court record data for Harris County. And the change happens from, you know, the week before to the week after November 7th of 2007. And so we started to do a lot of digging in terms of what was going on at that time period in Harris County. And this was a change we didn't see statewide, like the first one, but specific to Harris County. And what had happened on November 7th, 2007 is that the Harris County voters went to the polls and voted on a variety of propositions and measures. But included in that was a jail expansion ballot initiative. And so there was a conversation about building a new jail in Harris County. And a lot of discussion leading up to this vote about, you know, there were some concerns about the location of the new jail and there was also some concerns and, you know, expressed sentiment about potential overreliance on, you know, that type of punishment or kind of more severe punishment for lower risk offenders or defendants. And so what we think happened, and this is- so what we observe happened is following this ballot initiative and what happened was that jail expansion ballot initiative was very narrowly defeated. I think it was 50.6 percent to 49.4. So it's a very small margin of victory and I think quite surprising- there was a lot of discussion leading up to it. But for this type of ballot to fail in a county in Texas, I think this was surprising to people. And we think it might have been also surprising to the county judges.

 

Kevin [00:24:49] And so what we see is the following- the week following this election was this very large increase in the use of deferred adjudication for lower risk defendants that, you know, might be more likely to be sent to the county jail or kind of going in and out of the county jail. And that wasn't necessarily, you know, a behavior that was the- it's- suppose the jail expansion ballot initiative had passed. Obviously, the new jail is not going to be built overnight. So this was, you know, somewhat of a knee jerk reaction would maybe be the best way to describe it. But something that we see very clearly in the data that there was a very discontinuous shift in the probability that drug and property and other low risk felony defendants were getting the deferral agreements or getting the diversion. And so what- and this is an area, I think, if any of your listeners out there have any insights or institutional knowledge as to what was going on in Harris County at that time or the discussions that might have been happening amongst the judges. We've tried to get in touch with some of the judges and prosecutors, but haven't had much success in having some, you know, in-depth conversations with them about this time period, which, you know, now is over 10 years ago.

 

Kevin [00:26:09] But that's, you know, to our best knowledge, that's what really drove this policy experiment. And our experiment, our natural experiment is just relying on this very discontinuous change in the use of deferrals that's completely unrelated to what the underlying characteristics of the defendants coming into the courts were at that time. And so that's you know, it's not it's not as important that we know exactly why these changes happened. I think for our paper, the most important thing is that these changes happened in nature- they happened for reasons that aren't going to be, you know, aren't driven by unobservable characteristics about the defendants. But, you know, of course, it's it's important for us and important for the paper to kind of describe these changes and what was going on. And so that's, you know, as I said at the beginning, this is, you know, there are two changes and they are two changes for very different reasons and they're going in opposite directions. And that's one thing that makes the paper a bit challenging. But it also makes the paper that more, I think, fun or maybe interesting from both the research design perspective, but also the institutional background of kind of the nitty gritty details of what was going on in in Harris County at the time.

 

Jennifer [00:27:31] Yeah. I mean, you know, for the purposes of isolating causal effects, we all say we'd love a randomized experiment. But these kinds of natural experiments are way more interesting. Digging into the history and figuring out the the kind of quirky things that happened that gave you these nice policy experiments are- it's a really fun part of the job, I think. So you've alluded to this a couple times, but just to kind of step through the policy experiment or the the the natural experiment piece of this a little bit more clearly. So. So in general, the empirical challenge here is that we can't just compare people who are diverted to people who aren't because there's probably a reason for that difference in punishments. And more specifically, judges or prosecutors actively choose which people to divert, will likely choose people who are at lower risk of reoffending. And so if we see in the data that those who are diverted offend less down the road, we can't tell if the fact that they were diverted had anything to do with it or if they were just lower risk to begin with. And so, in other words, diversion might be correlated with better outcomes, but it might not be causing those better outcomes. So here you have these beautiful policy changes in Texas at two very different moments in time for two very different reasons that provide natural experiments and allow you to get around this problem. So tell us exactly how you're using these policy changes to measure the causal effect of diversion itself.

 

Kevin [00:28:51] Sure. And so I think, you know, it's building up for the- I've described in terms of these institutional backgrounds and these changes and, you know, the and what we see very clearly is that if we have defendants coming into the Harris County courts, you know, the week prior versus the week after these changes that I've described, we see very large differences in the probability that any given defendant is getting diverted. And the the kind of and this is, you know, I've used a lot of different quasi experimental research designs in my research, and this is one in which we call it a, or the research community, would label it as a regression discontinuity design. And basically what's happening is around this threshold, so at this date of September 1st, 1994, the date, November 7th, 2007, we're approximating a randomized controlled trial. And I think actually getting pretty close to what you would expect from a randomized controlled trial, because the individuals who just happened to be, you know, in the 2007 case, the individuals that just happened to be lucky enough to be coming into the court the week following the failed jail expansion ballot initiative and whose probability of diversion was was thus increased are no different than those that came in the week prior. And so you could almost imagine taking everyone who was charged and with a certain type of offense in a given month and just randomly kind of shifting individuals to one week versus the next. And that's what this regression discontinuity design is really giving us. It is giving us this approx- approximately a randomized controlled trial based on the assumption that, you know, whatever is driving this discontinuity in treatment is also not associated with any sort of discontinuity in the types of individuals we're seeing before and after. And so it's a really powerful research design, I think. And one that can get very close to what we would expect from a randomized controlled trial, you know, with the caveat that this is happening at a specific time. And it's you know, it's it's it's measuring an effect for people that are kind of treated or for whom, you know, coming in before, in the 2007 case again, what our design is doing-

 

Kevin [00:31:32] And so where you have a regression discontinuity design, we can compare the probability of treatment if we're thinking of diversion as treatment on each side of the threshold. And then we could compare the future outcomes, reoffending and employment for individuals on each side of the threshold. That would give us an idea of the overall impact. So we're using where you show up in terms of whether you're on the one side of the discontinuity versus the other to predict whether you got diverted to then uncover the causal impact of diversion on future outcomes. And so this would be - I described the regression discontinuity design - this would be a specific type of regression discontinuity design that we call a fuzzy regression discontinuity design. And it's just scaling up- so the way we can think about it, is if suppose we saw that, you know, the change in 2007 from 50 percent of the caseload getting diverted to 65 percent and then we saw if we just looked at total outcomes, say we saw reoffending rates on each side of the threshold. You know, I guess 50 percent on one side and 30 percent on the other. That 20 percent difference in reoffending rates, if the only reason that we see a difference in reoffending is because of diversion, we would want to scale that up by the fact that we saw 15 percent of the caseload go from being diverted, not diverted to diverted. And so that's what this fuzzy regression discontinuity design is giving us. It's giving us that ability to kind of scale up those aggregate differences. We see and see how much of what is implied in terms of the actual treatment effect of diversion versus not diversion in those in those experiments.

 

Kevin [00:33:27] The nice thing the other nice thing about RD and then I'll stop talking about the boring details of this methodology is that I mentioned that it does approximate a randomized controlled trial. The really nice opportunity we have with regression discontinuity designs is we can- so in a randomized controlled trial, you would want to compare the observable characteristics of your treatment group and your control group to make sure that there's no differences across, you know, gender or other characteristics that you think might be important for outcomes. And if so, that gives you reassurance that you're also balanced on all of the things you don't observe. In the RD context, we can do something very similar where we can look at the average characteristics of the- so for these discontinuities in the Harris County courts, we can compare the, say, the prior misdemeanor number of misdemeanor convictions for the individuals coming into the courts the week prior to the change and the week after. We can compare demographic characteristics and we can show or demonstrate that there is no discontinuous changes in these underlying characteristics, which provides us a lot of reassurance that all of the unobserved things that you mentioned in terms of things that would be correlated with reoffending risk are not changing as well.

 

Kevin [00:34:50] And so this strategy and this and it's you know, it's interesting to describe that's probably the first time I've ever had to just describe our regression discontinuity without pictures. And so it does present a little bit of a challenge because it is very visual and it's something in terms of the identification assumptions and assumptions needed to, we need to satisfy to then attribute a causal interpretation to these differences and outcomes are things that you can assess visually as well to show that there's no break. So of other observers of at least the things that we can observe. And so it's you know, it's a I think it's it's the appropriate methodology given these discontinuous shifts we see in diversion in Texas to then try to measure the the causal impact of diversion.

 

Jennifer [00:35:46] You're right. This is such a visual design and I encourage listeners to go look at the graphs because they're beautiful. Yeah, so basically the idea here is that you see this sudden jump or sudden fall in the likelihood of being diverted. And if we think that matters, then we should also see a sudden jump or a sudden fall in those outcome measures. And so but as you just described, you have- not everyone is affected by the policy change. And so the key to this kind of design is it it gives us an estimate of what that treatment effect is, like how diversion matters for the people who are actually affected by the policy. So you have some people who were always going to get a deferred adjudication, were always gonna get diverted, you have other people who were never going to get diverted. This design does not tell us anything about the effect of diversion on those subsets. Right? It just tells us about the population that's that's affected. And often that's not you know, that's not just like a super representative group. So in this context, what's your sense of how the people who are affected by this policy, by these two policy changes, how they are different from kind of the average first time or more low risk offender?

 

Kevin [00:36:57] Right. Yeah, and that's and I think this is a great question and an important point and something I think maybe we can do a little bit more of in our paper as we move forward with it. But one thing that I know that we were a little bit surprised about is the- so one scenario you could imagine with this change in the use of diversion in 2007, say, is that next week, following the failed jail expansion ballot, that judges come in and say, OK, we're going to take all of the defendants that we think are at the lowest risk of reoffending and just start diverting that group. Certainly, they are selecting based on types of offenses. And it's clear that the change is specific to offenders, both first time offenders, so it's much larger for individuals coming in for the first time, but also those specific types of offenses. But across within those types of offenses and those first time offenders, what we've done is we've- so we can take all of the things we observe about these defendants and their prior misdemeanor conviction history, as well as demographic characteristics, you know what courts they're coming through, and we can use that information to kind of have a very rough prediction of a risk of reoffending. So we can use that information and you know whether or not we see them reoffending, as you know, as a way to to calculate someone's predicted risk of reoffending based on these observable characteristics. And so what we do with that is then we can look at the- if we look at the distribution across the defendants in terms of their predicted risk of reoffending. And we look at for whom across that distribution benefited from this policy change, so this increase in the use of diversion, we were quite surprised to see a fairly even or, you know, not huge differences in terms of which of these defendants- so it wasn't that they were taking all of the defendants with the lowest risk of recidivism and they were diverting those. It seemed seemed to be an increase in diversion that was experienced by individuals kind of across the entire distribution of the predicted risk of reoffending, which which, again, was a little bit surprising from our perspective.

 

Kevin [00:39:28] There are some differences in terms of and we have some nice pictures of this in our paper as well. But there you know, there is it's slightly lower kind of increase in diversion for individuals kind of in that top half of the predicted risk of recidivism in the 2007 case. So they are- it does seem like they are selecting people- are more likely to divert people kind of towards the bottom half. But again, we see effects across the whole distribution. And maybe we'll talk about this a little bit later in more detail. But the really interesting thing with this exercise in terms of looking how individuals across this predicted risk of recidivism respond. So we can see that, you know, they're treated- the individuals across- the high risk defendants are treated, the low risk defendants are treated, and then we can measure whether that treatment affects- so does the causal impact of diversion differ across these different types of defendants? And one of the most intriguing and I think important findings of our paper is that we find the individuals who are benefiting the most from diversion are those within this, you know, class of relevant offenders we're looking at, are those with the highest predicted risk of recidivism. And so those with the prior misdemeanor convictions, those that are more likely to come from disadvantaged neighborhoods or have, you know, more likely to be African-American, young African-American males, are the the individuals that that, when treated, are benefiting the most from treatment. So those are the individuals for whom we see the biggest improvements and reductions in reoffending, as well as improvements in labor market outcomes.

 

Jennifer [00:41:24] So before we dig too much more into the results, I want to talk a little bit about the data you have. So you're using data from Harris County, Texas, which I've started joking has become the Scandinavia of the United States. They've made such incredible administrative data available to at least crime researchers. So tell us just briefly about what datasets you're using and how they're all linked together for you.

 

Kevin [00:41:45] Yeah, and there's a lot of detail in the paper about this. And there's datasets that I won't mention, but I'll just describe the primary data that we're we're using and it would be the Harris County Court records. And so we're seeing, you know, a lot of information about individuals coming through the court in terms of what their prior histories look like, as well as what their case disposition is. And we can track individuals over time within that data for quite a long period of time. And then the other key dataset in our analysis is the - and this is being done through the Ray Marshall Center at the University of Texas, Austin, which has been really fantastic to work with. But we've also been able to link administrative employment labor market data through the Texas Workforce Commission to these individual level criminal records. And so that gives us a lot more information and I think a lot of really important results in terms of how these differential treatments in in the courts are affecting individuals, not only in terms of, you know, whether we're seeing them with convictions or court appearances in the future, but also whether we're seeing them with formal employment for more quarters, whether we see more stable employment over a follow up window, or whether we see we can see their earnings through the Texas Workforce Commission data. And then, you know, with the caveat that this is all formal labor market outcomes. So the employment and earnings that would be reported for unemployment insurance purposes. So there could be effects on, you know, informal work that we're not able to observe. But in terms of formal employment and earnings, we're able to link that information to these Harris County Court records. And then we're also able to see, you know, jail experiences through the county jail records and prison state prison incarceration experiences through the statewide prison data. And so there's some other datasets that we also are able to work with. But the key ones are the Harris County Court records, as well as linking that to the Texas Workforce Commission data.

 

Jennifer [00:44:07] So you're using those data. And what do you find in terms of the immediate effects of diversion? I guess primarily on punishment. And then how should we think about that changing the potential consequences for future crime?

 

Kevin [00:44:21] Yeah, and I think I think I've probably already given away a lot of the exciting results, but I'll just step through them slower and more detailed from- and some of the dynamics and the results that we see. We see those individuals that are being diverted due to these discontinuous policy changes at the court level with much lower reoffending rates. And we look at, you know, in our primary outcomes, we're looking at a 10 year follow up window. So we're counting the total number of convictions that a person has over a 10 year period. But we also have a lot of stuff in the paper that looks at outcomes, you know, more short term and more long term, at least for the '94 group where we can follow for for 20 years. What we see is a very kind of consistent and and striking picture that the individuals who are getting diverted are, you know, there are immediate benefits in terms of lower rates of convictions and then these as well as improvements in employment and earnings. And it seems like what happens is there's you know, there's differences that start to emerge right away. And then these differences continue to grow over time. And so the best way I think to describe or understand this is I think what's going on is individuals are just put on a completely different trajectory in terms of kind of their lives and also their interactions with the criminal justice system, as well as with a formal labor market. And so we see in both of these cases a pretty clear picture of both short and then increasing benefits over the medium and long term when we're comparing these outcomes.

 

Kevin [00:46:17] And so it's you know, it's it's quite striking I think when you look at the timelines of these effects in our paper in terms of how they just continue to grow for 10, you know, even 20 years out. And, you know, after 10 years, I think just to put some numbers to this- I'm just going to make sure I quote the right numbers. We have a lot of numbers in the paper. So for a 10 year follow up period- and combining these two- and so we do see consistent consistent benefits, I guess, associated with diversion from the '94 change to that 2007 change. And so to summarize that, we see declines- so, diversion is associated with declines in future convictions over a 10 year follow up period by about 50 percent or 32 percentage points. And so we see having of kind of future convictions for the treated individuals. And the quarterly employment rates are improving by around 50 percent as well. And that's a improvement of about 18 percentage points. And to put this in earnings dollars over that 10 year period, we see the individuals who were diverted earn over 60 thousand more dollars in that 10 year follow up period than the individuals who were not diverted. If we're interpreting our estimates as the causal impact of diversion, there's about a 60 thousand dollar formal earnings boost associated with diversion over that 10 year period.

 

Kevin [00:47:57] To a lot of your listeners, I'm sure there's a lot more and increasing attention towards this, you know, how criminal records are. And a lot of you know your own work, Jen, a lot of criminal record- how criminal records and criminal backgrounds are influencing individuals in the labor market is a really, really important topic. And the fact that we're able to link that data in here, I think is is it's just a really incredible opportunity for us to better understand kind of the overall impact of diversion on on individuals over a very long period of time or a long follow up period.

 

Jennifer [00:48:32] So just to dig into the mechanisms a little more, so you're, for various reasons, not finding an impact on whether these people are incarcerated or the amount of time spent incarcerated, at least in the short term. So you're you're interpreting this very much as these are the causal effects of avoiding the first felony conviction, essentially. And you argue that the main benefit here is you avoid that felon label, and I think that seems reasonable, there's certainly a lot of increasing evidence that that label matters. But it also struck me as I was reading this, that avoiding the label on that first offense also means that the state reserves the power to label you a felon later. Right? If you offend again. And so in this way, the subsequent punishment in terms of at least the collateral consequences is going to be higher for diverted offenders who reoffend than it is for non-diverted offenders. You've already been labeled a felon. You know, once you have that label, you can't- it doesn't really matter if you get labeled a felon again. So that's part of the package here, too, right? There could be sort of a deterrent effect that remains for this diverted group, even if the potential kind of deferred punishment is the same for both groups. Does that make sense?

 

Kevin [00:49:42] Yeah, you're right. Yeah. And I can speak to that in a little bit more detail. First, I just, and I meant to emphasize this earlier, but earlier on we briefly discussed incarceration and what role is that playing here. And we've, I think, provided a lot of detail in the paper about this and have increased the amount of detail we provided about this over the different versions of the paper. But what's happening in these experiments, it seems like, especially if we look at the '94 one, is individuals are going from getting diverted with a three to five year probationary supervision period. So that's happening to about 70 percent of the caseload and then it's decreased to about 40 percent. And what the tradeoff, what the offset is there is that the individuals who were being diverted before the policy change and are not being diverted after the policy change, they're getting these probated incarceration convictions. And so they're getting convicted of a felony offense and then getting placed on a three to five year probationary supervision. And so the really the the only difference there is is the first group who is being diverted has the opportunity to clear or to avoid that label, as you termed it, of the felony conviction if they successfully complete the probation. And of course, yeah I think your point is really important in that when we're comparing the behavior of those two groups, the first one, you know, obviously the cost of a future offense within that probation period. So if I go out and recommit an offense or I reoffend, then I'm not only facing the punishment associated with that new offense, but I'm then being labeled or I'm being- it's triggering all of the punishment associated with that first offense that was diverted or deferred. And so there is certainly as as we call it, in the paper an overhanging deterrence mechanism during the probationary supervision period.

 

Kevin [00:51:57] And that's so I think when we look at the effects early on, so within the first few years, it's very difficult for us to to kind of distinguish the pure labeling effect from that overhanging deterrence or that overhanging sentence effect. But what I think is helps us and helps. And I think the you know, for for us at least, our interpretation of our results is that we have pretty strong evidence that labeling this labeling mechanism is is quite important in generating our results. And part of the reason we think that and think that it's also important also given this overhanging deterrence effect, is that we see benefits kind of outside of the standard probationary supervision period. So it's not that we only see these differences emerge for the first three years or five years. We see differences continue to accumulate past that probationary period. And so if it was if suppose the only causal mechanism here was this overhanging deterrence, then we would we would expect to see benefits for the first three to five years, and then we would expect the two groups or those benefits, I guess, to stop accruing or to stop growing. But we see those benefits continue to accrue.

 

Kevin [00:53:21] And then the other piece of evidence that I think is is quite interesting and a little bit limited in what we can do within our paper is that- so for the 2007- for the '94 experiment, it really is only- we only have kind of that discontinuous change in diversion for first time defendants and first time property and drug defendants. For the 2007 change, this is the one where the judges seem to have just adjusted their behavior following the failed jail expansion ballot initiative. There we see- so we not only see first time felony defendants being diverted or being more likely to be diverted. We also see some second time felony defendants. So individuals who already have a previous felony conviction on their record being diverted. And what that gives us is an opportunity to say a little bit more about the labeling or stigma mechanism. Because if labeling is is important, we would expect that, you know, that first felony charge be much more damaging or important than the second one, because the second the person with the second felony defendant with a previous felony charge already has been marked with that label of being a felon. And so there we have and we have a smaller sample of individuals we can look at this with. But we do see evidence consistent with this kind of labeling stigma mechanism and that we see larger benefits for those who are the first time felony defendants in that 2007 experiment. So those two, I think, pieces of- all of the patterns and results that we see and acting in the comparison between the first and second time felony defendants gives us- that's, I guess, why in the paper we're interpreting it as evidence that this labeling or stigma mechanism is is quite important.

 

Kevin [00:55:26] And then one final thing I want to say about labeling in this context is that I think when we- oftentimes when we describe or think about labeling or stigma, we might- at least, you know, I was trained in labor economics and so we might be thinking from the labor markets perspective. But there's also an important kind of labeling stigma type effect that could be present in the criminal justice system as well. Because, you know, as as we know and as your listeners will be well aware of, that judges, when you're seeing someone come through the court, you're going to adjust your punishment or you might be adjusting the way in which you're treating defendants based on their prior histories. And so whether or not someone was diverted in the past and whether or not they successfully completed a diversion agreement in the past or whether or not they have a felony conviction in the past is going to affect those subsequent punishments for future offenses. And so there is this labeling stigma effect associated with prior felony convictions within the courts. And then there is also which could be influencing outcomes. And also the labeling stigma effect, of course, in the labor market, which has been, you know, we discussed earlier with especially with Devah Pager's work and a lot of others in that area. And so that's you know, I think the- both of those things are going on. They both- we think of both of those as labeling mechanisms. And we think that in, you know, the totality of our results, our analysis really pushes very strongly towards that being the really important underlying causal mechanism in that setting.

 

Jennifer [00:57:08] So what are the policy implications here? What when policymakers ask your advice about what they should do for especially low risk offenders, what's your what's your takeaway from this work and other work in this area?

 

Kevin [00:57:21] Yeah, I mean, I think the basic policy implication would be that- we- you know, it varies across jurisdiction as to how much they are using these types of programs. But our results certainly suggest that there could be potential savings associated with diverting more defendants or providing more defendants a second chance. Of course, you know, there's there might be concerns or risk of, you know, if if some of those defendants you're diverting, if they go out and commit a heinous crime or a very costly crime, they might be concerned with some, you know, effects in those direction. We don't find any evidence of that. We actually find reductions in many different types of crimes, including violent crimes associated with diversion. And this isn't really a paper about whether you're locking someone up or not locking someone up. And so kind of the risk to the community, I think, associated with diversion, if the alternative is, you know, a conviction with probation, I don't think there's much differential in terms of what the community risk factor would be.

 

Kevin [00:58:33] But we, you know, our paper strongly suggests that that second chance for these first time felony defendants is is a way to achieve large, potentially very large savings in the future in terms of the costs in the criminal justice system that the future reoffending will bring. And I guess the benefits to these individuals and their families and their communities associated with, you know, more attached labor market experiences and more stable labor market outcomes, as well as the lower rates of reoffending, suggests that diversion is something that could really save costs in the future.

 

Kevin [00:59:16] So one of the things I haven't mentioned that I think is important from the economist's perspective is that if we all of a sudden start diverting a lot of offenders of a certain type, we might be concerned about the overall general deterrence effects of a policy change. So all of a sudden we've made, you know, committing certain types of offenses much less costly or potentially less costly from the offender- potential offender's perspective. And so I think those are important considerations in our setting because these changes happen so rapidly and, you know, are not very transparent even to us as researchers and, you know, digging around for these reasons, for these changes. It's you know, it's pretty clear that it wasn't very transparent to the to the individuals, the potential offenders as well. And we don't see these general deterrence effects in our setting, but our setting is a bit limited in terms of whether we can detect those types of aggregate changes, because it's not something you would expect to change just continuously, I guess. And so, you know, I think that's an important caveat in terms of, you know, what would happen if we just started diverting everyone. You know, I don't- there'd be some important kind of concerns about how that might affect the calculated potential cost of of criminal activity for individuals making those types of decisions.

 

Kevin [01:00:46] But overall, I think, you know, these these increases in diversion could elicit savings. And I think one of the more the I think- one of the things we like the most about this result or this analysis and the policy implications of our paper is that this isn't going out and creating a new, you know, really expensive treatment program or having, you know, an alternative court that's deciding on certain types of offenses, like a drug court, things, interventions that might be much more expensive from the jurisdiction's perspective. This is really an intervention that's relying on the existing infrastructure there. In terms of Texas, it'd be the you know, the probationary supervision. And it's just, you know, it's it's a less costly, I guess, intervention or tweak or treatment that can be can potentially achieve pretty large cost savings. So it seems in some sense it might be a more practical solution for criminal justice reform in jurisdictions that maybe have some room to kind of increase the rates at which they're diverting defendants or have, you know, defendants that might potentially benefit a lot from from. Those types of agreements or those opportunities to to, I guess, clear their record or over, you know, successful completion of some sort of supervision.

 

Jennifer [01:02:19] Those are all really good points. So what's the research frontier here? What are the next big questions that you think need to be answered in this space?

 

Kevin [01:02:27] Yeah, I mean, I think that's I think that's a huge research frontier. And so it's kind of hard to nail down a specific thing. I mean, I think, you know, the similar- so I, I do think there's a lot of opportunity out there. And, you know, for your listeners that do- are working in other jurisdictions and have access to, you know, similar types of administrative data and are very aware of kind of, you know, institutional policy changes in the past, I think there's a lot of opportunity to learn about diversion and conviction in these types of changes by looking back at some of the policy changes. And potentially, you know, I think one of the things that we're seeing more and more in the crime literature is a lot more regression discontinuity designs, because I think the nature changes often as such that you have these very large breaks or discontinuities in in treatment. And so there- I think there's a lot of opportunity to go see if we do see similar effects in other jurisdictions or with other types of diversion if it's, you know, a specific type of diversion from a drug court or drug treatment program. I think there's a lot of room for learning a lot more about diversion in that case. And really important, I think that, you know, we we described this paper as about the causal impact of diversion, which it is, but it's also, you know, we're in a specific context and, you know, using data where we can identify the effect of that. But I think expanding that to other contexts, other jurisdictions and trying to use these natural experiments that have happened in the past to learn a lot more about both the short and long term outcomes of differences in treatment is something that I hope to see a lot more of in future years.

 

Kevin [01:04:18] And then. I think, yeah, I mean, I going back to something I mentioned earlier, about, you know, where the research is and what's being done, I think there's going to be a lot more work done on, like, say, the maybe a randomized program that encourages defendants to get their records cleared that are eligible. So some sort of random variation and record clearing or expungement, I think is an area where we're going to see more research. But I think and I think that's really important. But I also think this is- our study's a little bit, you know, it adds a little bit something different because it really is, you know, and I think you phrased this nicely earlier, where it's it's about, you know, having a treatment and then seeing how someone responds to that treatment. And so the record clearing expungement work, I think is really important. But it's also going to be at a stage where someone already has a conviction or already has a record and may have already suffered a great deal from that mark of that criminal record. And so it's I think it's very important for us to figure out what happens when we start removing those those marks.

 

Kevin [01:05:35] But it's also very important for us to start thinking about and being proactive about what happens when you give someone incentive or give someone an opportunity to to avoid that mark altogether, avoid the potential labeling effect altogether based on, you know, kind of turning their life around. And so I think in combination, all of this stuff is going to be really important for us and, you know, the policy community. But, you know, having work and continuing to do work on all of these aspects of how criminal justice reform and in particular how reform that's focused on second chances, I think is is something that, you know, I hope to see a lot more on. And I think that's, you know, that's it's a pretty big frontier. And, you know, I think there's a lot of people working on these types of questions. But what the, you know, what I think we have brought and what economists can bring to the table is this focus on on using these quasi experimental research designs and trying to find past policy changes that really get close to approximating a randomized controlled trial. So I think there's a lot we can learn now with taking data and looking at what's happened in the past. And then, of course, a lot of really interesting work that can be done with assigning- randomly assigning various interventions and then tracking those outcomes over over both the short and medium and long term.

 

Jennifer [01:07:03] My guest today has been Kevin Schnepel from Simon Fraser University. Kevin, thanks so much for doing this.

 

Kevin [01:07:08] Great. Thanks, Jen. It was a lot of fun.

 

Jennifer [01:07:15] You can find links to all the research we discussed today on our website, probablecausation.com. You can also subscribe to the show there or wherever you get your podcasts to make sure you don't miss a single episode. Big thanks to Emergent Ventures for supporting the show. Our sound engineer is Caroline Hockenbury. Our music is by Werner and our logo is designed by Carrie Throckmorton. Thanks for listening and I'll talk to you in two weeks.