Episode 47: Greg Midgette

 
Ep47_Midgette_photo.jpg

Greg Midgette

Greg Midgette is an Assistant Professor of Criminology and Criminal Justice at the University of Maryland.

Date: March 16, 2021

Bonus segment on Professor Midgette’s career path and life as a researcher.

A transcript of this episode is available here.

Episode Details:

In this episode, we discuss Prof. Midgette's work on the effects of the 24/7 Sobriety program:

“Criminal Deterrence: Evidence from an Individual‐Level Analysis of 24/7 Sobriety” by Beau Kilmer and Greg Midgette.

OTHER RESEARCH WE DISCUSS IN THIS EPISODE:

 

Transcript of this Episode:

Jennifer [00:00:08] Hello and welcome to Probable Causation, a show about law, economics and crime. I'm your host, Jennifer Doleac of Texas A&M University where I'm an Economics Professor and the Director of the Justice Tech Lab. 

 

Jennifer [00:00:18] My guest this week is Greg Midgette. Greg is an Assistant Professor of Criminology and Criminal Justice at the University of Maryland. Greg, welcome to the show. 

 

Greg [00:00:27] Thank you so much for having me. It's really exciting. 

 

Jennifer [00:00:29] Today, we're going to talk about your research on a program in South Dakota called 24/7 Sobriety, which aims to reduce alcohol related offending. But before we get into that, could you tell us about your research expertize and how you became interested in this topic? 

 

Greg [00:00:43] Sure. Yeah. So I'm - as you said, I'm an Assistant Professor in the Criminology Department in University of Maryland, but I'm a policy researcher by training. So I'm sort of a social sciences generalist with applied microeconomics. I do a lot of research on crime and drug policy and particularly how crime and drug policy interact with public health outcomes and sort of understanding the structures and mechanics of markets. So sort of the economics of crime and illicit markets, but then also a lot of policy and program evaluations. And this is one of the examples of the big program evaluations that I started doing as part of my dissertation, actually even before my dissertation, and have continued on in this sort of bane of research for a while. But I learned about 24/7 as a student in an MPP program at UCLA. I was a research assistant for Mark Kleiman and he was writing a book called "When Brute Force Fails," based on some of his prior research, particularly looking at Project Hope, which maybe some people have heard of, and was looking at other examples of these - at the time he was calling coerced sobriety, but eventually became a swift, certain and fair programs and 24/7 was anecdotally a program that was working really well in South Dakota. And so I'd heard about it a little bit, read about it, you know what was available online, and at that point, there was no peer reviewed research on it. Then started at RAND as a grad student at Pardee Graduate School and started working with Beau Kilmer, who won an RO1 from a NIAAA to study the program. And so that is sort of what we heard about it. And then I spent the next many years learning more and more and spending time in the Dakotas and Montana, actually observing how the program functioned, eventually measuring whether it works or not. 

 

Jennifer [00:02:34] So your paper is titled "Criminal Deterrence: Evidence from an Individual-Level Analysis of 24/7 Sobriety." It's coauthored with Beau Kilmer, as you just mentioned, and it was published this past summer in the Journal of Policy Analysis and Management. So let's start with the basics. What is the 24/7 sobriety program and who does it target? 

 

Greg [00:02:53] Yeah, so it was started as a pilot in 2005. It has a kind of pleasant, folksy origin story. There was a judge in a very rural county, Bennett County, which is now, I think, Lakota Sioux County in South Dakota named Larry Long, who noticed that he saw the same few people coming into his court on Monday morning. And it was guys that had gotten in trouble for drinking too much on Thursday or Friday or Saturday night. He said, if I can keep them from drinking, I'm going to keep them from showing up in court. And so he came up with this sort of what eventually became 24/7 as a sort of pragmatic way to not have to see these same people recidivate. And his idea was to test people frequently, basically twice per day, because for alcohol use after they're rearrested for a DUI, so it initially focused on drunk driving, and it actually - he focused specifically on people that had already had at least two DUI arrests. And they would test twice per day. If they didn't show up for a test or if they showed up for test and blew into a breathalyzer a value of anything other than zero, then they'd immediately go to jail for typically overnight, sometimes maybe a little longer. 

 

Greg [00:04:10] And so that was the idea of the program that when he eventually became Attorney General of the state and they were looking for solutions to drunk driving, which had, you know South Dakota's among the worst states in terms of drunk driving arrest per capita. They looked to this program, started it as a five county pilot, where they sort of picked a little bit randomly, not truly randomly among the counties across the state, a couple of big ones, a couple of small and a medium sized one, geographically dispersed. And then from that pilot where results looked really good, it grew by word of mouth. So judges heard about this sort of funky program working well and started implementing it shortly after the pilot phase in 2005. But then by 2007, they got state funding. A law was passed at the state level that allowed for funding for any county that wanted to implement it to support that implementation. But at that point, it was basically statewide, they were just imposing regulations and consistency across the counties for what the program would be. 

 

Jennifer [00:05:14] And so this is a program, so it's targeting people who have a history of alcohol use that gets them in trouble such as a DUI arrest. And then this is basically an alternative to traditional supervision. How does it compare to what would have happened without it? 

 

Greg [00:05:29] Yeah, so the traditional supervision case in South Dakota was - actually in a lot of states in the US - is regarding drunk driving and intoxicated driving relies really heavily on the threat of a severe sanction to prevent recidivism for drunk driving. So typically, if you're arrested for DUI, you're told not to drink anymore. You go through the judicial process and you receive a huge fine, potentially some time in jail. In South Dakota, the law is that for your third DUI, it's actually considered a felony, three DUI convictions is a felony. So, again, very punitive, but it relies on the act of drunk driving to impose a sanction and to impose any sort of outcome. From sort of survey data, know that there's not a whole lot of power behind that threat of detection and punishment because AAA, which they say is nationally representative, but I think AAA's survey was actually - so the Auto Club of America did a survey in 2014 asking whether people think they drove after drinking too much in the past year, and like 13 percent of people said yes. So that's a huge proportion of the population that thinks they drove drunk at least once. And that's imperfect. But we also know from more imperfect data at the national level, about a million to million and a half arrests for DUI, so not people, but arrests occur for DUI each year. So the detection rate is really low. And if the detection rates are really low, if you think about what creates a deterrent, it's the combination of - well Beccaria said it was swiftness, certainty and severity. And if something isn't very certain in the judicial process, the criminal justice process is really long and drawn out for good reason, then severity can be whatever you want, it still won't be very soon. And so that's what 24/7 sort of replaced - this highly uncertain but very punitive approach. 

 

Jennifer [00:07:37] And am I remember correctly that this is for most of these people, this is like pretrial. 

 

Greg [00:07:42] Yeah. Yeah, that's right. And that's shifted over time a little bit and other jurisdictions have changed the approach to focus more on post conviction. But in South Dakota, it's largely a pretrial program. So you're arrested. You already have a DUI or some other offense that you've been convicted of in your criminal history. If you're arrested again, then a judge can put you on this program or a court officer can put you on the program. But generally, it is a pretrial program. 

 

Jennifer [00:08:12] Got it. And then the alternative supervision, you would still be correct that one of the rules would be that you couldn't drink. They just weren't enforcing it. Is that the way to think about that? 

 

Greg [00:08:22] That's right. Yeah. Yeah, that's that's exactly right. So and that's true for a lot of states in the US where lots and lots of places you're told that if you're arrested and sort of might stand trial for an offense related to drunk driving, you'll be told - or intoxicated driving - you'll be told don't drink and don't consume drugs. And so 24/7 puts consequences behind that order and enforces it where most of the time it's not really enforced, even when people where - in a lot of jurisdictions people are ordered to have a ignition interlock device on their car or to wear an alcohol monitoring bracelet on their ankle. Often those are the data collected and then maybe observed and considered through court proceedings, but no action's taken until something bad happens, so a criminal event occurs. 

 

Jennifer [00:09:16] Okay, and so you talk about this a little bit, but let's step through it in a little bit more detail why we might expect 24/7 Sobriety to improve behavior. What are the underlying mechanisms for change here? 

 

Greg [00:09:28] Yes, so I'm tempted since I'm in the Criminology Department and nominally a Criminologist, that I should invoke theory, but -. 

 

Jennifer [00:09:36] Go for it. 

 

Greg [00:09:38] But I don't want to. I think - so, the the program reduces complexity in a bunch of ways in terms of decisions that people face after being arrested for a DUI. So before you go to work and after you come home from work or whatever you do during the day, you blow into a breathalyzer and you pay a dollar for the straw that you use. That's the consistent thing that occurs. That's keeping you from drinking too much, right, and thus keeping you from driving drunk. In that - in that way, my coauthor Beau Kilmer refers to the program as losing your license to drink, where if you've shown up in the criminal justice system multiple times for drunk driving or for another offense related to alcohol or drug consumption, then you need to demonstrate through the logic of this program that you can maintain sobriety, not necessarily abstinence, but sobriety. So once you establish that the program even refers to it as graduating from the program, that you can get through some amount of time with basically only the threat of sanction, there's no other intervention as part of the program and that sort of front loads and makes consistent a decision to not drink with a really punitive consequence, which might not be appealing for researchers or policy designers. But that's what they do. And that makes the decision relatively simpler than, well, I'm going to drink just to relax and maybe I'll have two. But then once you have a second, the third one is a relatively trickier decision because you're becoming inebriated and so forth. And so your ability to make good decisions might erode the more you drink, whereas if you prelude that decision, and say just don't drink at all, It's a simpler decision. And so that sort of, I think, is the mechanism behind how 24/7 is changing people's behavior. 

 

Jennifer [00:11:43] So before this study, what did we know about how to reduce alcohol related offending in general? 

 

Greg [00:11:49] So we know that people respond to prices. Don Kinkle had some research showing that increases in taxes are associated with decreases in the rates of DUI offending. I think it's for a 10 percent increase in taxes, DUI arrest decrease by about seven percent and a little bit higher rate among women. So if it was easy to increase taxes on anything, if that was a feasible policy option, we could probably just increase taxes on alcohol and make the world a little bit safer and healthier. But it's really hard to change tax rates and they don't often change. So we don't see that - those potential benefit. In terms of criminal justice based interventions, they're generally very expensive and complicated and complex, so things like DUI courts, but potentially effective, or they're less complex and not as promising. So things like ignition interlock devices where we've seen that if you're ordered to get an interlock device on the start of your car, you are probably not going to drink if that's how you get around. That's the only way you get around. Thus you're not going to drive drunk. But once that device is removed, so far, the evidence says the effect on reducing drinking and DUI is also removed. 

 

Jennifer [00:13:15] And these are devices have things that they - where you have to blow in to the monitor before the car turns on. 

 

Greg [00:13:22] Yeah, that's exactly right. It's a device that's attached to - it goes in between - it's an interlock, so it goes in between your key and the starter. And you have to basically demonstrate with your breath that there's no alcohol on it and then the car will start. Those are expensive devices, but otherwise there's relatively low administrative burden. At the same time, people might have more than one car. It's hard to enforce the policy where someone is having the device installed because there are a lot of DUI arrests. So those are sort of solvable problems. The thing that's tricky is that it seems to be when the device is removed, the effects of the device just don't stay with the person. And so then there's the evidence around specialty courts like DUI courts in particular is kind of mixed. They're really promising, really well done DUI courts seem to do very well, but they're complex. It involves a judge and a lot of interaction with the justice system and the proper identification of effective programing for people. So it takes a lot of effort on the part of the systems that are called on for the program and then buy in from those people, as well as from the people who are assigned to the program to get them to work well. And the particular mix of what is most effective for each person isn't perfectly understood, but they are really promising. I think National Academies recommended that every state implement DUI courts, but the perfect version of the court is not sort of settled science. 

 

Jennifer [00:14:57] Yeah, yeah. It does seem like there is very mixed evidence on those. 

 

Greg [00:15:00] And, you know, this sort of RCT level evidence, I know it's an older study now, but John MacDonald and other former RAND employees did an RCT, and the program, just didn't seem to work all that well. But it's been done many, many times in a lot of locations, and it's sort of up and down. Generally promising, but complex. Ben Hansen used a regression discontinuity and looked at the impact of a jump in severity of sanctions at the .15 BAC level in Washington state, so if you're arrested for DUI and your breath alcohol content, is at .14, you receive one sanction and then if it's .15, it gets much more severe. And so he's using some really cool detailed data, looked at that threshold and found that those who were just on the other side of it were less likely to recidivate for DUI in the future. So that means, you know, punishment or the severity of punishment could be meaningful, that it isn't useless to be more severe. It's just it should be done in a smart way. 

 

Jennifer [00:16:09] Right. So we're saying alcohol related offending, since we're trying to stop people from drinking too much, obviously that's going to be most salient when it comes to DUIs, but alcohol is involved in a lot of different types of crime, right? 

 

Greg [00:16:20] So the evidence around the social costs of excessive alcohol consumption is kind of staggering to me. Or it was when I first started studying this stuff. CDC had a report on 2016 that estimated the social cost of excessive drinking were on the order of a quarter trillion dollars per year. 10 percent of that is borne by the criminal justice system. The rates of substance abuse or alcohol abuse or dependance in people involved in the criminal justice system are really, really high. 

 

Jennifer [00:16:57] Basically, people do a lot of really dumb stuff when they're drunk, right, including committing crime. 

 

Greg [00:17:02] People are facing when they're drunk right. People that don't get in trouble normally all of a sudden get in lots of trouble when they've had too much. But one of the things that was surprising, I suppose we may talk about it in a little bit, was that even though this program, 24/7 was meant to target DUI, the impacts of it at the community level from one of our earlier studies were seen in domestic violence arrests and in assaults and in traffic crashes. So there's like a diversity of impacts of reducing drinking among people involved with the criminal justice system. It's not just reducing DUI, though that's a big part of it. 

 

Jennifer [00:17:47] So what did we know about the effects of 24/7 Sobriety, specifically before you and Beau both started this most recent paper? 

 

Greg [00:17:53] Yeah. So we had previously looked at community level outcomes. So when the county implemented the program, what happens, the sort of turn of events, sort of analysis. And so prior to our first work, there were descriptive studies that were funded by the states of South Dakota and North Dakota. They didn't go through peer review. They might have left some opportunities untouched for sort of a stronger research design. But our first study that was published in the American Journal of Public Health in 2013 looked at when counties implemented the program, how did repeat drunk driving offenses for a second through if they're more DUI, domestic violence, traffic crashes and assaults respond? What were - what do those counts look like? And was associated with about a 12 percent reduction in repeat DUI offenses at the community level, at the county level, county-month level, and about a nine percent reduction in domestic violence. And as I mentioned before, there wasn't really an intention to target domestic violence in the early phase of the program, though once they recognize that program seemed to be effective beyond just reducing DUI, they included more participants with different - that had been arrested for other sorts of offenses that seem to be linked to alcohol use. 

 

Greg [00:19:27] And so after that AJPH study Beau Kilmer, Nancy Nicosia, and Paul Heaton published another paper using basically the same research design, evaluating the impact on mortality. And they found that in counties where 24/7 was implemented - and I haven't really mentioned this yet, but 24/7 is a really large scale program in South Dakota. Probably over 50,000 people have been enrolled in it through last year. So between 2005 and 2020 and in some counties, over 10 percent of the men between 18 and 40 years old have been on this program. So it's a really large scale program. And so measuring the county level impact on mortality, they found a reduction on cause mortality and then particular concentrations of that effect were in circulatory illness and acute injury. So traffic crashes and the effects were a little bit bigger among women than men. So that was that was sort of the evidence to this point on 24/7. But community level difference-in-difference models are interesting and compelling, but you sort of want to measure at the individual level. And so that was the point of the show you were talking about. 

 

Jennifer [00:20:49] Right. You want individual level data. You ideally want to run an experiment, and that's hard to do in settings like this. So what are the hurdles that researchers like yourselves have to overcome in order to measure the causal effects of a program when you can't just run an RCT? 

 

Greg [00:21:03] Yeah, yeah. And ideally, we would want to run an RCT. As a researcher, you want to. You want to run an awesome field study, but it's tricky to pull that off. I don't think it's impossible. But the nature of a program, you know, randomly assigning some people to the potential for a really punitive outcome versus other people for much lower, maybe in probability higher potential costs, but by design not as immediately punitive. It's hard to get a defense bar and a judge to sign on to that. And I understand that. So practically sort of a nonstarter. And the data side, it's a monster. So a lot of the latter half of my PhD career of my grad school experience was cleaning data from criminal records that we collected from South Dakota and the administrative data associated with the program and then linking those together. And, you know, a smarter person than me, a data scientist, that a lot has been done since 2012 13, 14, when we started stitching these data together to make it easier to do that matching. But essentially it was getting lots and lots of information together that wasn't meant for research. Right. There's no code book understanding how people enter the data and what the bumps and works of the criminal history information we had were. It took a lot of time. 

 

Greg [00:22:31] But at the end we had basically the universe of people arrested for DUI in the state of South Dakota and detailed information in, so including all their criminal history information to basic demographic characteristics, including where they live. So we're able to bring in community characteristics as covariants in our models, but also understand how the program was functioning and how data were generated from the program. And that was a big task. And, you know, if it was a smaller intervention, it would have been a lot less of that - of figuring out how to stitch the data together. But because it was bigger, even though it is a real pain in the butt to get everything together, we had a lot of information on people arrested for DUI, some of whom were assigned to 24/7. 

 

Jennifer [00:23:20] Yeah. And then just to clarify the causal inference problem for people, you know, it might - the  sort of naive approach, might be to just compare people who were assigned to 24/7 to people who weren't, see what their outcomes looked like down the road. But that could produce a biased estimate for a couple of reasons. Right. Absolutely. Yeah. So we start with the naive estimate in the paper, and the idea is that there might be something about the people who are assigned to 24/7 versus those who aren't. It's potential that there's stuff that leads a judge to say this person's good for 24/7 or this person is not. That doesn't show up in the relatively rich, but not perfect set of descriptive data we have on individuals and their criminal histories. So that leads us to think there is a potential for selection if there's problematic selection. And the intuition behind our idea approach is that the program did roll out in that sort of ad hoc word of mouth kind of way. And generally, when judges have the option to assign people to 24/7, they do it. 

 

Jennifer [00:24:25] Yeah. So let's walk through that empirical strategy a little bit more. So you have this ideal experiment where you just randomly assigned people to 24/7 or business as usual. So tell us about your approach and how it approximates that ideal experiment? 

 

Greg [00:24:40] Sure. So I guess the intuition is kind of close to like a block RCT with non-compliants. So if you have an RCT and not everybody gets assigned to treatment and receives, you should use the assignment as an I.V. You don't just rely on assignment. Here, similar sort of thing where counties because the program's growth spread in this sort of hodgepodge kind of way. Over time, as counties start running the program, people start being eligible for 24/7 where you might be on one side of the border, driving 100 miles an hour on Interstate 90 in Brooking's County and then cross into Minnehaha County. And all of a sudden you go from a non treated to a treated jurisdiction. If you're arrested there, you'll be assigned to 24/7. If you were arrested a half mile earlier, you would just be assigned to business as usual. So the important thing there is that those counties really were not cherry picking when to enter 24/7. It wasn't just the highest need counties that were going first. And then we would therefore be different in some important way than the people because the people make up the county, then the people that would have been in the "untreated counties.". 

 

Greg [00:26:00] So then it gets to the idea of how do you consider - there is a lot of qualitative information that we collect in the field that said, it really was just this sort of ad hoc process by word of mouth. The judges started trying this program, but we tried to evaluate that as best we could empirically and looked at before counties entered or started running 24/7, were they higher DUI rate counties or lower DUI rate counties than those that didn't enter 24/7. And by and large, they looked quite similar in terms of the pre trend. 

 

Greg [00:26:34] And then among individuals, do we see balance across those who were assigned to 24/7 versus those who weren't. And we do. Across the things that we have measures for, the people look very similar. Again imperfect. It's not an RCT, but it seems like it's a plausible way to get it an identification, you know more convincing or more rigorous way than just saying those assigned to 24/7 are identical to those who aren't on average. 

 

Jennifer [00:26:59] Yeah. So if you have like the two individuals who look similar in terms of all the information you have, but one's assigned to 24/7 and one's not, usually we'd be like we wonder well is there something we can't see. Like there must be some reason that the person was assigned to 24/7 and this other person wasn't. And here you can say, well it's probably or at least more plausibly that this person lived in the county where 24/7 was in operation and this other person wasn't. And that's what led to the difference. It's not something unobservable.

 

Jennifer [00:27:27] So you're going to use an instrumental variable strategy where the presence of 24/7 Sobriety in your county is an instrument for whether you're assigned to that program. And these strategies are based on some key assumptions. So the first is that you're actually correlated with the treatment of interest, that's straightforward to show. But the other assumption, which is a bit trickier, is the exclusion restriction. And in this case, that means that whether 24/7 sobriety is up and running in your county only affects outcomes like recidivism through its effect on your participation in the program. So talk us through that a bit and why you think that's a reasonable assumption in the setting. And I guess if there are certain types of situations that might violate that exclusion restriction as examples that might be helpful to kind of think through. 

 

Greg [00:28:13] Sure, yeah. The empirical strategy relies on the idea that people will only enter 24/7 when it's offered and the conditions under which it's offered are conditionally random. If it ends up, for example, that DUI enforcement changes when 24/7 is put in place then we're confounded. That doesn't seem to be true. We have no evidence to think that that would be the case. If alternatively, you know, people living in counties with 24/7 or where 24/7 is implemented are then given differential exposure to treatment, some alternatives that would be related to their likelihood to drink plus drink and drive, then the estimates would be confounded. We rely in South Dakota, the availability of treatment and diversion programs is relatively low compared to a lot of the rest of the country. So that's less of a concern for us in this case. But it's not something that we directly measure. We don't have people's exposure to treatment or the utilization of treatment at the individual level. So the - fundamentally, though, the idea is that 24/7 being made available in the county really does - isn't driven by anything about the individuals or the characteristics of those individuals, thus creating or not solving endogeneity problem we're supposed to. And, you know, again, best we can tell, the roll out of the program from those five pilot counties up to basically statewide was just driven by word of mouth through most of them, and then toward the end, potentially by funding availability. 

 

Jennifer [00:29:56] Okay, so that's your empirical strategy. What data are you using for all of this? 

 

Greg [00:30:01] Yeah. So we - the people in South Dakota - we relied on the Attorney General's Office to provide data and they were amazing. That Midwestern time was absolutely true where we received criminal history information for every single person arrested for a DUI between 2000 and 2012 - through 2013, sorry. And that would include, if you're arrested for DUI, your entire rap sheet, including any information up until that moment where they sent us the data file. So that created - that allowed us to identify everybody who is arrested for their second, third, fourth, etc. DUI and also characteristics about them, including whether they were rearrested and what they were rearrested for, whether they were on probation or parole, whether they had violations of probation or parole, their age, gender, a race/ethnicity variable that had some missingness problems and that was merged together with the 24/7 participation data, which included everybody who was assigned to the program, what they were assigned to the program for, specifically what the arresting event was or what the number one offense in that arresting event was, when they were assigned to the program, and their participation characteristics, so all of their testing results. We have a data set of 11 million breathalyzer results, 99 percent of which are clean. But that is - it's a fascinating data set for other sorts of analysis. But it gives us some information about the length of time people are exposed to the program and how they did while they were on the program. How many times did they violate, for example? 

 

Jennifer [00:31:49] So let's get into the results. What do you find is the effect of participation in 24/7 Sobriety on being arrested for a new offense or having your parole revoked? 

 

Greg [00:31:59] Yeah, so we we looked at DUI two and DUI three participants in that comparison set - the counterfactual set of people who are arrested for DUI two or three - and found that at the one year mark, about a 15 percentage point decrease in the probability of arrest for any offense, at the two year mark about a 13 percentage point decrease, and then at the three year mark, a 14 percentage point decrease. And so at one year it's about a 50 percent reduction, 35 percent at two years and 25 at three years. So our estimates at the two and three year marks are precise, using our main model that bivariate probit IV, but they're not as precise using two stage least squares. So we're more cautious about interpreting those effects as being meaningful. 

 

Jennifer [00:32:50] That's still big coefficients all the way around. 

 

Greg [00:32:52] Yeah. Massive coefficients. Yeah. 

 

Jennifer [00:32:54] Yeah. So you run a bunch of additional checks to test the robustness of those main estimates. So maybe pick two or three of your favorites. Tell us about those and why you think they're helpful. 

 

Greg [00:33:05] Oh, I love them all equally. So I think that one that's interesting - cause when I think of 24/7, how the program actually functions, the program itself, if you are perfectly compliant, you never see a jail cell while in the program. But you do see, no matter how compliant you are in the program, you spend time each morning and each night, or if you're wearing a SCRAM bracelet, which we haven't really talked much about, and I probably should have, you're engaged with the program and interacting with the program office. So there's this degree of supervision. It's more intensive than typical. And if that's crowding out other activities that might have been associated with drinking, then it's sort of like the pseudo incapacitation. We're just eating up a bunch of your time where you might have been doing something else, either productive or unproductive. And I wondered if that - the effects we were seeing were just because the people who were at highest risk of rearrest were instead being violated on the program over and over again. And so just a combination of 12 hours of incapacitation through the brief jail stay and just the mechanics of being in the program was driving our results. And so when we limit to look at that, we limit - so it's that sort of people at the - in the long tail. But if we cut them out and just look at people around the program for two years or less, which is still the bulk of the people, we see even bigger effects. So it's not just the people who are quasi incapacitated for a really long time that are driving the program's results. It's - the effect is stronger among those who are compliant with the program and for a shorter amount of time. So that to me is interesting about the mechanics, you know the mechanism by which the program is effective. 

 

Greg [00:34:59] The other thing is that our IV approach is based strictly on our assumption or interpretation of the rollout of the program state wide. The threshold we decided on and this started with our first community level paper was 25 percent of those who were arrested for DUI in the previous year did enter 24/7. But that 25 percent considering that the on switch for program implementation is arbitrary. I could try and convince you 25 percent is meaningful, but I don't think I can do it effectively. So if we change that threshold to 10 percent or 40 percent, the intuition being 10 percent means very few people that will eventually be exposed to the program are counted as exposed. So we're sort of understating the program's rollout by using a very early threshold, like 10 percent and then relatively overstate, being more concerned with the 40 percent threshold. When we change those threshold values and then rerun the analysis. So how to consider the program to be active, our results aren't drastically different at all. So it's not - our results aren't very sensitive to how we define implementation in South Dakota. So that, from an empirical standpoint, is reassuring. It's not just we're picking a magic number where the effect is robust. So those to me are from - on the researcher side, interesting and informative and sort of reassuring about the approach. 

 

Jennifer [00:36:30] Yeah. And you also consider differential effects by type of initial charge. So whether it was someone's second or third DUI offense. So what did you find when you did that? 

 

Greg [00:36:39] Yeah, so I think to me this is the most interesting piece of the study, piece of the results is that we broke it up by DUI two arrestees and DUI three arrestees, and so there are many more DUI two arrestees then DUI three because not everyone recidivates, right. And we would expect the DUI three participants to have a harder time with the program because probably they're more likely to need some behavioral intervention or treatment or diversion and 24/7 doesn't have any of that. It's just - it's all stick, no carrot. So I would have expected our results to be concentrated in DUI two participants, but they're actually - the result is mostly driven by DUI three participants. We see relatively bigger effects among those who are on the program for DUI three. And that to me is surprising and also interesting that this sort of intervention, it's just testing and in this case, punitive outcome then can be effective for relatively higher risk people. It might also mean the program is too burdensome or unnecessarily burdensome for DUI twos. But we don't have evidence to say that with any real confidence. 

 

Jennifer [00:37:53] Yeah, cause this program is basically all enforcement, right. And yeah, I imagine a lot of the opposition to this kind of program is people who have alcohol use problems need real treatment. And so this is just fundamentally not going to help them. They are not going to respond to it. And your results show the opposite. 

 

Greg [00:38:11] Well, yeah, maybe. There are sort of extra wrinkles that I think are important to consider for future research. It may be that we do see stuff that on its face says, well, it's people who are probably higher risk of recidivism, are benefiting relatively more from the program. That may be about as much about the setting as the program. We might expect that if you ran an identical program and placed it like a Lego piece on top of the structure as it exists in Maryland, where I am, or California, where I wrote this paper, there are a lot of other programs and services and interventions that someone might be exposed to in those other states that don't exist in the same way, at least at the same level or intensity in South Dakota. So this might have just been pent up demand for an intervention of any type and 24/7 was the thing that existed that was effective. 

 

Greg [00:39:07] The other complication is that 24/7, because it's effective for many, but not everybody, the people who reenroll in 24/7 who are reoffending, it's demonstrated to not be effective for them. So we see that people who are reentering 24/7, it's not like, well, it's going to - it's going to take hold the second time or third time. It's less likely to. And we actually have empirical evidence that it didn't work before, so let's try something different. So we've seen - since the program started, I believe the first time I looked at the statistics on the web - on the Attorney General's website, something like 99.6 percent of breathalyzer tests were taken and passed, which is exceptionally high. Today, I think it's right at 99.0 percent. And that's still exceptionally high, but it's way lower than 99.6. And that, in part, is driven by the people who are entering the program now are people who have been on it before. And so it highlights the idea that this can be effective. But I think a lot of scholars of drug and alcohol use and interventions have recommended you can't just do one thing. I think "Paying the Tab," Phil Cook's book talks about a portfolio approach, and this is an example of one very effective thing, but it can't be the only thing. 

 

Jennifer [00:40:29] Right. There's going to be at least a subset of people who need more than this or something entirely different. Yeah. So the last piece of the analysis in the paper that I want to talk about is you look at the types of new offenses that are deterred by the program. So what types of - whether it's future DUIs or violent crime or property crime that people are no longer committing after they go into this program? So what do you find were the effects on each of those crime types? 

 

Greg [00:40:52] Yeah, so we found that among DUI two, DUI three participants, the - most of the effect on rearrest was on DUI. We saw a little bit of evidence of a reduction in violent crime at the one year mark. But the majority of the effect is DUI two participants not getting another DUI two. Some of that is just because violent crime and property crimes, other sorts of criminal justice involvement are relatively low in rate for this subset of the participants. And this is about 50 percent of 24/7 participants in South Dakota are DUI participants. But yeah, it's 24/7 for DUI offenders or people who are arrested for DUI, or a repeat DUI, prevents future DUI, mostly, maybe some effect on violent crimes. And that's interesting when we think about the effects that we saw from the community level paper on domestic violence. So maybe that small effect that we imprecisely estimate for violent crimes is picking up the domestic violence measure. It might also be that it's people who are assigned to 24/7 for domestic violence are those who are - where we see the reduction, but we haven't measured it yet. 

 

Jennifer [00:42:12] All right, so that's your paper or your most recent paper. Have any other papers related to this topic come out since you first started working on this study? 

 

Greg [00:42:20] A few. We're the only 24/7, when I say we, Beau Kilmer, Paul Heaton, Nancy Nicosia and I have been working on this a lot for a long time. The sort of swift, certain, fair approach has been studied in other contexts so the hope replications, for example, are looking at a similar program, but not identical, a different program when it comes to the particulars where the replications of hope RCT from Hawaii didn't look as good as that RCT, but they might have been underpowered. Replications, replication studies that we've done for 24/7 look pretty promising. So I basically took the model from the community level analysis of 24/7 in South Dakota, and we did it for the North Dakota version of the program and found again that the program reduces DUI, though importantly, the North Dakota program has some subtle differences in the way it's implemented. So it produces a lot more violations, meaning there's maybe some of the effect that we're measuring is incapacitation because people are going to jail. So it's working a little differently. But in terms of the size of the impact on 24/7 or on drunk driving arrests, it's very similar. So that's one study. 

 

Greg [00:43:43] And then there are a couple others under review that until they go through peer review completely, I probably shouldn't hang my head on the results. But essentially there's a lot of promising evidence from the Upper Great Plains that 24/7 is effective. There hasn't been much evidence outside of those studies, though. 

 

Jennifer [00:44:00] So that's great. So is North Dakota, South Dakota and Montana - are those the states that you just mentioned? 

 

Greg [00:44:05] That's right. Yeah, that's right. Yeah. And similar pilots have been tried in a lot of different places. I haven't seen anything get into journals yet from those though. 

 

Jennifer [00:44:15] Interesting. All right. So what are the policy implications of the results of your own work and the other work in this area? What should policymakers take away from all this? 

 

Greg [00:44:24] I think that this provides evidence that there's a relatively low cost and low burden way to reduce drunk driving, to reduce intoxicated driving. If you target participants who've been arrested for drunk driving a couple of times before this program where you are basically giving a few simple instructions with a strong incentive to comply, which is a punishment, you can reduce drunk driving. The finer details about how a program would look outside of the Dakotas and Montana are also interesting because a lot of jurisdictions are put off by the structure of this program that it is so punitive and there's not really any other component to it and sort of a rehabilitation component or treatment component. 

 

Greg [00:45:12] The interesting thing, from my perspective, is the potential for alternative versions of the program that are not as punitive and instead rely on incentives early on, because a lot of people, about 50 percent of participants in South Dakota never violate. So those who never violate, it's effective. Those who violate once, about half of them don't violate again. So if we're able to structure a program that relies on incentives for - early on, maybe we'll get similar results and not have to be punitive. Further, it provides data about people's ability to maintain compliance with an order not to drink. So an order not to drink and drive practically just by testing them and looking at the information with a threat of punishment, and if they're not able to comply with that, you can ratchet up services and you can prioritize who should receive the service. So Angela Hawken referred to that as a sort of behavioral triage model where we can use information generated from the program to sort people into other sorts of interventions as appropriate. And that hasn't really been done yet. So I think there's a lot of evidence that the program is promising and then there's a lot of potential for the mechanism that the program uses to be effective in some other ways. 

 

Jennifer [00:46:32] And so speaking of that, what do you see as the research frontier here? What are the next big questions in this area that you and your colleagues will be thinking about going forward? 

 

Greg [00:46:41] So related to the idea of using incentives or positive reinforcement rather than sanctions, can we find the minimum effective sanction or would a reward structure work? And since this program is reaching people who might not have much contact with the criminal justice system, can we use it to combine other sorts of interventions or services? We haven't really looked at whether 24/7 or a program like 24/7 can be combined with treatment or help with the sort of risk and needs assessment in an empirical way. It's different than a reliance on a survey and criminal history information. So how do we use the information generated by the program to figure out more effective, more complex but more effective interventions for people that keep showing up in the criminal justice system because of their drug or alcohol use. 

 

Jennifer [00:47:37] My guest today been Greg Midgette from the University of Maryland. Greg, thanks so much for talking with me. 

 

Greg [00:47:41] Thank you so much. It's been a pleasure. 

 

Jennifer [00:47:49] You can find links to all the research we discussed today on our website probablecausation.com. You can also subscribe to the show there or wherever you get your podcasts to make sure you don't miss a single episode. Big thanks to Emergent Ventures for supporting the show and thanks also to our Patreon subscribers. This show is listener supported, so if you enjoy the podcast, then please consider contributing via Patreon. You can find a link on our website. Our sound engineer is John Keur with production assistance from Haley Grieshaber. Our music is by Werner and our logo was designed by Carrie Throckmorton. Thanks for listening and I'll talk to you in two weeks. 

 

Jennifer DoleacInstrumental-Variables, Health, Pre-Trial