Episode 22: Phillip Levine

 
Ep22_Levine_photo2.jpeg

Phillip Levine

Phillip Levine is the Katharine Coman and A. Barton Hepburn Professor of Economics at Wellesley College.

Date: February 4, 2020

Bonus segment on Professor Levine’s career path and life as a researcher.

A transcript of this episode is available here.

Episode Details:

In this episode, we discuss Professor Levine's work on gun exposure and accidental shootings:

"Firearms and Accidental Deaths: Evidence from the Aftermath of the Sandy Hook School Shooting" by Phillip B. Levine and Robin McKnight.

OTHER RESEARCH WE DISCUSS IN THIS EPISODE:

 

Transcript of this episode:

 

Jennifer [00:00:07] Hello and welcome to Probable Causation, a show about law, economics and crime. I'm your host, Jennifer Doleac of Texas A&M University, where I'm an Economics Professor and the Director of the Justice Tech Lab.

 

Jennifer [00:00:17] My guest this week is Phil Levine. Phil is the Katharine Coman and A. Barton Hepburn Professor of Economics at Wellesley College. Phil, welcome to the show.

 

Phil [00:00:27] Thanks. Glad to be here.

 

Jennifer [00:00:29] We're here to talk today about your research on the effects of gun exposure, on deaths from accidental shootings. But before we get into all that, could you tell us about your research expertise and how you became interested in this topic?

 

Phil [00:00:43] Well, so by training, my degree's in labor economics, and in theory, that should be sort of what most of my work is about. It turns out that I've expanded a lot in terms of the things that I work on to now I consider myself to be an expert on social policy—so things that affect people's well-being, I care a lot about. Those sorts of things often are topics that are not generally considered to be economics. But that doesn't mean you can't use any economist's toolkit to address them. And that's sort of my expertise and I got interested in this topic, to be quite honest, just from a- it was a New York Times article one day. There was a story, I don't know, maybe four or five years ago, three, four years ago, that just the the main item of the story was a graph that showed trends in gun sales and focusing on some of the spikes that exist in those trends. And there is, you know, an extremely large one—which we'll detail later—right after right following the Sandy Hook school shooting. And, you know, as an economist, that just, you know, it's sort of like a light bulb going off in your head, like, well, what's up with that? You know, why did that occur? If that occurs? You know, if there's like all of a sudden there's there's millions of extra guns on the street, you know, certainly that must have some effect on something. And that's what started this whole project for for Robin McKnight and I, who's my coauthor. She's also at Wellesley College.

 

Jennifer [00:02:19] All the best research papers come from- out of newspaper articles somewhere along the way. Yeah, so your paper is titled, "Firearms and Accidental Deaths: Evidence from the Aftermath of the Sandy Hook School Shooting." As you said, it's coauthored with Robin McKnight. In it, you consider the effects of this spike in gun purchases in the wake of the mass shooting on subsequent deaths from accidental shootings. So to set the stage for what you're doing in this paper, tell us a bit about accidental shootings. How common are they? Who are the victims of these types of shootings and how often do accidental shootings result in death rather than injury?

 

Phil [00:02:53] Let's just start out by saying they occur way too often, so roughly 6,000 accidental shootings occur per year. Of those, 500 of them result in a death. Among children age 14 and under, they're pretty common 900 times a year an accidental shooting occurs, and that leads to 60 deaths per year. It turns out to be the case that firearms are the third leading cause of death for children under the age of 14 behind motor vehicle accidents and cancer. And one third of children firearm deaths are accidental.

 

Jennifer [00:03:28] So gun violence is an important topic and gun control is an extremely controversial one, you're focused here on the effects of gun exposure or gun access. So before this study, what had we known about how exposure to guns affects various outcomes?

 

Phil [00:03:45] Well, it turns out to be the case that there is a decent amount of evidence out there, most of it is correlation- pretty much all of it is correlational in nature. So if you look over time, it turns out that gun sales have been rising pretty continuously over the past 20 or so years and at the same time, firearm deaths are falling. So, you know, that correlation is negative. And so it seems like from that evidence, you know, more gun sales leads to fewer deaths and sort of, you know, opponents of gun control sort of point to that as evidence of their point of view. On the other hand, if you look across states, states where gun sales are highest also have the highest rate of firearm deaths, and that suggests a positive correlation. And so, you know, advocates of gun control tend to point to that sort of evidence. Of course, in the end, you know, we all recognize the point that correlation is not causation, and so neither of those pieces of evidence are particularly compelling. So we obviously need to go beyond that. I think in that- for that particular question that we address in this paper and that that evidence examines, there really isn't a whole lot else. You know, there's broader evidence on the impact of gun control legislation, you know, more specifically, things like background check laws, open carry laws, things like that. You know, that evidence is reasonably controversial but does go in the favor, you know, I would conclude of indicating that the greater access to guns does lead to increased crime.

 

Jennifer [00:05:20] So let's talk about the empirical challenges to studying the effects of gun exposure that lead to, you know, the relatively thin literature in some places or controversial literature that you just talked about. So what are the hurdles that you and Robin had to overcome in order to answer this question? Is this question difficult to answer because of data constraints or identification challenges or is it some combination of the two?

 

Phil [00:05:41] Yeah, I- obviously both. Data is data is always an issue. So, you know, I think the underlying question is about gun exposure and that's a relevant concept that makes a lot of sense, but it's not obvious exactly how you measure it. You can think about measuring it with gun sales or gun ownership or things like that. You don't really have perfect data on those sorts of things. What we use in our analysis is very good data that we have on background checks that are required when somebody purchases a gun by law. And so, you know, a common reason for getting a background check is you're about to go buy a gun. So it's certainly correlated with gun sales, which is, again, correlated with exposure. And that's sort of how we measure it in this paper. That leaves us with a question of whether accidental deaths rise in response to an increase in sales. That definitely raises issues in terms of identifying a causal relationship. You know, perhaps there's a spate of high profile accidental firearm deaths that occurred and led people to be worried about buying guns. That would lead to a negative correlation between sales and deaths, but that wouldn't be causal. So what we need is a way to measure changes in sales that that occurred as if it were an experiment. Because, you know, the key element of an experiment is randomization, and that's where the spikes in gun sales come in. We treat that as random in the context context of trends in accidental shootings.

 

Jennifer [00:07:06] Right. So you're going to use the Sandy Hook school shooting as a natural experiment because some combination of the shooting itself and policymakers conversations afterward led to that spike in gun purchases that you observed. So tell us more about the context of what was going on at this time.

 

Phil [00:07:23] I think there's a common perception that gun sales spike any time there's a mass shooting. That isn't really true. So, for instance, if you look after Las Vegas, we don't see that sort of thing. What the data show is that sales spike whenever people are fearful that gun control legislation may be enacted. A mass shooting is one potential trigger for that, but not the only trigger- potential trigger for that. Sandy Hook definitely represented one of those triggers. Immediately after the Sandy Hook shooting occurred, within two or three days or something like that, President Obama began speaking very forcefully about the need for greater gun control legislation. And he delivered a formal speech, like I said, a few days afterwards, indicating that he'd be working towards enacting new legislation. That was in December of 2012. After the Christmas holidays in January, he followed through on his promise and and work and worked towards introducing new legislation that got a ton of publicity. And you know, what really affected the gun sales was not the event itself. It was the fear that additional legislation may be enacted and that, you know, people would be taking guns away from gun owners. That generally- so that's the context of this event. That's why gun sales really spiked at that time.

 

Jennifer [00:08:46] And you use a few different datasets to show that the exposure to guns increased in the wake of these events. This is easier said than done, you alluded to the data challenges before. So you had to get creative in finding the relevant data here. So talk more about what datasets you use as a proxy for gun exposure.

 

Phil [00:09:03] Sure, so the main source of data that we use is these FBI background checks. So basically, for the most part—this isn't uniformly true and it's one of the policy issues that people fight about—for the most part, if you- when you go to buy a gun, a background check needs to be conducted. And so there's a process where you go to the gun store or whatever, and you have to fill out an online- there's an online form that gets filled out. It goes in the FBI. The FBI does a quick background check on you and determines whether or not you can buy that gun. So every time that that request is made, it's recorded in an FBI database. So there's, you know, even just online, you can access these data. And that's really what we use are these FBI data and sort of counts of background checks. Depending on circumstances, it could be by month and year, by state, or whatever. There's different ways it gets categorized and we rely on those data heavily in our analysis.

 

Phil [00:10:04] The other thing which we do use a little bit is data from Google Trends. So Google Trends is the service that Google provides that enables users to to see or to measure what other people are searching for. So if you type in, you know, New England Patriots, because I'm from Boston, if you type in the New England Patriots, you can see over time how many people are searching for for the term New England Patriots. So like on a, you know, fall or winter Sunday, you'd see a spike on one of those days for people searching for a term like that. We got creative in our use of Google Trends data and wanted to search for things like how often do people ask Google about buying a gun. Like "where do I buy a gun?" or "cleaning a gun" so, you know "what's- how often do I need to clean my gun?", searches like that. So basically, you put quotation marks around the term, it gives you exactly that term. And so "buy gun" and "clean gun" are things that we used. And, you know, amazingly, you see spikes, you know, immediately after Sandy Hook, essentially, like literally within the day of President Obama's speeches, you see these these sorts of spikes. And that is, again, sort of further indication that what really is being captured in the debate on gun sales spikes is this notion that people are afraid of having their guns taken away.

 

Jennifer [00:11:37] And I think both of us are- or it's obvious to both of us because we work in this space but might not be obvious to people listening, perhaps more obvious data to get for something like this is precise data on gun ownership, right, like who actually has a gun in their house and where is it kept or something like that. And that data just doesn't exist.

 

Phil [00:11:55] So, I mean, there's there's periodic surveys of things like, you know, rates of gun ownership. And certainly on an individual level basis, you're not going to get like a census of gun owners, that doesn't really happen. You know, but we know basic statistics like, you know, roughly a third of households have guns. They tend to have- if they have a gun, they tend to have multiple guns. You know, it's from sources like that that we know something like, you know, there's more guns in the United States than there are people in the United States. You know, roughly, let's say something like 400 million guns exist in this country, maybe more than that for the, you know, 330- or -40 million people that live here. So we have basic statistics about gun ownership, but nothing that would really be, you know, useful in a statistical analysis, the way we seek to do.

 

Jennifer [00:12:40] Right. Those survey data wouldn't be useful here. OK, great. So so you're, of course, interested in measuring the causal effect of gun exposure on outcomes. So as a first step, you need to measure the causal effect of Sandy Hook on gun exposure. So tell us how you do this.

 

Phil [00:12:58] So we do this in two different ways, the first way that we do it is we use national data and, you know, looking at gun sales over time, sort of on a monthly basis. And at that point, we're really looking for, you know, not just a little blip, but sort of at least obviously noticeable, if not large, spike in sales that occur exactly at the moment that we care about, like, you know, starting in December of 2012 and in the months that follow. In those kinds of analyses, you have to do things like abstracts from ongoing ongoing trends. So as I mentioned earlier, gun sales have been rising, you know, reasonably steadily over time through this period. So you don't want to confuse the general rise in sales as capturing something about Sandy Hook. You also don't want to confuse normal seasonal variation. So, for instance, it turns out that December is in general a very good month for gun sales every year. And so you need to adjust for the fact that, you know, different months of the year have these different seasonal patterns. And so we use statistical methods to control for those as well. And so on a national level, basically, the goal is to abstract from the underlying trends, abstract from the seasonal variation. And then you can look to see are there any, you know, month to month movements that are at all meaningful? Mostly what you see is like what looks a lot like statistical noise. It just kind of bounces around, goes up and down and whatever.

 

Phil [00:14:37] But then in just a few instances, you just see this very large jump in sales that follows immediately after the Sandy Hook school shooting, timed exactly with President Obama's push towards gun control legislation. And you're sort of left with this conclusion like, well, geez, that seems like it's related. You know, what else could it have been? We can do the same sort of thing using state data, which also is available from from the FBI and sort of the advantage of using the state data is we can do exactly the same sort of thing. But we can also rely on the fact that in some states, preferences for gun ownership are stronger than in other states. And if that's the case and people are worried about having their guns taken away from them, in the states where gun ownership is high, we should see larger spikes. And we see patterns that look a lot like that as well. So all of this goes towards, you know, there's a shock that occurs, it occurs exactly at the right time, it occurs in exactly the right place, and that certainly seems a lot like a causal effect.

 

Jennifer [00:15:51] Right. So the underlying assumption here is that there wasn't anything else that happened at exactly that same time, exactly the right places that would be driving gun sales other than Sandy Hook.

 

Phil [00:16:01] It's not impossible.

 

Jennifer [00:16:01] It's not impossible, but. Right.

 

Phil [00:16:04] But it just seems pretty unlikely.

 

Jennifer [00:16:06] Right. Exactly. So how big was that- was the effect of this event on exposure to guns? And I guess the next question then is how do you convince yourselves that the effect was big enough that it wasn't just due to random noise in gun purchases?

 

Phil [00:16:20] Yeah, it's really big. So, you know, in normal months, you see, you know, the normal monthly noisy fluctuations or, you know, maybe it goes up 50,000 or down 20,000 or whatever per month. But this is a 3 million- 3 million additional guns were sold in the five months following Sandy Hook. You know, this is on top of the normal monthly level of sales that takes place. So, you know, roughly speaking, about 2 million guns are sold per month. So, you know, over a five month period, it should have been, quote unquote, something like 10 million sales. And we're observing 13 million during this period, even after controlling for trends in seasonal variation and things like that. My personal belief is I think that's really big. The fact that the timing is so strongly tied to the Sandy Hook shooting. And then even when we confirm further using the Google Trends data to literally see the day the president spoke, you see sales shooting up. And those sales really continued at a higher level until the day that the legislation died—I think it was in the Senate that the legislation died—when they had the final vote No in April, sales returned right away to right back to where they were, to their normal levels. It just you know, it really is difficult to imagine what else it could have been. And it's, like I said, I think pretty big.

 

Jennifer [00:18:00] So next, you consider whether that big spike in gun exposure resulted in more accidental shootings, the outcome that you actually care about. So first, why are you focused on this outcome? And second, where do you get the data to measure it?

 

Phil [00:18:15] So I'm going to reverse your question and answer the data part first. The data comes from vital statistics data, so it's possible to get from the Centers for Disease Control sort of microlevel, individual level data on death certificates with precise causes of death. It requires additional permission and you have to sort of be deemed qualified to use those data, but it is possible for researchers to get access to those data. So essentially, I have on my hard drive—or had on my hard drive—you know, every single death certificate during the sample period that we used with a specific cause of death, and the specific causes of death include things like firearm deaths. And so for firearm deaths, we could measure things like homicides, suicides, and accidental shootings. It made sense to us and perhaps we can talk about this later. You know, we did investigate other forms of shooting, but it made sense to us that accidental deaths were the ones where we felt like we are more likely to find an effect. And the reason for that is- and what you worry about with an accidental shooting following a gun sale is that the person goes to the store. They buy their gun. You know, it's sitting out on the coffee table. You know, perhaps inadvertently it's loaded and someone picks it up and a bad thing happens. You know, it's a it's a tragic event. But you can you can see how it could potentially happen. In some sense, it's about the sale, and sometimes the reason why it's out is because it just got purchased. This also by the way is related to this notion- broader notion of exposure, that it could be that, you know, the fear of gun control legislation not only may lead people to buy a gun, but also if they already own a gun, to take it out of their locker and inspect it, thinking: this is- well, should I buy a new gun? Should I not buy a new gun? And then they take their old gun out. And again, once it's out, out is a bad place in terms of likelihood of an accident. And so sort of the mechanism for how an accidental death could result from these additional gun sales seemed clear clearer to us.

 

Phil [00:20:41] Homicides and suicides sort of require the access to the weapon, but they also require more than that. There's some intent that's involved and that seemed less likely to us to be related to the purchase of the gun. And so, you know, to be quite honest, we did investigate those outcomes as well, and we didn't find anything. Not anything in sort of like, yes, we know it didn't have any effect, but like it was just too difficult to tell given the noise in the data. And so in some sense, what we found kind of matched what we had a little bit been expecting, this notion that like if we were going to find something, it would be in accidental firearm deaths.

 

Jennifer [00:21:22] Yeah. And so the mechanism here then is both, as you said, you know, the gun is out now when it might have been locked up before or it hadn't even been in the house before. It seems like when you're talking about new gun sales—as I'm listening to you describe it—there's also an element of perhaps these are people who hadn't owned a gun before and might be inexperienced in keeping it or handling it safely. Is that part of the potential mechanism too?

 

Phil [00:21:49] Yeah, at the end of the day, you know, what needs to happen is there needs to be a loaded gun that's out and somebody not caring for it properly at that exact moment. Not to place any blame, these are obviously accidents. But, you know, it's it's not where it's supposed to be for whatever reason. And then a bad thing results.

 

Jennifer [00:22:16] Right. OK, so using the same strategy as before when you were measuring the impact of Sandy Hook on gun exposure—so it's essentially a difference in difference analysis comparing the time period after Sandy Hook with the same time period in other years and controlling for seasonal variation and all of that—you measure what we researchers would call the reduced form effect of Sandy Hook on accidental shooting deaths. So what do you find in that analysis?

 

Phil [00:22:42] So I like to think about, you know, difference in difference methods are things that economists sort of routinely like to talk about. I actually think about this as a little bit different than that. I like to think about this is like a spike to spike difference in the sense that, you know, you see this obvious spike in one thing, that spike certainly seems like it was a random event or can be thought of as a random event, at least in terms of its timing. Does something else spike at exactly the same time? That's the idenitification strategy. It's like you see one spike here. It has this potential to cause an impact something else on something else. Do we see a spike in that something else at exactly the same time that starts when the gun sales spike starts and stops when the gun sales spike stops? Again, if that's the case, it's possible that there's other things that are going on that happen to have occurred at exactly the same time, but it just seems so unlikely that that's the case that we can certainly think about attributing causation to that.

 

Jennifer [00:23:51] Yes, totally agree. It's a very- I agree, especially when you see it visually. You've got some nice figures in the paper that make it very clear that this is- everything's lining up perfectly. So what do you what do you find in that analysis? What are the effects that you are measuring there?

 

Phil [00:24:06] Yeah, so like you said, you know, through using statistical methods and even visual methods, because it shows up in the graphs, you know, we see the spikes occurring at pretty much exactly the same time. And so, you know, in terms of the numbers, what we find is that the 3 million additional gun sales that took place at that time, you know, led to roughly 60 additional accidental deaths.

 

Jennifer [00:24:29] OK, and then you build upon that kind of straightforward, reduced form approach by using the timing of Sandy Hook as an instrument for gun sales in a two-stage least squares analysis. The assumption there is that the timing of Sandy Hook affects accidental shooting deaths only through its effect on gun sales. So if that assumption holds, then this approach allows you to quantify the effect of one additional gun sale on accidental shooting deaths rather than just the effect of Sandy Hook on shooting deaths. So talk us through that a bit more. I think it seems like a plausible assumption, but convince us that's a plausible assumption.

 

Phil [00:25:01] Sure. So, you know what would have to happen for this not to be true? So suppose, for instance, there were, you know, a terrorist attack and people were nervous another one would occur. They may buy more guns. And because they're on edge from the first attack, they also may be more likely to fire their weapon if something startles them. Additional death that way would be about the terrorist attack and not the gun sales. In that context, I would say this is a plausible alternative alternative interpretation. It's hard to imagine how Sandy Hook could have led to additional accidental deaths otherwise through any other mechanism besides the spike in gun sales. One of these other things would have had to have happened at exactly the same time. That doesn't mean it is impossible, but it seems just extremely unlikely.

 

Jennifer [00:25:52] Yeah, it actually—thinking about the, you know, not looking at homicide and suicide also—it actually seems like there there are other potential mechanisms right there, which we might expect Sandy Hook to have effects on homicide and suicide. They're more in line with your your terrorism story. And it's another reason to just look at the accidental deaths. Seems like much harder to think of other stories there.

 

Phil [00:26:12] Exactly. Because, you know, at the end of the day, the accidental deaths are accidents.

 

Jennifer [00:26:15] Right. Right. OK, so what do you find is the effect of gun sales, one additional gun sale on accidental shooting deaths using this IV strategy?

 

Phil [00:26:25] So, you know, at the end of the day, we find the exact same result using the IV strategy as the reduced form strategy, which is a good thing. It seems to suggest that maybe we were doing something right. That's what's supposed to happen.

 

Jennifer [00:26:37] Right. OK. And finally, you take advantage of the fact that U.S. states are quite different in terms of their attitudes toward guns. Not everyone responded to talk of gun control by buying more guns. So how do you identify the states where people were more likely to respond in this way?

 

Phil [00:26:55] So, again, we have these background data by states over time as well. And spikes in gun sales were larger in some states than others following Sandy Hook. And so the states with the biggest increases were the ones where gun ownership rates were highest already. And this is consistent with concerns about having their guns taken away. If Sandy Hook led to successful gun control legislation.

 

Jennifer [00:27:20] And then how do you use that information to build upon your previous analyses and what do you find there?

 

Phil [00:27:28] I mean, so in essence, we're doing exactly the same thing at the state level with sort of this additional level of complexity, like the spike to spike analysis should not only be spike to spike, but spike to spike and bigger in the places where the gun sales increase was larger. And so we implemented that approach and again, find evidence completely consistent with our hypothesis that there's this potential relationship between the gun sales and the accidental deaths.

 

Jennifer [00:27:59] So this paper was published in Science in late 2017, somewhat recent, but there've been a few years in between. So what other research has come out since you wrote this study that contributes to our understanding of gun exposure and maybe gun violence more broadly?

 

Phil [00:28:16] So I don't know of any other work that specifically has addressed this question of accidental accidental deaths. But, you know, we do have additional evidence in terms of the gun sales and spikes themselves and what generates them in the first place. Unfortunately, we seem to periodically have these sorts of events for where, you know, mass shootings take place. And this notion that, you know, the reason why spikes occur in gun sales isn't because of the mass shooting, it's because of the fear of having- people having their guns taken away from them. That seems to be completely to completely line up with the evidence that we see in subsequent shootings. So San Bernardino was so the next major event where we see a spike in gun sales. That was the one that was motivated by terror- by terrorism. President Obama also at that time spoke about the need for gun control legislation and then the incorporation of terrorism also tends to have a fear inducing effect. So we see a pretty good sized spike in gun sales about at about that time, although only, you know, maybe not quite half the size or roughly half the size of, I'd say, of the Sandy Hook spike.

 

Phil [00:29:36] And the other event, which really generated sort of a noticeable spike is Parkland. And in that case, we are now in a political environment where I don't think there's much concern of national gun control legislation being enacted any time in the near future. But listening to the compelling stories of the Parkland kids and their, you know, heartfelt push to get gun control legislation enacted, at least at the state level, certainly seems to have had a similar sort of effect, although, again, even even smaller than both Sandy Hook and San Bernardino. So I think we know a lot more about patterns in gun sales and why they take on the, you know, why spikes occur when they do. The relationship between those those events and subsequent shootings, you know, we don't have as much knowledge about.

 

Jennifer [00:30:31] So putting it all together, the results of your study with Robin and the other studies we talked about, what are the policy implications of this work?

 

Phil [00:30:40] I think a direct policy implication is the desire to have stronger safe storage gun laws. You know, I think that the the direct effect of what generated the spike in in accidental accidental shootings is this notion that guns were out and loaded and accessible at times when people weren't paying attention. And policies to reduce the likelihood that that occurs seems like that has the potential to save lives. We could talk about sort of additional, you know, other forms of gun control legislation and things that may make more or less sense in different contexts, but this one, I think, is very direct.

 

Jennifer [00:31:29] Mm hmm. And then what's the research frontier? What are the big open questions in this space that you and others will be thinking about in the years ahead?

 

Phil [00:31:39] I mean, the- an important question that that Robin and I are just just at the very beginning stages of thinking about is sort of this notion of school shootings and what impact school shootings have. There was what started our interest in this, you know, aside from a general interest in gun exposure, was a Washington Post story—I think it was last summer or last spring or summer—that estimated the number of students who were exposed to a school shooting. So they were enrolled in a school on a day when when a shooting took place at their school. And the number is, you know, a couple hundred thousand kids. Seems like that has the potential to have some impacts on children's subsequent development and outcomes. I think that's a really important question to explore as we go further. So, you know, we're going to- what do we actually know about school shootings? I think that from from the limited work that we've done so far—we have you know, we've gotten started on this project, though we haven't gotten a lot done—it's sort of remarkable how little we actually know, despite the fact that, you know, they occur, again, far too frequently. When they do occur, they get a tremendous amount of national attention. But surprisingly, we know very little about them. And that seems like it needs to change.

 

Jennifer [00:33:11] My guest today has been Phil Levine from Wellesley College. Phil, thanks so much for talking with me.

 

Phil [00:33:16] Glad to be here.

 

Jennifer [00:33:23] You can find links to all the research we discussed today on our website, probablecausation.com. You can also subscribe to the show there or wherever you get your podcasts to make sure you don't miss a single episode. Big thanks to Emergent Ventures for supporting the show and thanks also to our Patreon subscribers. This show is listener supported, so if you enjoy the podcast, then please consider contributing via Patreon. You can find a link on our website. Our sound engineer is Caroline Hockenbury with production assistance from Elizabeth Pancotti. Our music is by Werner and our logo is designed by Carrie Throckmorton. Thanks for listening and I'll talk to you in two weeks.

Jennifer DoleacGuns, Difference-in-Differences