Probable Causation

View Original

Episode 69: Molly Schnell

Jennifer Doleac

Molly Schnell

Molly Schnell is an Assistant Professor of Economics at Northwestern University.

Date: March 15, 2022

A transcript of this episode is available here.

See this content in the original post

Episode Details:

In this episode, we discuss Prof. Schnell's work on how exposure to school shootings affects students' outcomes:

“Trauma at School: The Impacts of Shootings on Students’ Human Capital and Economic Outcomes” by Marika Cabral, Bokyung Kim, Maya Rossin-Slater, Molly Schnell, and Hannes Schwandt

OTHER RESEARCH WE DISCUSS IN THIS EPISODE:

TRANSCRIPT OF THIS EPISODE:

Jennifer [00:00:08] Hello and welcome to Probable Causation, a show about law, economics and crime. I'm your host, Jennifer Doleac of Texas A&M University, where I'm an economics professor and the director of the Justice Tech Lab. My guest this week is Molly Schnell. Molly is an assistant professor of economics at Northwestern University. Molly, welcome to the show.

 

Molly [00:00:26] Great. Thanks so much for the invitation. I'm really excited about this opportunity to talk about my work.

 

Jennifer [00:00:31] Today, we're going to talk about your research on how school shootings affect students. But before we get into that, could you tell us about your research, expertize and how you became interested in this topic?

 

Molly [00:00:41] Yeah. So I'm a health economist by training them and as you mentioned, I'm based at Northwestern, and I really think about the paper that I'm going to talk about today is part of a broader research agenda with some of my co-authors. So going back a bit in time, a couple of my co-authors and I, Maya Rossin-Slater, who's at Stanford and Hannes Schwandt at Northwestern. We started talking about the impact that school shootings might have on survivors. So sort of what we were thinking is that when one of these events happens, there's obviously a lot of attention on them by the media and policymakers and the public and they tend to, you know, really focus on the individuals with physical injuries and particularly the victims who who were killed. And, you know, we of course, think that, you know, these individuals should receive a lot of attention and resources, but we were really starting to think about, you know, whether or not these events could also have impacts on people who are exposed to the events and survive. And so, you know, Maya and Hannes had done a lot of really great work on on mental health, and I had done a lot of work on the prescribing practices of physicians.

 

Molly [00:01:39] And so we decided to team up to look at the impacts of these events on the mental health impacts of students and so basically, what we decided to do is to see whether or not prescriptions written by providers in the nearby area responded following one of these events or thinking of antidepressants prescriptions as an important indicator of mental health. And so, you know, that paper was published last year. It was also what Sam Trejo, who's now at at Princeton and Lindsey Uniat who is at Yale. And then we started working and talking with Marika Cabral and Bokyung Kim, who are both at UT Austin, and we were thinking about expanding this work to look at whether there are lasting impacts of exposure to these events on students educational and economic trajectories and here our thinking was really that, you know, this is a really first order question.

 

Molly [00:02:23] These events happen at schools, which you know, is the exact place where we think a lot of human capital accumulation takes place. And so there really, you know, is the scope for these events to have these lasting impacts for survivors. And so that's really what we were coming into this project thinking.

 

Jennifer [00:02:38] Yeah. So your paper is titled "Trauma at School: The Impacts of Shootings on Students, Human Capital and Economic Outcomes," as you have described itt's a team effort so it's co-authored with Marika Cabral, Bokyung Kim, Maya Rossin-Slater and Hannes Schwandt in this paper, you consider the effects of school shootings in Texas over the course of a couple of decades. We are all unfortunately used to seeing media coverage of horrific school shooting events. But what are typical school shootings like that is the kinds of shootings you'll consider in this paper. Are they like the ones we see on TV?

 

Molly [00:03:10] Yeah, I mean, that's a really great question and it's one of the first things we learned when when looking into the data that you know, the events that we that we see on TV are, you know, they're certainly there in far too common by any measure. But there is a lot of other gun violence that also happens on school grounds that often isn't covered and so what are these types of events? So this would be something like a student brings a gun to school and shoot someone, but there weren't any fatalities and so, you know, it's not going to be headline news. There are also cases where a student will bring a gun to school and commit suicide on school grounds. You know, again, that's often not reported on in the media and all of these events, you know, we think that they're different and but they, you know, they all really are similar in the sense that they have the potential to make school feel like an unsafe place for students and to disrupt their their educational trajectories.

 

Molly [00:03:57] I just want to note at this point that, you know, if people are interested in this, there's a there's a really great working paper by Phil Levine and Robin McKnight, which outlines the different types of school shootings. And they kind of create a categorization of how you can put these into different bins and show that different types of students tend to be exposed to different types of events. And so, you know, there's really a lot of gun violence that takes place at schools, and it doesn't always fit exactly into the bin of of what we think of based on what we see on TV.

 

Jennifer [00:04:21] So how common are school shootings in Texas in particular? And are there particular types of schools that are more likely to be exposed to shootings than others?

 

Molly [00:04:30] Yeah. So I mean, shootings at schools in Texas, you know, just like in the rest of the U.S. are, you know, unfortunately far too common. In terms of where they happen across Texas this is going to look very similar to if you look across the whole U.S. and that where two school shootings happen, they happen mainly in urban and suburban areas where there's a lot of students who are going to school. So in Texas, think of the events being, you know, predominantly concentrated in and in the outskirts of big cities so things like Dallas, Houston, San Antonio and just because of where these events are actually taking place, the schools tend to be larger, so they're going to have more students than the average public.

 

Molly [00:05:07] And they're also going to have a more diverse student body than the school system as a whole. And again, that's just because they're really located in these urban and suburban areas.

 

Jennifer [00:05:16] So you're going to measure the effects of these shootings on students education and employment outcomes. What are the various ways we might expect these events to affect those outcomes? What mechanism should we have in mind here?

 

Molly [00:05:28] Yeah. You know, there are a couple of ways in which one would expect something like exposure to a shooting at school to influence educational outcomes, you know, both in the short run, the longer run. And then also, you know what happens when students end up reaching the labor market. And so, you know, first related to the work that I was leading off with with Maya, Hannes, Sam and Lindsey, we know that school shootings have these detrimental effects on the mental health of exposed students and so, you know, there's a link between mental health and academic performance and labor market outcomes.

 

Molly [00:05:57] And so these mental health effects themselves could be leading to worse trajectories for these students, but I think also we think that shootings at school are going to be unique relative to other types of violence to which students could be exposed. So think of something like violence at home or in a student's community and the reason we think that this exposure might be different is that shootings that take place at school are happening in the exact location where a lot of human capital accumulation is taking place and so, you know, they, on the one hand, have the ability or the potential to make students feel unsafe at school, you know, and that could disrupt the learning process. But they might also affect things like school resourcing decisions and so the types or the number of staff that schools are hiring. They might also affect the ability of schools to retain the staff that they had and so it could lead to increased turnover churn among teachers, you know, and all of that could have effects on these, these outcomes for students and some of that we're actually going to be able to look at the data and we're going to see some evidence of this taking place so changes in sort of the hiring practices of schools and also in terms of, you know, their ability to retain teachers and the amount of turnover that they're experiencing.

 

Jennifer [00:07:00] And I'm realizing some listeners might be thinking, well, obviously the effects here are going to be negative. But you note in the paper, it's not totally obvious because you have the negative effects of the shootings initially, but kids are resilient, perhaps. And also, I guess the schools could be compensating in these ways. Are those the counteracting effects that you're trying to measure in this paper?

 

Molly [00:07:20] Yeah, exactly. I think I mean, there is a literature on, you know, the ability of children to basically bounce back from trauma. And if you think of these, you know, sort of single events that take place, it's not obvious that it's going to have these lasting impacts on students, right? You could also imagine looking in the data and seeing, you know, effects like in the week or the few weeks after, and then they sort of dissipate over time. And we're going to be looking up to a decade after the event and so at least it wasn't immediately obvious for me that this was really going to be sticking with students for that long. And then as you're saying, you know, schools, you know, we're going to be looking at actually how schools to some extent are responding to these events. And you know, you could imagine that they, you know, completely changed the resourcing, bringing a lot of social support and things that, you know, they're basically able to mitigate these what's actually happening to these students and I think our results are really going to speak to, you know, they won't be able to get at everything that schools are doing schools could certainly be doing things that we're not measuring in the data. But to the extent that we're finding these big negative effects, it suggests that, you know, schools aren't doing enough. And so I think there's a lot of scope to think about and to discuss both in the school system. But, you know, within communities of really what we can be doing to help mitigate these harms for survivors.

 

Jennifer [00:08:25] So you've already talked about this a little bit, but before this paper, what did we know about the effects of school shootings?

 

Molly [00:08:31] Yeah. So you know, before this work, it was a small but definitely growing literature on the impacts of school shootings on sort of a range of outcomes. So, you know, I had mentioned my work with with my Maya, Hannes, Sam and Lindsey looking at the mental health impacts. There's a really great concurrent working paper that I should mention by Phil Levine and Robin McKnight and there they're they're examining the subsequent mortality and educational acts of two of the most really the most horrific mass shooting events that took place on school grounds so looking in the aftermath of Columbine and Sandy Hook. There's also been, you know, other work looking at short run effects on educational outcomes in more aggregate data. So think of using like school level or district level data and I think what we're really going to be bringing to the table are adding to this literature is that we're going to have really, you know, unique rich individual level data that's going to allow us to, you know, follow the students who were actually exposed to these events over up to a decade after the event. And so we're going to be providing some of the first estimates of the longer run impacts on a comprehensive set of outcomes for these students.

 

Jennifer [00:09:30] And then what did we know about the effects of violence on students more broadly?

 

Molly [00:09:34] Yeah, I think that, you know, it's a really important question in terms of positioning our work in the literature in that, you know, school shootings are, of course, only one particular type of gun violence. And there was a really great an emerging literature in economics looking at the effects of exposure to gun violence more generally. So Desmond Ang has a wonderful paper that was published in the QJE in 2020, looking at the effects of exposure to police killings.

 

Molly [00:09:58] So what he's doing is he's using detailed data from Los Angeles and looking at how things like high school graduation and enrollment compare for student who were very near the police killing when it takes place versus those who are slightly further away, both before and after the event. And he's going to document really compelling evidence of negative effects of these police killings within the community on these educational outcomes for students. There's also a really fantastic paper by Prashant Bharadwaj, Manudeep Bhuller and Mirjam Wentzel, which it was recently published in the Journal Public Economics, looking at the aftermath of the horrific mass shooting in Norway and in 2011 and they're going to also document, you know, really detrimental effects on a wide range of outcomes for the survivors. And so, you know, this is just some of the work.

 

Molly [00:10:41] There's been really great work, you know, in other settings as well. So, you know, there's recent work looking at what happens when there is a homicide on a student's path to school in Brazil by Koppensteiner and Menezes and so you know what, we think our work is is really adding to this is that it's, you know, it's complementing this work and showing that events that are often less deadly, right, when we go back to thinking about the types of events that we're thinking about, but that take place in a school setting that we're showing that they can also have these really large human capital economic costs for for exposed students. And so I think it's important to not only take into account the severity of the event, but also the context in which they're taking place. And we're finding that that's very important in terms of these effects.

 

Jennifer [00:11:17] So what are the challenges in studying this topic? So as you and your coauthors were, first, you know, thinking about how to measure the long run effects of school shootings were the main hurdles mostly about getting the right data? Or was it mostly the identification challenges? Or is it both of those things?

 

Molly [00:11:35] Yeah. So I think it's definitely both on the data front, right? Whenever you're interested and looking at the effects up to a decade after an event, you're going to need data that allows you to actually follow these people over time and you know, we see a lot of that and more of a Scandinavian setting. But it's been difficult to get that type of data in a U.S. setting. And so I think that has been one of the limiting factors for this literature. But, you know, there's also an empirical difficulty here that a lot of applied micro researchers will be very familiar with that students who are exposed to shootings might be different than students who aren't and so we're going to have to think very seriously about what our control group is in the setting and spend a lot of time convincing both ourselves and and readers of the paper.

 

Molly [00:12:17] You know that we've done a good job in finding a comparable group of students so that we're going to be able to interpret these effects is actually the causal effect of the shooting and not something else. And, you know, I think it's important to keep in mind that this problem only becomes more pronounced when you start to look at something like up to a decade after the event because then it could be the case that, you know, very small differences in the backgrounds of these types of students could have these longer run effects and so we're going to have to think really seriously about that in this context. And so both on the data side, it's difficult to get the type of data to allow somebody to do this type of analysis. But then we're going to also have to think very carefully in terms of the empirical side of, you know, what we can actually do to get at causality in the setting.

 

Jennifer [00:12:54] So for research purposes, we would ideally like a randomized experiment where some students or communities are exposed to these events and others aren't for obvious reasons, a randomized experiment is impossible or desirable in the setting. So you and your coauthors use the timing of school shootings as a natural experiment that divides students into plausible treatment and comparison groups. So walk us through how you measure the causal effects of shootings on your outcomes of interest.

 

Molly [00:13:21] Yeah. So we're going to have two different empirical strategies that's going to depend on the type of outcome that we're looking at. So for outcomes that we can observe many time for the same student both before and after the event. So think of something like attendance of whether or not you're actually showing up at school. That's going to lend itself very naturally to a within student design where we can look at changes in this outcome following the event for an individual student. So I can see how my attendance changes after a school shooting relative to what my attendance was like before the event. We're going to go further than that because an analysis like that could be confounded by general time trends. Right.

 

Molly [00:13:58] So one thing we're going to see in the data is that as students get older, they tend to miss more school. And so when you're aging, as you go through that trajectory of high school, a school shooting happens at some point, you know, we're going to naturally see that absences after are greater than absences before. But that's just because students were getting older and wasn't something about the event itself and so we're going to compare these within student changes and outcomes following the event to analogous within student changes at matched control schools.

 

Molly [00:14:25] So as we were talking about at the top, these shootings aren't happening randomly across Texas. They're more likely to happen in these large urban and suburban schools and so we're going to have a matching design that's going to allow us to select control schools that look observationally equivalent to the schools at which shootings are happening. The only difference is that they didn't experience a shooting. And so our main empirical design for these shorter run outcomes and outcomes that we can observe many times for the same student before and after the event will be comparing these within  student changes in the outcome following the event to the analogous changes that are happening at these matched control schools.

 

Molly [00:15:02] And so that works really well for things like attendance or disciplinary action it's not going to work so well for things like whether or not you graduate from high school or what your earnings are like at ages twenty four to twenty six, because those types of outcomes we only observe once for each student and we only observe them after the event. And so we're going to have a complimentary design that's going to allow us to look at these longer run effects. And it's I think the intuition is very similar. It's just that the structure, it's going to be a little different. So since we can't do these within student changes in those outcomes, what we're going to do is a within school across cohort change. So think of students who are measuring an outcome like high school graduation among students who are in the school at the time when the shooting took place and we're going to compare that to that same outcome among students who went to those exact same schools, but who graduated five years before the event or who left the school five years before the event took place.

 

Molly [00:15:56] And so they're basically too old to be exposed to the event. And so we'll construct these within school across cohort changes to get a sense of the effects at that school. And then again, to net out general time trends we're going to compare those two analogous changes that are taking place at the matched control schools. And so we think that these two designs are going to help deal with a lot of the the concerns we would have with getting out causality in this setting.

 

Jennifer [00:16:19] And so both of these designs are what we call difference in difference designs and the underlying assumption in those kinds of designs is that your treatment and comparison groups exhibit what we call parallel trends. That is that the trend in the comparison group tells us what would have happened in the treatment group if, in this case, a shooting had not occurred there. So how do you identify your comparison group? Tell us a little bit more about that matching process and how do you and your coauthors convince yourselves that this parallel trends assumption holds?

 

Molly [00:16:47] So first, in terms of the matching procedure, so as I mentioned, if you just look at the schools that experience shootings in Texas and you compare them to the average public high school, so we're going to have data on public schools will be focusing on the public school system in schools in which a shooting take place look different on some observable characteristics then you know, just the average school, as I mentioned, they're larger, they're going to be more diverse. And that's again coming from the fact that they're in these urban and suburban areas. And so we want to basically find a way to, you know, in the data, pull out schools that look observationally equivalent to the schools that experience shootings with again, the only difference being that they didn't experience a shooting on school grounds.

 

Molly [00:17:24] So what we're going to use is called a nearest neighbor matching procedure, and so it will match each of the schools in our shooting sample to two control schools in the data and the way we do this as we first exclude schools that are in the same district. And the intuition there is that we were worried about spillovers, right? There might be a chance that if there was a shooting at a school right nearby to yours, that students at that school would be affected as well and so we don't want a control group that's also treated. And so we're going to first exclude schools that are in the same district.

 

Molly [00:17:53] We're then going to do what's called an exact match on grade levels and campus type. So the idea here is that if we're trying to match a high school, we're going to get rid of all non- high schools. We also have measures of campus type, which is basically just giving you a sense of the population and the proximity to an urban area. So zoom in on the set of schools that have the same grade levels, have the same campus type and then we'll do what's called a fuzzy match on a range of other variables. So we'll be matching on total enrollment and basically the socio demographic composition of those students. And what this allows us to do is basically pull out the schools in the data that look observationally basically equivalent to the schools that experienced shootings. They just didn't have a shooting happen at their school and we can see in the data that this works quite well. And that's actually one of the first things we're going to do going to the question of looking at parallel trends is, you know, we first need to make sure that that match actually worked and so will verify that the matching works and that we're going to see no differences in terms of observable characteristics between our treatment schools and these control schools.

 

Molly [00:18:52] And then to get it parallel trends, what else we're going to do is we're going to look at trends in the raw data. So for our outcomes, what we can do if you think of something like attendance is we can just plot average attendance over time in a school that experiences a shooting and overlay that with attendance over time in a school that didn't experience a school shooting, so our control schools and what we're going to see is that both the levels and the trends and these outcomes in the raw data are going to be, you know, sort of strikingly similar between the treatment and control schools. So that is going to already suggest to us that they're potentially parallel trends. We'll do this more formally also by putting it in event study design, where we'll be able to put some confidence intervals on that and see that there are no significant differences in any of these outcomes before the shooting in the schools that either had or didn't have a shooting. And that really helps reassure us that the schools that experienced shootings would have continued on in a similar path as the control schools in the absence of the event itself, so we can attribute any of those post shooting differences to the event that took place.

 

Jennifer [00:19:49] Yeah, I think in these kinds of designs, we're used to looking at those events, study style graphs that, you know, we're such a visual design. And so it's tougher to talk about an apology. But you you see these like beautiful flat differences before the treatment and then suddenly you see this change, which is, of course, what you guys find here. But I it was really striking to be able to see that even in the raw data, which is not always the case usually there's so much other stuff going on, but the fact that you see it even in the raw data, I think, is especially compelling. All right. Well, let's talk about that data. So you have two big data sets, I guess, here. So the first up? How do you create your list of relevant school shootings?

 

Molly [00:20:28] Yes. So to compile a list of school shootings, we're going to rely on two databases that we're going to bring together. So the first one is that the Center for Homeland Defense and Security, they maintain what they call a K through 12 school shooting database and this is going to just think of it as including all events where a gun is on school grounds and fires since 1970. And so it's, you know, this really comprehensive set of guns on school campuses over time. We're going to add to that information from The Washington Post has done a really fantastic job of tracking down a lot of information about all school shootings that have took place since the shooting at Columbine. And so we're going to combine those data sets together to get the sort of most comprehensive look that we can get at the events that have taken place over this time. That's going to leave us with a lot of school shootings, and we're going to only be ended up looking at outcomes in the Texas public school system. So we're going to end up zooming in on the events that took place in Texas public schools over our time window. The other restriction that we're going to make is that, you know, we were interested in the effects of these events on individuals or students who we know are on school grounds. We think we're on school grounds when the event occurred.

 

Molly [00:21:32] And so just that the sort of simple way to do that in the data is that we're going to exclude school shootings that took place after school hours, right when we think a lot of students might not be present. We're also going to exclude ones that happened during the summer so students might not be around. And so we're really zooming in on the shootings that took place in Texas public schools during school hours. And then the final restriction will be to basically leave enough post period for us to look at our outcomes. So for these shorter run outcomes like attendance, we don't need a lot of post period window, right. We can look just a couple of years after the event to see what's happening with attendance for things like earnings at ages 24 to 26 or whether or not you go to college, whether or not you graduate from college.

 

Molly [00:22:12] We're going to need to have a lot of data after the event in order to follow these students and so that's going to narrow the set of shootings that we're looking at there, where they're going to be shootings that took place in our short run analysis. They'll be over two decades so 1995 to 2016 and our longer run analysis will be focusing on shootings from 1998 to 2006. So again, to allow for that longer post period window to follow students over time.

 

Jennifer [00:22:33] OK, so then you're going to link those data on the shootings with amazing longitudinal micro data on Texas students. So tell us about that data set.

 

Molly [00:22:43] Yeah. So this data is really amazing. I had seen other people work with it in the past, but this was the first opportunity that I had to get involved with it.

 

Molly [00:22:51] And so what's going to be great is that Texas is going to have a lot of information about students who go to their schools and they're going to have sort of three different data sets that then are going to speak to each other. And so you're going to have a student ID that you can link them across these different datasets. So what are these datasets sets. So first, the Texas Education Agency maintains individual level school records for all K through 12 public school students. So think of this as sort of any information that the school is going to know abou the student is going to be in that data, so we'll have things like attendance, graduation and disciplinary actions. We'll also know a lot about the students themselves so  think age, gender, race, ethnicity, language spoken at home, receipt of free and reduced price, lunch, special education, and so on and so forth.

 

Molly [00:23:31] And those are a lot of the characteristics that we're going to be using about the schools in order to make that match in the first step. So that dataset sort of everything that Texas would know about students who go to their public school system and get grades K through 12, the Texas Higher Education Coordinating Board maintains a database that's going to be somewhat similar, but it's going to be individual level records for all public colleges and universities in Texas. So then if you go on to the Texas Public University System, they're also going to have a lot of information about you and again, the really unique thing in the setting is going to be that there's an individual I.D. that allows these two datasets to speak. And so you can link them to follow students from kindergarten through college, and it's not going to end there.

 

Molly [00:24:11] The Texas Workforce Commission maintains a database of individual level quarterly data on employment and earnings for anyone working in the state of Texas. It's the sample basically of anyone who was working for a company that reports to the unemployment insurance program, and so it's going to be really comprehensive data on people working in the state of Texas. And that also is going to have at individual level I.D. that's going to link back to the educational databases and so will be able to follow these individual students over the course of decades in the data.

 

Jennifer [00:24:38] And so people might be thinking, you're going to lose people if they leave Texas later, you will be able to observe if they go to college somewhere else or work somewhere else. But I imagine that Texas is the kind of state where fewer people leave the state than in other places. Is that your sense as well?

 

Molly [00:24:54] Yes. So that's definitely the case that people who are born and raised in Texas are seem to be very likely to actually stay in Texas and so attrition might not be as much of a problem in this setting as it is in other settings. But this is also something that we can look at in particular, and it's going to be really important once we get to something like the long run analysis right there when we're leveraging these across cohort designs. If you had that, everyone who say all the very high achieving students, you know, immediately left the school or left the state of Texas following the event, then you know, we're comparing these cohorts that just are, you know, apples to oranges. And so we could get, you know, results that aren't reflective of the causal effect of school shootings.

 

Molly [00:25:32] And we can actually look at a lot of this in our data and so on. One thing we can do is we can, you know, rerun basically our short run analysis, but on an indicator for whether or not the student still appears in the Texas data and so the nice thing about our data is they can move across schools in Texas, right? You can move from one public school to another public school, and we're still going to see you in the data and we're going to follow you and you're going to be included in our analysis. It only becomes a problem sort of once you leave that data set. But reassuringly, for our purposes, we're we're going to see no evidence of differential attrition between our treatment and our control schools and so we are going to see that there is just some natural churn that some students are going to start leaving the dataset, but it's going to be at an equivalent rate between our treatment and our control schools.

 

Molly [00:26:12] And so, you know, that brings a lot of confidence that it's not the event itself that's leading these students to be leaving the schools and so that ends up just not being sort of as relevant a concern in the Texas setting as it might be in other locations.

 

Jennifer [00:26:24] Great. OK, so tell us a bit more about your sample. What do the students look like in the treatment and comparison schools?

 

Molly [00:26:33] Yes. So you know, as I mentioned in terms of where these shootings are taking place is they're taking place in large urban suburban schools. And so these are going to be schools that are, you know, are large and are relatively more diverse than the average public school in Texas.

 

Molly [00:26:47] And so you can think of that as being kind of our, you know, our sample of schools as these large urban suburban, relatively diverse high schools will have both high schools and non high schools in the data, but that's, you know, sort of that the image you should have in mind of the type of school that we're considering and the matching procedure is going to work really well and so are control schools are going to observationally look very similar. So we're going to again be choosing these large urban suburban schools that are large and have a relatively diverse student population.

 

Jennifer [00:27:11] And what outcome measures are you most interested in?

 

Molly [00:27:13] Yeah. So I think our outcome measures could be divided into three different bins. So, you know, this is aligning with our empirical design that I outlined. So first, we're going to have what we're going to call short run measures, and these are going to be outcomes that are measured annually both before and after the event and so we can see them many times for the same student. And so here, think of things like an absent rate. We're going to define this as we can see in the data that the days absent relative to the days enrolled within that school. We're also going to construct from that absence rate an indicator for chronic absenteeism, which is commonly defined as an absence rate of greater than 10 percent. So the reason for looking at both continuous absences and an indicator for chronic absenteeism is that, you know, it's not clear that we think that missing one additional day of school should really have these detrimental effects over the longer run what sort of people have arrived in the education literature is that, you know, once you're missing a lot of school, that's when there's the scope to have these longer run effects.

 

Molly [00:28:07] And so we're really interested in whether or not these events move around chronic absenteeism and are inducing students to start missing a lot of school. We'll also look at an indicator for grade repetition. So whether you're being held back and need to repeat a grade and we'll look at things like days of disciplinary action. So think of suspensions and expulsions and other behavioral interventions.

 

Molly [00:28:25] Over the longer runs for our second bin of outcomes would be what we'll call our Long-Run outcomes, and these are the outcomes that are only measured after the event for a given student. So for educational outcomes, we're going to measure these at age 26 and we're going to say by age 26. Did you graduate from high school? Did you enroll in college? Did you enroll in a four-year college and did you ultimately receive a B.A? In terms of labor market outcomes we're going to measure these at ages twenty four to twenty six for students, and we're going to see over that time period. Were you ever employed? What are your average earnings and then what are your average non-zero earnings or earnings conditional on employment?

 

Molly [00:28:58] Our final bin of outcomes is going to be turning to looking at a school member. So one of the sort of key mechanisms that we had in mind that might be underlying these effects is that violence that happens on school grounds can really disrupt a lot of stuff that happens within the classroom. And so we're going to leverage the fact that we have a lot of this information in the same dataset that we know about what's happening on students from from Texas. And so here we're going to be looking at two types of outcomes. So one is total full time equivalent employment per a thousand students will be looking at this across different types of employment within the school.

 

Molly [00:29:32] So we'll look at teachers will look at teaching support, social support, so things like counselors and school psychologists and we'll also look at school leadership, principals and assistant principals. And then that, you know, gives you a sense of sort of the aggregate resourcing decisions in the school. We were also really interested in this idea of the potential for increased churn that maybe, you know, maybe you have to keep hiring teachers because you need people to be there because the students are there. But what happens to the people who are there when the event happened? And so we'll look at basically holding the sample of people fixed of who was at the school in the year of the shooting and look at the probability that stay in that school over time and again, we'll look at that separately for teacher is teaching support staff, social support staff and school leadership as well.

 

Jennifer [00:30:13] All right, let's talk about the results. What do you find is the effect of exposure to a school shooting on students education outcomes?

 

Molly [00:30:21] Yeah. So in the short run, we're going to find that exposure to a school shooting is going to lead to increased absenteeism and grade repetition. These effects are going to be pretty large. So for increased absenteeism, we're going to see a 12 percent increase in just a measure of continuous absences. When we turn to looking at that indicator for chronic absenteeism, so whether or not a student is chronically absent, we're going to see an increase of 28 percent relative to the baseline rate and so pretty large effects in terms of the chances that students are not only missing school but missing a lot of school. The effects on grade repetition are also going to be large, so it's going to be about a one and a half percentage point increase in grade repetition.

 

Molly [00:30:56] And there is, you know, luckily not a lot of grade repetition within the Texas public school system and so this effect is going to be really large in percent terms relative to the baseline mean.

 

Jennifer [00:31:05] And then what are the longer run effects on things like college going and employment?

 

Molly [00:31:09] Yeah. So over the medium term, so we're going to see reduced high school graduation, college enrollment and college completion. Again, these effects are going to be quite large. So in terms of high school graduation, we're going to see about a four percent reduction in the chance that a students actually graduating from high school. We're going to see a reduction in any college enrollment of 10 percent, a larger percent effect on four year college enrollments or a reduction of over 17 percent and then a 15 percent reduction in the chance these students ultimately receive bachelor degrees. And so this doesn't seem to be something that's only affecting them in the short run in terms of their absenteeism and grade repetition.

 

Molly [00:31:44] It's really following them through through a lot of their educational trajectory. And then, you know, turning what happens to them when they enter the labor market we're going to also find these sizable effects that are going to follow these students. And so we're going to see reduced employment and earnings at ages twenty four to twenty six. The magnitude of these effects are going to be for employment. It's going be about a 6 percent reduction in the chance that these students are employed at all over that time period and even for students that are employed, it's going to lead to a pretty big reduction in their earnings. And so a reduction in annual earnings of about three thousand dollars, or about thirteen point five percent relative to the baseline mean.

 

Jennifer [00:32:18] Do these effects vary at all across either groups of students or types of shootings?

 

Molly [00:32:23] Yeah, that's a really great question, and I think something that we were very interested in and using the unique nature of our data.

 

Molly [00:32:29] So a one great thing about when you have this individual level data is that we can actually, you know, zoom in on different subpopulations of students and see, are these effects more pronounced among students from from different types of backgrounds? And I think, you know, going into this analysis that I don't think the hypothesis was entirely clear and it really depended on, you know, who we were talking to. So some people would say, oh, you know, I think the effects are going to be particularly pronounced for students from affluent backgrounds because they might be less likely to be exposed to violence outside of a school setting. And so, you know, this individual event might be more jarring and have more negative impacts for them.

 

Molly [00:33:03] On the flip side, other people had pointed out that if you come from a more disadvantaged background, you might not have access to the type of resources that are going to help mitigate the harm from these events. And so, you know, I think you could sort of tell a story that it could go either way. It was ultimately an empirical question and I think that the really striking thing and I think in looking at these heterogeneity analyzes is that we're seeing that all types of students are affected, that there really doesn't seem to be any student population that isn't suffering from the negative impacts of these events. So we see that male students, female students, students that qualify for free and reduced price lunch students from different backgrounds that they're sort of all showing the negative impacts of exposure to these events, which I think you know for us was really striking, and that it suggests that the negative impacts of exposure to gun violence in a school setting are relatively universal and there doesn't really seem to be a type of student group that's that's really protected from these harms.

 

Molly [00:33:52] We can also, as you mentioned, look at heterogeneity not only by student characteristics, but also by the type of shootings. So I had mentioned that Levine and McKnight developed these categorizations where they, you know, group different school shootings into two different types. So think of things like personally targeted or suicides or crime related and you know, we can rerun our analysis on those different sets of shootings. This analysis is a little bit harder in our setting because we're starting to get into small sample sizes where we don't have that many events within each of those bins. And so, you know, this is subject to a lot of statistical caveats in terms of, you know, interpreting a difference this year. But I think sort of the main thing that struck me about these analyzes is that sort of all of the events really look like they're contributing to our negative effects. And so, you know, for some outcomes, one might look like it's one type of shooting might look like it's kind of jumping out more than the other, but taking the picture as a whole, it seems like all of these types of events are really contributing to the negative effects that we find.

 

Jennifer [00:34:45] And then you also consider whether baseline access to school resources like psychologists or counselors might change the effect of school shootings. So why do you do this and what do you find?

 

Molly [00:34:57] Yeah. So you know, the reason we did this, I think, was to start to move more in a policy direction of thinking, you know, are there types of resources that could help mitigate these harms, you know? To the extent that these events are going to happen, you know, is there anything that schools could really do to help mitigate these impacts for students. You know, we do not have exogenous variation in any of these school resources.

 

Molly [00:35:15] And so, you know, this should be viewed through a lens of a descriptive analysis and that, you know, these aren't the causal effects of these different types of resources across different schools. But again, we're going to see something very similar to the earlier heterogeneity analyzes where sort of none of these features at the baseline seemed to be really mitigating these effects for students in these schools. I think, at least for me, the more striking thing about doing that analysis was just how rare a lot of these resources are within schools and so, you know, our idea to do this empirically was to split schools by whether they have an above or below median provision of a certain type of resource within their schools and for things like school psychologists and counselors, we actually had that only, you know, a few of our schools in our sample had any positive full time equivalent of these types of providers within the school system.

 

Molly [00:36:00] And so, you know, it's above a local median divide just became did you have any, you know, which might be a very low FTE versus none. And so it seems like there's still a lot of scope to examine whether or not these resources do have positive impacts in these settings and you know, I think that's a really promising area for future research.

 

Jennifer [00:36:16] Yeah. So one way that school shootings might affect students is by changing the school environment, as you've mentioned a couple of times. So, for instance, you could change who works there. So what effect does shootings have on the employment and retention of various school staff?

 

Molly [00:36:31] Yeah, so what we do here, this is going to be an analysis that you should think of is very similar to our short run design.

 

Molly [00:36:36] And so we basically can just aggregate up to the school level and what happens to the total number of teachers per 1000 students, you know, before and after the event, again looking relative to the analogous outcomes in our in our control schools. And what we're going to see when we first look at employment is that we're not going to really see any changes. We're going to get very precise zeros in terms of the number of teachers per students that are on ground on these schools. And I think, you know, that makes sense. We, you know, first we looked at students and we weren't seeing big effects on enrollment and so if you still have a lot of students around you, you need people to teach these classes right? And so they're we're seeing that the number of teachers for students is being held relatively constant.

 

Molly [00:37:15] We see the same thing for teaching support. And so other teaching aides that might be in the classroom. We're not seeing any effects of the shootings on those aggregate resources. And also on social support staff. I think that was one that we were particularly interested in that as I just mentioned with that heterogeneity exercise that, you know, a lot of these resources were very uncommon in a Texas public school setting. And so you might have imagined that following an event that they start to hire more of these resources. Do they bring in a school psychologist or people like that in order to to help with this on the ground within that school? And we actually received no effects on aggregate social support services within that school in terms of employing more people in those positions. We do find effects, though, on school leadership staff per thousand pupils within the school.

 

Molly [00:37:56] And when we look into the data, what this is really being driven by is it's being driven by schools hiring additional assistant principals. And so what assistant principals tend to do in a U.S. context is that they help with a lot of disciplinary actions within the school system and so that seems to be one way that schools are responding to these events are basically bringing on more administration to help deal with disciplinary actions within the schools. So, you know, we see these effects on some categories of aggregate resources. We then turn to looking at effects on turnover and I think, you know, this was something that we were really interested in, of not only our students on school grounds and they might be exposed to these events, but you also have teachers and support staff and people who are on school grounds and, you know, could also be influenced by these events.

 

Molly [00:38:37] And what we're seeing is that teachers who were working in the school in the year before the event that they're much less likely to stay with that school over time. So again, there's going to be natural churn the same way there is with students, with some teachers leaving, you know, each year from any given school, but we're going to see no significant differences in terms of churn among students, but we are going to see this increasing trend among teachers that teachers are going to be much more likely to leave our treatment schools relative to our control schools following the event and so we think this is something that could also be contributing to the effects is this just additional disruption in the classroom when you have new teachers coming in and, you know, becoming familiar with that environment and working with these new students.

 

Jennifer [00:39:14] You use all these estimates to consider the social cost of school shootings in terms of the reduced lifetime earnings of the students who lived through them. So what do you find there and how does the effect you measure compare with estimates for related work?

 

Molly [00:39:29] Yeah. So one thing that we can do is we can basically just take the effects that we observe up through age 26 and say, you know what, if these effects persisted throughout the rest of the individual's life and so we can just project them out to age 65 and then discount them back till today. And they're we're going to find really sizable effects. So this is going to be about one hundred fifteen thousand dollars in terms of the reduction and present discounted value of lifetime earnings per exposed student. Now, of course, that calculation is subject to a lot of caveats. It's not clear whether these effects will persist over time. Right.

 

Molly [00:40:00] You could say that now you know, students are going to start getting additional resources or over time, these effects might, might dissipate. I think on the flip side of that, you could also say that they're going to grow over time, but if you enter the labor market at, you know, in in a position, if you're less likely to be employed and you're earning less than that, could you know that's been shown to affect your trajectory throughout the entire course of your working life. And so you could kind of go either way, but I think this is a good way to just get a sense of sort of the overall magnitudes that this could be something on the order of over $100000 per student throughout the course of their life relative to previous work that's been done. I think it's helpful to benchmark our study to some of the work that I mentioned before on exposure to gun violence more generally.

 

Molly [00:40:39] So for example, going back to the paper by Desmond Ang looking at police killings, he finds that students who are nearby a police killing are going to be about three point five percent less likely to graduate high school and two point five percent less likely to enroll in college. So we're going to find, you know, very similar effects on high school graduation and much more pronounced effects on on college enrollment. And I think this, you know again, points to the idea that violence at school potentially, you know, has the ability to be more disruptive than gun violence in the in the local community. We can also benchmark our results to the paper that I had mentioned by Bharadwaj and coauthors looking at the 2011 mass shooting in Norway. And they're they're finding that survivors are 12 percent less likely to graduate college and earn about 12 percent less.

 

Molly [00:41:21] Our effects are going to actually be of a very similar magnitude, which I think you know at first might be surprising because the events in Norway was obviously much more severe than the events that we're considering here. I think it makes sense once we think about the context in which these events were happening and to the individuals that were exposed, and so in the Norwegian setting, these were relatively affluent students in a country with a very broad social safety net, whereas in the Texas school system, in the Texas public school system, we have more disadvantaged students and sort of a less pronounced social safety net and so that could explain some of these differences in terms of the results. Lastly, I think, you know, it's also helpful to benchmark our results to work by Steve Grell and coauthors looking at the effects of a peer experiencing domestic violence at home and how that influences other students in the classroom.

 

Molly [00:42:03] And there they find a three percent reduction in earnings at ages twenty four to twenty eight. So to sort of position our results relative to this, this would mean that our effects are on the same order of magnitude of about four and a half additional violence exposed peers. You know, going back to mechanisms, I think this is important to keep in mind is that we know peer effects are really important in a classroom setting and so when we think of something like a school level shock, anything that's happening at the individual level could just be amplified by the experiences of their peers as well. And so I think that really helps explain sort of why we're getting the sizable effects that that we're finding in this context.

 

Jennifer [00:42:38] OK, so that is all your paper. Are there any other papers related to this topic that have come out since y'all first started working on this study?

 

Molly [00:42:47] Yeah, I think that's been the really exciting thing about working in this space is that there has been a lot of recent attention of looking at school shootings and taking a really wide lens in terms of what we think of as the costs of these events. And so, you know, it's been great work looking at what happens in the community. So think of things like housing prices or mental health more generally in the community. There's also been a lot more work at the individual level effects of what happens to birth outcomes and fertility decisions. And so, you know, I think all of this it's been a really exciting literature to be a part of in that it's growing and moving very quickly. And I think we're sort of all recognizing that these events have the ability to influence a wide range of outcomes, both within the community and at the individual level and so when we think of the costs of these events, we need to be taking this very wide lens.

 

Jennifer [00:43:32] So what are the policy implications of the results from your paper and the other work in this area? What should policymakers and practitioners take away from all this?

 

Molly [00:43:40] The first thing to take away is that prevention is important, right. So the first best would be that these events just don't happen to begin with, you know, and that would get rid of these negative effects that we're finding. And so I think, you know, taking these additional costs of these events and making that part of the policy discussion and recognizing that we do have some solutions and things that we know really help with this in terms of gun control and the availability of guns within the community and to into the perpetrators of these events. And so one, I think it really puts an emphasis on prevention and that that is the first order thing to do from a policy perspective, but to the extent that these events are going to continue to happen, at least in the short run, I think then that leads to the idea of, you know, what can we really be doing on the ground in order to help mitigate the negative effects for these students?

 

Molly [00:44:24] And so I think there's a lot of interesting work that could still be done in terms of looking at how schools, parents, communities can respond to recognize that these events are going to have potentially these lasting impacts for survivors. And so trying to find ways to intervene with policy and resources in order to mitigate these effects for exposed students.

 

Jennifer [00:44:42] And what's the research frontier? What are the next big questions in this area that you and others will be thinking about going forward?

 

Molly [00:44:49] So as I, you know, I just mentioned, I think there is a lot more that could be done on mitigation. So, you know, what can we actually do if you're a school and you know you are in the terrible position that something like this happens? You know, what can you do to help the students, the teachers, everyone who is there, you know, continue to feel safe at school and thrive and to help prevent some of these effects from manifesting and so I think, you know, identifying those strategies, I think, is very important. It's, you know, you can't solve a problem that people don't know exists and so we see this work as sort of identifying these effects and showing that, you know, even if schools are doing something to the extent that we're still finding effects, it suggests they're, you know, they're really not doing enough. And so I think this is really just a starting point and a springboard for future work to see ways that we can help mitigate these effects. I also think related to the discussion of the recent literature in this space.

 

Molly [00:45:34] I think there's a lot of other outcomes that people can can be looking at, right. Once we recognize that school shootings have the potential or are impacting the long run educational and economic outcomes of these students that opens the door to these events affecting many other aspects of their life and I think there's a lot of there's a lot more important work to contribute to that area in terms of quantifying and identifying exactly how these events are affecting communities and students lives. I guess related to that, I also think, you know, when we're going back to when we were talking about the types of events that we're considering, that we're, you know, our samples, including these more severe events where there's fatalities, less severe events where people may have been physically injured, but luckily nobody was killed.

 

Molly [00:46:14] And so, you know, we're sort of thinking this is more comparable to other types of violence to which students could be, you know, commonly exposed at school. But I think, you know, looking at these other types of violence, which students are exposed at school and looking at the effects of those as well. So if we think of things like bullying, either, you know, on school grounds or cyber bullying and all these different types of things that could disrupt the learning process for students. I think there's a lot of really important work that could be done there. You know, this all comes from the perspective of if we're interested in improving children's lives through the school system. You know, we spent a lot of time looking at sort of classic school based inputs, and we're now sort of coming around to recognize that other things happen at school at the same time that we might not have intended to happen, but that's going to really interact with a lot of the things that we're doing on the school side.

 

Molly [00:46:58] And so, you know, looking at other types of violence and other traumatic shocks that students are exposed to it at school and measuring those effects and so that one we know whether and if that affects exis and also then turning back to mitigation of ways that we can help students succeed and thrive in our classrooms.

 

Jennifer [00:47:14] My guest today has been Molly Schnell from Northwestern University. Molly, thanks so much for talking with me.

 

Molly [00:47:20] Great. Thanks again for the invitation. This was really fun.

 

Jennifer [00:47:28] You can find links to all the research we discuss today on our website probablecausation.com. You can also subscribe to the show there or wherever you get your podcasts to make sure you don't miss a single episode. Big thanks to Emergent Ventures for supporting the show, and thanks also to our Patreon subscriberrs nd other contributors. Probable causation is produced by Doleac Initiatives, a 501(c)(3), so all contributions are tax deductible. If you enjoy the podcast, please consider supporting us via Patreon or with a one time donation on our website. Please also consider leaving us a rating and review on Apple Podcasts. This helps others find the show, which we very much appreciate. Our sound engineer is Jon Keur with production assistants from Nefertari Elshiekh. Our music is by Werner, and our logo was designed by Carie Throckmorton. Thanks for listening, and I'll talk to you in two weeks.