Probable Causation

View Original

Episode 16: Stephen Billings

Jennifer Doleac

Stephen Billings

Stephen Billings is an Associate Professor of Real Estate at the University of Colorado at Boulder.

Date: November 12, 2019

Bonus segment on Professor Billings’ career path and life as a researcher.

A transcript of this episode is available here.

See this content in the original post

Episode Details:

In this episode, we discuss Professor Billings' work on how interventions to address lead exposure affects kids' criminal behavior:

"Life after Lead: Effects of Early Interventions for Children Exposed to Lead" by Stephen B. Billings and Kevin T. Schnepel.

OTHER RESEARCH WE DISCUSS IN THIS EPISODE:

Transcript of this episode:

 

Jennifer [00:00:07] Hello and welcome to Probable Causation, a show about law, economics, and crime. I'm your host, Jennifer Doleac of Texas A&M University, where I'm an Economics Professor and the Director of the Justice Tech Lab.

 

Jennifer [00:00:17] My guest this week is Stephen Billings. Steve is an Associate Professor of Real Estate at the University of Colorado at Boulder. Steve, welcome to the show.

 

Stephen [00:00:26] Thanks, Jen. Thanks so much for having me. Big fan of podcasts, and it's an honor to be on here.

 

Jennifer [00:00:31] Well, thanks. It's good to have you. So we're going to talk today about the effects of lead on criminal behavior. Lots of people have asked me to have a show on leading crime. So this is it. Very exciting. And even more importantly, your paper looks at how to mitigate the negative effects of lead exposure, which I also really like. But before we dive into that, could you tell us about your research expertise and how you became interested in this topic?

 

Stephen [00:00:56] Absolutely. So I would say of myself is I'm a pretty eclectic researcher. So you look at my bio, you'll see that it says I work at the intersection of housing, schools, neighborhoods, and crime. And even for this study, I would say you could add environment. So I kind of care about lots of different things and how they work together. I think I really tend to focus on neighborhoods and the importance of place in our lives. And so the question to me is, how does where we grew up matter for long term outcomes? I'm going to make kind of a more of a movie reference here. But what if you really think about some of my research inspiration it comes back to the movie Trading Places. So they put – in Trading Places maybe I'm a little bit older than some of the audience and you don't remember this movie, but it's about this wealthy, privileged guy, Dan Aykroyd in the movie. They take away everything from him and put him in prison. They take away his resources, his housing and social support, everything, and they do the exact reverse to the kind of down and out Eddie Murphy, who also has a life of privilege. And so I think it's such a telling story about how much does where the environment you  grew up in matter for how people succeed and shows in the movie, of course, they completely reverse their outcomes by based on being put in these different environments. But I really wonder how much we could do that as a society, how much we can do that in terms of policy. And I think that's like always in the back of my mind when I think about this a lot.

 

Stephen [00:02:19] Today that's a project we're going to talk about, though, is this paper Life after Lead, which is kind of the effects of early interventions for children exposed to lead. And I guess one of the things you kind of want to know is where did – how did I become interested in this? Where did this all start? And it was really back in 2013 when I reconnected with a former rival in ultimate frisbee and previous guest on this podcast, Kevin Schnepel, and we started talking about research. So we're old friends in other contexts, but decided, hey, we could do some fun research together. He was really interested in this topic. He was talking all about all this media coverage of lead exposure and kids, and that what a substantial issue that was in the US. I was fairly naive, and I didn't realize that this was still a problem.

 

Stephen [00:03:06] We know lead was bad. Why would we still have problems with kids exposed to something that's a known toxin? And if you look at all this kind of information, you start to see that there's this really strong negative correlation between kids exposed to lead in all kinds of short and long term outcomes in education, health, behavioral. And so it's really an important question if we can say how much of this is due to lead and if we remove this lead exposure and really limit this problem, can we really have a big change on society? And so, I mean, one of the stats you can see is that there's over a million kids that have had lead paint exposure in the U.S. I mean, not a trivial amount, tons of kids. And so we started really immersing ourselves in the policies and the data in this Life after Lead project that we're going to discuss in great detail today is really the first of three projects in the space, and they're really the only one we've completed. We've completed two of them and we're still working on the third one.

 

Jennifer [00:04:03] Great. So, yes, your paper is titled, as you said, "Life after Lead: Effects of Early Interventions for Children Exposed to Lead," coauthored with Kevin Schnepel. So let's talk more about lead. So you just gave us one stat about paint lead exposure anyway. Do we know how common lead exposure more broadly is and how are people typically exposed to this toxin beyond paint?

 

Stephen [00:04:24] Right. So lead based paint is probably the biggest source of exposure, but we also used to have lead gasoline. And so the lead that we used to have and gasoline that we used to use in our cars really back before the mid-70s when they got rid of this is still around. I mean, lead's a heavy metal. It kind of just doesn't go away no matter what you do. It's something that's going to stick around for a long time. So it's in the soil. It's still in different elements of kind of society, near highways especially we'll find a lot more lead in the soil. But if you look at some of the stats, I think one thing to note is that lead exposure really has gone down over time. We've done a lot to address this. So here's a good stat for you. In 1984, 17 percent of all preschool children had blood lead levels that exceeded what's the threshold here, 15 micrograms per deciliter.  Anything above 10 is considered quite harmful. We'll talk more about even where maybe lower levels are probably also quite harmful. But back then, 17 percent. I mean, that's an astronomical number if you think about it at a really harmful level of lead.

 

Stephen [00:05:33] And then we move forward in time up to 2014 well after we saw all this removal, the banning of lead paint in 1978, the phase out leaded gasoline in the 70s. 2014, we see only half a percent of children with levels of 10 or greater. So a tremendous change in that story, but the problem is we still – the fact that we still have half a percent of children that are testing at the levels above, that is still a problem, right? It's still something that we know is bad. We should be able to do better to deal with this. In terms of how lead affects us or our bodies you know, I'm not a medical person, a public health person, but it's a very well established literature. And in these areas that says lead is really dangerous to children because they're growing bodies absorb more lead than adults. So children are really the kind of the cohorts that are affected and it really affects your brain and nervous system. So it's something that goes right to your brain and it affects your nervous system. Of course, that's going to lead to lots of problems in terms of – and we'll talk a little bit, I think, in a little bit to talk about some of the how it actually affects the biological neurological effects. But another issue is that babies, young kids, they tend to touch everything and put everything in their mouths. So they're primarily getting lead through dust and soil. So both to the lead paint or even lead gasoline, people think that they, you know, kids touch it, they put in their mouth and it causes them to get pretty high levels of exposure from those types of things.

 

Stephen [00:07:05] There's also some other sources which people may be less aware about, they're a lot smaller, so I don't really focus much on that. But there is lead in things like dishes and glasses. If you go to your like grandma's old glassware, you'll see that there is actually some probably still has lead in it. And so you could get elements from that type of, from those types of things like dishes and glasses as they wear down over time or even there – for a while we had a lot of problem with toys that had leaded paint, too. And so that's been less so, but still can be an issue that they're sticking around people's homes. Adults can also get exposure to lead it just tends to be not as clear or not as high or kind of as direct an exposure, meaning that it really has more problems in your brain developing when you're younger, it tends to have a lot more problems and you also don't as an adult you don't eat dirt as much. So that's one way we're not going to get quite the exposure. Right. So it's a lot harder to understand how adults are being exposed. But there is people – adults do get or do have lead exposure. So this is happening still.

 

Jennifer [00:08:12] So, yeah, let's talk about the biological and neurological effects from the science literature. My sense as someone who has not paid super close attention to this, but have been sort of fascinated with the econ literature more, it does seem like the science has really advanced also in the last decade or so, which is really fueled a lot of this interest. So give us your sense of the latest from the scientific literature in this space.

 

Stephen [00:08:34] Yeah. So, you know, they call it a systematic toxin. That means it affects all of your organs. But the main effect is going to be through your central nervous system. And especially, like I said before, with kids, the developing brain, it's going to lead to lots of neurological disorders, brain damage, retardation, behavioral problems, nerve damage, Alzheimer's, Parkinson's, schizophrenia, lots of really bad neurological problems. The one thing and that, you know, as you get higher levels exposure, there's going to be more it's going to be more acute. It's going to be more apparent. But lower levels of exposures, we still believe there's lots of effects going on. They just take longer to kind of be symptomatic. And so often lead targets, the prefrontal part of the brain and people, you know as much as I know about the brain, but there is this element of the prefrontal brain which is involved in things that are very important for behavior. So if your prefrontal brain has problems, you're going to have really bad impulsivity, attention issues, hyperactivity. These are all things that are quite a problem, of course, in being a good student, doing well in education. But one of the big problems we see in is often linked to lead exposure is things like crime, deviant behavior, anti-social behavior more broadly. And that's where a lot of you see and we'll talk more about this later seems to be a lot of the effects tend to show through that area. And so, you know, we expect that lead's going to lead to all kinds of education, behavioral and, of course, even some health problems as it affects the nervous system.

 

Jennifer [00:10:10] So you mentioned that, you know, from the science literature, it does seem like there's evidence of at least an association between lead exposure and things like Parkinson's, Alzheimer's, you mentioned way down the road. I mean, I guess my hunch is that those are associations that they're finding. Right. And basically, you can do a randomized trial with maybe lab rats or something, but you can't do it with people. And so this strikes me as the place where the economist's toolkit really brings something to bear in being able to look at what are the true long term effects of this, given that, you know, human behavior can mitigate exposure, perhaps. Is my hunch about what the science is telling us accurate there?

 

Stephen [00:10:50] So I think yeah, I mean, of course, I'm an economist. I love our toolkit. You know, we can do things really well. But I do think there is a biological element which is quite linkable. If I you know, if I see damage in the brain here, I know that this can likely cause, you know, things like Alzheimer's or nerve damage or other things. So I think there is that element. And they and I think they've you know, I'm sure they've done lots of – kind of looking at people after the fact and kind of analyzing where the damages and what diseases they had. But I agree there's all kinds of other factors that going on that I think the economists are really good. And what we'll talk about pretty much the rest of the time here is the story of, you know, we can say very much about the story of what type of people are more likely to be exposed to lead and all these bad outcomes, they might get.

 

Jennifer [00:11:40] Right, kind of put another layer on this story. So what do we know – you alluded a little bit to this, but what do we know about what constitutes a dangerous level of exposure to lead?

 

Stephen [00:11:51] So it has changed over time. So meaning this has been an evolving story. If we go back for the study here and I can't really talk as much about earlier time periods, but I know the levels have just gone down over time that are considered dangerous. But for the study we'll talk about today, 10 microns per deciliter. I'm not a scientist. I'm not going to explain to you what microns per deciliter means, but it's just an amount of concentration in the blood. But the 10 levels is considered a dangerous threshold and where we start to see people intervening a lot more.

 

Stephen [00:12:23] Currently, though, they've actually moved these down. So the study we're talking about today is more in the 90s. If you look more recently, the Center for Disease Control, who kind of dictates or puts out there what levels are safe, they actually say 5 microns per deciliter is now considered dangerous. And so really in this past 10 to 15 years, we've seen this go down substantially, what they consider dangerous levels. And my guess is we might get to the point where really any exposure is going to be considered dangerous as we continue to see bad outcomes. Now, of course, we just don't know how much we can directly say is lead versus these other things, which will kind of confounding factors which might come into play, but at least the recommendations continue to go down. And we know at very high levels of exposure it's fatal. So, you know, if you're talking well above 50's, in the 100's, you're going to probably – it will be a fatal dose. And there's also other things that matter, too. So one thing that componds this issue is if you're healthy in other ways, your body absorbs less of it. Your brain absorbs less of it. So if I'm really have really good nutrition, and I'm exposed to lead, what will happen is my brain will absorb less of that lead. And so nutritional kind of information or how well you're taking care of yourself in other ways will impact this kind of how your body reacts to this exposure. So, yeah, those are – most things we're now at this 10, probably 5 is really more considered now the standard.

 

Jennifer [00:13:58] Yeah. And the nutritional side, it does sort of raise, you know, one of these potential confounders that makes it difficult to study and link this to outcomes. Right. It's like it could be that there are other things going on in your life that lead to bad outcomes. And we can't attribute it all to lead, which then leads us to kind of what we did know. So lead exposure is often held up as a potential driver of criminal behavior, particularly the big mysterious increase in crime in the early 90s, and then the equally mysterious big drop in crime after that. Kevin Drum of Mother Jones magazine has been a leading proponent of this lead-crime hypothesis that it was actually lead all along, that led to the big rise and then fall. But that hypothesis has been notoriously difficult to test for the reasons we're sort of circling around here. So before the study, what had we known about the effects of lead exposure on crime?

 

Stephen [00:14:46] So, yeah, I mean, I've heard this story, of course, many times before, and I think I mean, I think a lot of people believe this story. I think it's I do think it probably contributed in some way, but it's definitely not a kind of a slam dunk. There's no good slam dunk support for this hypothesis. I mean, there really has been some nice recent studies that I'll talk about a little bit later that really get after, I think that really have been pushing my kind of my needle on what I believe is how good the evidence is for this topic and the link between lead and all of these outcomes. And, you know, I think a lot of people done some good stuff with where we saw the ending of the leaded gasoline, where they no longer allow leaded gasoline and they no longer allow leaded paint in the 70s. But we're still not quite there yet. We can't like – to make that claim, I think is a bit of an overstatement. But we are getting closer. And so one of the things that kind of comes up, though, that we're talking about for is that there is lots of public health and medical literature that correlates lead exposure with all these bad outcomes, especially criminal outcomes, things like arrest, delinquent behavior and things like impulsivity, attention issues, hyperactivity. But again, I mean, this is probable causation. You'd like to get a little more towards probable causation. These are quite just really correlations. We don't really know if those are the main – the lead exposures is the driver or something else about those individuals.

 

Jennifer [00:16:18] Okay, so let's talk about the empirical challenges to studying this issue. What are the primary hurdles to measuring these causal effects, particularly the causal effect of lead on crime and other outcomes we may care about, as well as the causal effects of interventions aimed at helping those who've been exposed to lead. So talk us through kind of both the data and identification challenges you faced here.

 

Stephen [00:16:41] Absolutely. I mean, this is like some of the hardest causality work you can get at, right? We have, you know, something really bad that happens to kids. And then they also have, like, all these other likely bad things going on in their lives. So, you know, kids that live in home with lead are often worse off in so many other ways: lower income, less educated parents, have other health learning disabilities. We can go on and on. So which one is it? What is really driving these outcomes? And, you know, this is one of those scenarios, of course, where we'll never get a good field experiment. We'll never be able to do anything kind of in a really nice design an RCT, randomized controlled trial, because it would be unethical. I can't think of a scenario where we can do some type of random exposure. I will tell you, there is one study that has gotten close to that. Okay. So there was one RCT, randomized controlled trial, study of the impact of different lead abatement methods on childhood lead exposure. Baltimore, it's called the Baltimore Lead Paint Study, out of John Hopkins. In the 1990s, they randomize homes with known lead to different abatement activities. So of course this sounds interesting. Like we want to know which is the best way to have – to fix up a home so that the lead exposure is less or less likely for it to be transferred to children. So they did all these different abatement activities and it was very informative in telling us which was the best.

 

Stephen [00:18:07] The problem is that the kids living there and the kids living afterwards were not really told any of this. They allowed them to go ahead and live where in homes that were still likely had higher potential lead exposure. And so, you know, it provided this benefit, but it was clearly pushed beyond what is considered probably okay to do type of research and that we did provide some harm. We did it. We allowed some kids to live in areas which we know had lead exposure potential. And so even anything close to that won't be possible.

 

Jennifer [00:18:40] Right. No, IRB would sign off on that today.

 

Stephen [00:18:42] No, no. It's surprising that this IRB signed off. You know, but, you know, science has a troubled past in many ways. But, you know, I think that's informative. So that's the closest we got. We're not going anywhere near that. So, you know, here we're trying to do something a little different. So we're not going to get to give you – I'm not going to ever in this talk or in this paper, do we ever say we can say anything about the causal impacts of lead on kids' outcomes. And what we can say is something about what happens when we intervene on these kids, what happens when we do a bunch of stuff to help these kids out? And I'll talk about the details of what this this experiment kind of does in North Carolina. But, you know, it's one of those stories where we can say something, I think, which has a really nice policy implications, but doesn't help us with the fundamental understanding between lead and kids' outcomes. But I'll take it because I think the policy to being able to do something about it is really important, too. There's also some data challenges, of course. I mean, the data we're going to talk about today is quite detailed. These are administrative data sets. And so you have all the challenges of getting approval to use them, through IRB, through government agencies, data sharing agreements and so on, and linking them all up. I would not – I mean the reason I think I said – it's just this paper started back, you know, 2013, and this now was published in 2018. So part of the reason it took those five years, which I thought was pretty quick considering how long it took us to get the data, you know.

 

Jennifer [00:20:21] It could have been so much worse.

 

Stephen [00:20:23] It could have. And so like the good data is going to be the key to get past these identification issues. That's I think always the case that the more and more better detailed data, the more I feel we can get closer to that. The right story.

 

Jennifer [00:20:34] Yeah. I don't want you to undersell the causal results of this paper too much. I do think that the results speak a bit to the impacts of lead to begin with and we can talk more about that later. But also, I do also want to highlight, you know, it's hard enough to causally identify what the problem is in these types of settings. But I always really appreciate when a paper is able to tell us what to do about it. And that's part of the reason, one of many reasons I really like this paper is it actually tells us that, you know, this intervention has an impact. So it's very nice. So, yeah. So you found this wonderful natural experiment in North Carolina that is not an RCT but does enable you to measure the effects of a CDC recommended intervention targeting children with high blood lead levels. So tell us about the intervention.

 

Stephen [00:21:24] So this is actually an intervention that is common across lots of states, so the nice thing is this  is considered the Center for Disease Control's recommended intervention for children with high blood lead levels. And so the key here is that this is a common intervention that we look at that is involved and we are using this data from North Carolina. But in essence, the way it works is that there is a specific thresholds that where you're supposed to do different things for these kids. So let me walk you through a little bit of exactly what these guidelines are. So whenever a kid initially is tested, they are, you know, their blood test is then actually sent to a state agency that kind of collects all this information and then certain recommendations are given out. And this – these recommendations are known to the doctors. And the doctors are actually often screening kids in their first two years of checkups. So a kid will come in. If you're a kid that's two or under and you go to your doctor, they actually ask you a couple of questions. Mostly they ask you, do you live in a home that was built before 1978, which is the year that lead paint was banned? They also often will ask you if you live in certain zip codes which have high levels of lead exposure. If you are yes, in both those scenarios or even and I believe in one of those scenarios, they'll often they will go ahead and test you. Some states do vary, they might test you more often. North Carolina is a little less so. Only if you are considered at risk will they actually test you. And they test you, they report it, and if it comes back at a high level, anything above 10, they're going to immediately they're going to go ahead and say, you need to be tested again. And so we have every kid who gets tested above 10 gets a follow-up testing.

 

Stephen [00:23:24] And for this kind of – for this intervention, the way it works is that if you have two tests that are above 10, all of a sudden it triggers a certain amount of intervention. So the intervention, if you're above 10, but below 15, is that you get some education, some counseling, some nutritional counseling that tells you about the dangers of lead. If you get two test above 15, but below 20, all of a sudden you're involved. There's a case management element. So now you're in the social service system. And then for 20 and above, there's a bunch of other interventions that are included, medical evaluations, there's an environmental investigation, remediation, meaning they'll go in and remove or make the home safe from lead exposure, which would in essence often involve removing windows and doors, painting and sealing up any type of lead exposure.

 

Jennifer [00:24:18] Okay, so we're going to be thinking about this intervention as a package of all of that stuff, which are very careful in the paper to never say, you know, it's this thing, this one small element that's driving everything. This is going to be a big package, but it is the CDC recommended package. So very interesting. Yes, so tell us about the natural experiment piece of this. You have some people who test high for lead and they're eligible for this multifaceted intervention from the state. But, of course, then you need a control group in order to measure the causal effects of that treatment. So how do you identify that control group?

 

Stephen [00:24:50] Right, so it's actually quite simple, we are going to have, you know, two groups, we have a treatment group and a control group. The treatment group is going to be the group that had two tests above that 10 threshold. And we'll also break it down later on by ones that are above 15 or above 20. And then the control group in this case will be children that had an initial test that was above 10, but a secondary test that was between 5 and 9. So in essence, they're just going to miss out on the trigger for these interventions, but they're still going to appear – at least have had some lead exposure. We'll talk a little bit. It's kind of a bit of a noisy measure. So this leads to lots of variation from test to test, even though you might think, oh, is this something we should consistently measure? Interesting enough, lead has a really short half life in your bloodstream, but a really long time, kind of half life in your brain. So you can get exposed, it affects your brain. But if I test you or wait too long to test you, it might not actually show up right away. You might have already kind of processed it. And so it makes it kind of notoriously difficult to always nail down your exact exposure level with this type of testing.

 

Jennifer [00:26:04] So basically, yes, you're arguing it's essentially random if you test just above or below the threshold that second time around. So you have to talk more about what explains the randomness in this process. I think the lack of precision here will be a surprise to many listeners.

 

Stephen [00:26:20] Yeah, it is. It was a surprise to me. I mean, I have the raw data that shows each individual test and I'm like, why in the world would they be jumping around so much from test to test? But this is often how it's done. And they will test differently. They can test you through a blood kind of finger prick or even through a vein, draw from the vein. Reality is, anyone who has young children, you would know that how difficult it would be to actually draw blood via a typical how we treat adults when drawing blood. The fingerprint tends to be the way they almost always do it. But there's really the like I said, lots of variation based on the timing of when you're tested. Just also about how if there's contamination, one of the issues that comes up often is that people might have lead dust on their skin or clothing. And that could, of course, impact whether your – it's showing up. Also just how your body processes the lead exposure. Like I talked about before, nutrition can matter, other things can matter.

 

Stephen [00:27:18] People vary in their ability and their how they actually process something like this. And so that that can lead to kind of all kinds of issues. One of the things we kind of later on will kind of hang our hats on and say, hey, if there's anything wrong, it's going to help us, it's going to make us find negative impacts of intervention. Meaning that the story is this, that someone who gets a higher test both times is also likely to simply have higher levels of exposure. If it's a noisy test, but, you know, we expect on average, if you have more exposure, it's slightly higher test that's going to be in our treatment group. They're probably worse off than lots of other ways. So the nice thing is that if we're messing this up, that this is kind of causing problems because people are just have a little bit higher exposure and we're calling put them in the treatment group and control groups a little bit lower exposure. It should, in essence, make our estimates of our benefits kind of conservative, or smaller than they should be.

 

Jennifer [00:28:15] Yeah. It'll bias you toward finding negative effects. Okay, so anyone who is familiar with your work knows you have a bunch of cool administrative data from North Carolina. You are a master of getting these administrative data sets and linking them all together. So tell us what data you use in this project to analyze the effects of this CDC intervention.

 

Stephen [00:28:37] Yes. And, you know, I do think I've consistently tried to get more and more data, but this project involved lots of kind of I think cool and interesting data sets. So the three main data sets that are going to come into play for this study are (1) is the blood lead surveillance data. Surprisingly to most people, but if you actually look into it, it is quite feasible you can get individual level data on this blood lead testing for children. It's considered a public health exemption to HIPAA based restrictions, and it's done for purely for research purposes. So research affiliated kind of institutions and so on can get this type of data. The nice thing is it has enough detail that you can then link it to other data sets. And the data sets I link it to our public school records from Charlotte-Mecklenburg Schools, which has been a partner in many papers, and then also criminal arrest records, which are actually public record in North Carolina. And so you can actually – people listening to show should do this. You can go and look up – there's a North Carolina arrest look up website where you can look up anyone who's been arrested and find out lots of information about them.

 

Stephen [00:29:47] So these three data sets are going to comprise the main kind of core of data for this study. And I'll talk a little bit on some of the outcomes, some where, you know, what I get out of this data. I mean, the blood lead surveillance is going to give us the details on these above 10, below 10 threshold stories. Public school records give us all this information on testing, dropping out of high school, disciplinary stuff like suspensions, absences, and then a criminal arrest record will tell us people arrested at a young age. The nice thing in North Carolina, I don't think I don't personally agree with it as a policy, but the way it works in North Carolina is that 16 and older is considered an adult in North Carolina. So we can start observing people at a pretty young age, which is helpful given the span of data we're looking at. We're going to focus back in time to about the late 80s, early 90s. Kids kind of born in these early – I think we focus on cohorts from like 1990 to 1997, and that way we can observe them as adults later on in our data.

 

Jennifer [00:30:47] What's the sample that's included in the study? Who gets tested in North Carolina or at least how many people get tested in North Carolina and how many of these folks are you capturing here?

 

Stephen [00:30:58] So I don't have exact number on the number of people tested in North Carolina at hand. But least in terms of what we're able to kind of identity at this high level of exposure and match up to all these datasets, we actually end up with only about three hundred observations. So a slightly, I mean, not the largest sample size, but enough of a very kind of select group of people where we can get some good information off of.

 

Jennifer [00:31:27] And basically is the way to interpret that number that this is probably the universe of highly exposed individuals in this area, or do you think that there are a lot of people out there you're missing?

 

Stephen [00:31:37] So we should not be missing that many kids. I mean, anyone who's been at that level of exposure we're talking about is often going to be tested. Of course, we could miss some kids. Some kid gets exposed and then their parents don't take them to the doctor and they refuse testing. We can miss that. It doesn't sound like that's likely. We're going to lose maybe 30 percent of the kids just because we can't match them across these data sets. But we are talking, you know, one county, a couple cohorts, and also the reality is Charlotte is just not a city or part of the country, which has extremely high levels of exposure possible because it's a newer city. Right. The way lead works is if we go to places like, you know, the Northeast or the Midwest, we'll find cities that have a lot of older housing stock, a lot more lead exposure possible. So these numbers are going to be less than some other cities. And so it just has to do with the nature of the city that we're talking about.

 

Jennifer [00:32:35] Yeah, it's really interesting. And as you mentioned earlier, obviously lead exposure has fallen dramatically in the U.S. over the past several decades. I gather this is still a much larger problem in developing countries. So this is you know, it is a problem for the kids who are exposed in the U.S. and it is also a major problem worldwide. But yeah, but this the sample size around the 300 kids that you're working with who are just like you, who test over the threshold once and then either not again or test over the threshold one more time, you're down to about 300 kids. The sample size is probably the biggest issue you guys had to deal with. I imagine some people listening, the economists listening are going to think, oh, they're going to do a regression discontinuity right around that threshold. Look at people who test just below the 10. You need a bigger sample to do that. So you guys are going to do a nice difference in difference, if I'm recalling correctly. And so you're going to compare these people who test over once, but not the second time with the people who test over the 10 threshold twice. So before we get into the results, what mechanism should we have in mind for why that treatment group might have different outcomes than the control group in this dif indif framework?

 

Stephen [00:33:40] Yeah, so I think that's a great question and lots of people have always kind of asked this what's really driving it? You know, we're not going to be able to disentangle it completely. We're going to kind of get snapshots, say we think this is more likely and this elements more likely. But it seems to be the evidence points towards a couple things. One is that I think case management is important. These kids are now in the system when they get a lead intervention. So if they're kind of at the in the intervention group. They're going to now be part of the kind of social service system, and that means they're going to get they're going to have a lot more kind of services available to them. They're going to have more people checking in, making sure they're okay, that they're safe. And so I think that is something that often we don't think much about. But it probably matters a fair amount for kids that are in this population, a population that is really struggling in lots of ways beyond lead exposure. So I think that's likely part of what's going on. Of course, lead paint remediation is likely mattering a lot too. Kids are no longer exposed after they get this intervention, meaning that we go in, someone goes in there, takes away all the lead paint or covers it up in a way that they can't be exposed to it anymore. So this is going to limit repeated exposure, which is often the case. Often these kids are just getting exposed again and again and again. It's just building up in their bodies. There's probably some benefits, too, from things like medical services, parental education, things like that. But that's harder to nail down in these results. But it seems like this all should matter some.

 

Jennifer [00:35:13] Yeah, and there's a nutritional component to the intervention too, is that right?

 

Stephen [00:35:17] That's right. Yup. So with the medical services, nutritional education, parental education. So that could be that could absolutely be mattering that all sudden these kids are now eating healthier, pay attention in school. I mean, you remember everything compounds when we look at something, even maybe something that seems relatively smaller for a kid that's only two years old. If you do that initially, all the benefits can compound over time. All of a sudden you pay attention better in school, all of a sudden there in school, you then have access to a teacher who kind of looks after you and so on and so on. And you stay out of trouble from some kids in the neighborhood. It just builds. So often these small things can actually matter even more so in the future.

 

Jennifer [00:35:56] Yeah. All right. So let's dive into the results. First, you look at the effects of being in the treatment group on just basic compliance with the recommendations for additional testing. So first, why do you do that? And second, what do you find?

 

Stephen [00:36:10] So ideally, we would actually have data from the social services agencies that tells us exactly what they do with these kids as part of this intervention. But we don't. We don't know for sure that the kids got a medical exam. We don't have data that says that. We know they're supposed to, but we don't have the actual case management data. So these are things we, of course, we tried to get. These are just very protected types of data sets. So I'm not surprised that we had trouble getting them, and they said no. But we do this kind of initial story to test, to see are the kids in the treatment group, are they acting differently and are they acting in a way that's consistent with the recommendations of additional testing that go along with this intervention? So I don't know if I said it before, but is pretty strict guidelines about what type of follow up testing you need to do once you're in the intervention group. So you have the two test above 10 and they say, well, you need to come in here and get tested again every three months, depending on the level or even every month or even every two weeks for a couple months, the next couple of months. So there telling them to come in and get tested and we're testing and say, do they actually do this? And so one thing we see very strongly is that kids definitely follow up with those recommendations and kids in the intervention group get tested a lot more and a lot quicker, a lot sooner after that second confirmatory test that went down. They also – we also see some evidence that they are actually getting also lead paint remediation activities going in their houses too where their addresses. So that's consistent also with intervention.

 

Jennifer [00:37:47] Okay, great. So it does seem like the interventions actually happening for the kids in the treatment group. All right. And then you consider the effects of that intervention or at least eligibility for the intervention on a variety of educational outcomes. So, yeah, I guess first, walk us through what education outcomes you can see and what you find there.

 

Stephen [00:38:07] So for education stuff, we're going to be limited mostly to where we're going to call end-of-grade test scores. So at the end of every grade, kids in third through eighth grade take a standardized test for math and reading. So look at those in middle and in elementary school, and then we'll also look at grade repetition. And so what we find is that primarily we find effects in terms of test scores, not as much in grade repetition, but we do find that using what we call a summary measure of educational outcomes, that kids intervention group end up being about .1 standard deviations higher. So let me let me kind of break that down a little bit. What we do is we put everything into a kind of an index because we want to look at lots of things and we want to we don't want to just get one or two results, say, oh, we're just cherry picking. This is just the you know, oh, look, you got this result. But we want to have a summary measure of everything. And so it's going to be a means 0 standard deviation 1 and normally distributed type of index. And so the idea is that, okay, what is the impact on this intervention group? Well, it's about .1 standard deviations. This is actually somewhat comparable to some other early education interventions like Head Start and preschool. So this is a pretty sizable effect that this package of interventions is comparable to some of these early preschool, pre-kind of preschool type of programs.

 

Jennifer [00:39:31] And this is the intent to treat effect, right? You're just measuring eligibility so you can't see who's actually getting the treatment. And it's probably biased toward zero for that reason. That's going to be a conservative estimate of how powerful this is.

 

Stephen [00:39:44] Yes, I'd love to, yeah, we don't have a great sense of what that scaling should be to get an effect, but I agree.

 

Jennifer [00:39:50] Yeah. And then so in terms of the next results, the moment, we've all been waiting for, you consider the effects of this intervention on antisocial behavior such as criminal behavior. So what outcomes do you have there and what are the results?

 

Stephen [00:40:05] So we have a lot more richer outcomes for this for these kind of criminal antisocial behavior. So and also this is the area where we expect there to be the largest impacts because of how lead works. And as we talked about before, it has this impact on impulsivity. Impulsivity seems like such an important thing when we talk about antisocial behavior. And so we look at things like school suspensions, absences, school crimes. These are required crimes that are reported as part of the school system. So if you do a certain activity in school, it's required by state law to be reported. So it's a very good measure of kind of criminal activity at a younger age. We also have young adult arrests. So we know for kids 16-18 years old had they been arrested for violent or property crimes. And so we, you know, we'll look at a kind of all these things individually. You know, in summary, as a summary measure, we get about .18 standard deviation effect. That's a pretty sizable effect. But let me put in terms of something more tangible. If we look at 6th to 10th grade suspension days, so the number of days suspended in 6th through 10th grade, we get about a six day decrease in the number of suspended days for the treatment group relative to the control group. And the control group averages about 17 days. So this is a quite a sizable effect. Yeah, and for adult arrests, we even get you know, that's even more a larger effect. We go we had a 10 percentage point decrease in the ever being arrested, ages 16 to 18. And this is off of a baseline of 18 percent for the control group. And so really, we're talking about a 50 percent decrease, that was more than that for this type of crime activity.

 

Jennifer [00:41:54] Yeah, and so you got this package of stuff the CDC does, but it is just, you know, a relatively straightforward intervention. And then to have that – to transform these kids lives that much I think is just really remarkable. And so, you know, you mentioned at the beginning that you can't really say anything about the effects of crime. But I, you know, reading the paper, certainly it seemed like the fact that the treatment group here does so much better than the control group does tell us a bit at least about the damage that lead does with that, at least without the intervention. So what's what's your takeaway here from the evidence that you have on this?

 

Stephen [00:42:27] So I do agree maybe and I think you're right. Thanks for pushing back in the sense that I think we can say something. You're right. We can say something about that because we are, in essence, looking at kids that look kind of similar, similar type of exposure. One gets his intervention and we feel that makes a difference and moves the needle in that direction. So it does it is saying that, you know, the lead exposure matters, right? That that's something that's causing or at least, you know, we'll talk about correlation and causation, and, you know, we could debate that, but it does seem to be consistent with that story that the lead exposure is mattering and causing these types of outcomes. You know, but it does seem like also I mean, the I main, I think the main takeaway and and I think one of the reasons you told me in the past you like this paper is because it's a little bit more of a positive message in some ways. But these interventions are super impactful. And even though we can't really reverse some of the damage done by lead exposure, there does seem to be some hope that these interventions and limiting more exposure seem to really matter. And I think that's a really important takeaway, because now we we can act on that. You know, we can't act on the fact that kids have been exposed in the past. But we can do something about, to the kids that have been exposed or even prevent future exposure.

 

Jennifer [00:43:41] Right. This is a story about, you know, past lead exposure leading to sort of a bunch of kids being a lost cause somehow like there is something we could do. Yeah, which I think is really important to know. So you can't completely tease apart the different parts of this intervention since it was administered as a full package, as I've mentioned, a few times. But you can do a little bit to try to figure out which mechanisms seem to be driving the effects. So what do you test for there and what do you find?

 

Stephen [00:44:10] So we do a couple of tests, I mean, this will be incomplete, but this is kind of the best we can do. But we do find that kids you saw the largest drop in their blood lead levels and follow up testing, so this would be a third or fourth test, fifth tests and so on, generate the largest impacts. So that's going to be consistent with some type of removing continued exposure. Right. So the kids who get like, you know, we'll see kids in the intervention and control group get lots of – will get – they'll all get followed, they'll get season follow up testing for both of them even. But for the what we do see in this intervention group, ones which have the largest drop, they really do seem to be better off. So that's consistent with the kind of removal of exposure type of story. So that's important. We also do some stuff that shows that future residents in the treatment group. So you can imagine that a home, a kid was in a home, they had a high level of exposure. We had this intervention. They fixed up the home, made it safe from lead. Another person moves in later on. We can actually look and see if that person who moved in later on, the kid, we'll see him in the school system, do they have similar types of outcomes, meaning they're probably similar types of kids because they live in a similar place in a similar part of town. And they don't. They really tend to have really a lot, really almost very little effects of being in the treatment or control group. So we feel that future residents in the treatment group residents had better outcomes and future residents in the control group residents and that it must be the case that we're talking about some type of removing this exposure for this to be consistent with that story.

 

Jennifer [00:45:47] Yes. There were changes made to the actual home as opposed to just – wasn't just the education of the parents or something like that.

 

Stephen [00:45:55] Exactly. Because we imagine that people choose homes there. If you choose the same home, you're probably similar in lots of ways. We also find that we find effects for kids between 10 and 20 thresholds. The fact that we do find some different effects, whether we were just the 10 threshold or even above the 20 threshold, is consistent with different parts of intervention kind of mattering. Right. And so it can't just be a story of purely lead paint remediation, because we do get effects for kids that are below that 20 threshold when they go in there and fix up the home.

 

Jennifer [00:46:29] So I always like to see a cost benefit analysis in a policy relevant paper like this, and you and Kevin did not let me down. So walk us through how you estimate the social benefits of this intervention and how they stack up relative to the costs.

 

Stephen [00:46:44] Absolutely. I'll be honest, we originally didn't have a cost benefit analysis because it was super challenging to come up with one for this paper. But a referee luckily made us do this. So that was good.

 

Jennifer [00:46:57] Thank you, referee.

 

Stephen [00:47:01] So I would love to have a more definitive cost benefit analysis. Absolutely so – but the best – this is the best we're going to be able to do, we're going to have to use some estimates from the literature, say they're going to be similar, capturing similar elements as our study does, and then we can actually use that as our benefits and then the cost we can get pretty good estimates for just by talking to people in the social service, the people who actually implement these interventions, like what do they think or what are the costs likely to be for this type of story? And so, you know, taking all this kind of benefit estimates from the literature and these are literatures like stuff that people look at, the long term impacts of things like Head Start, Perry Preschool, a lot of these what I call early childhood educational interventions. That's where the literature is the thickest. That is where we have some good long term impacts on things like the social cost of crime and lifetime earnings effects, things that we can translate into dollars and bring back to think about long term benefits. And so we get about a $10,000 benefit from these literatures and estimating based on our estimates on the impact on these kids in the intervention group relative to the control group. And this would come at a cost of about $5,000. So seems like a good return in that way. These are likely quite conservative. I mean, in all this cost benefit analysis, I mean, it could honestly be quite a bit more in terms of benefits.

 

Jennifer [00:48:32] This paper was published in AEJ: Applied in July 2018. And of course, it was out in the world for a little while before that because of publication lags. So what else have we learned about this topic since you first wrote the study?

 

Stephen [00:48:46] So there's been a lot of papers in this literature since then. Interestingly enough, when we started this project in 2013, this was well before Flint, Michigan. Flint, Michigan, I think woke up a lot of people to the importance of this type of research. And so I feel like it was a little bit – there was a lag. There wasn't a whole lot. And Jessica Reyes, who's a great lead researcher, was really one of the first before to really get into this as an economist. And but all of a sudden, I've seen tons of papers in this space in the last couple of years. Some of the ones that I think – I'm going to name a few, but there's lots of other good ones. And I don't mean any disrespect for the ones I forget, but the ones that kind of come out and then really have stuck my mind. At the same time, we were publishing this paper and Anna Aizer and Janet Currie had two papers in this space. They look at lead based paint remediation and showed as a negative impact of lead exposure on kids, on educational outcomes, they used the deleading of gasoline and showed negative effects on behavior outcomes of lead exposure. So there are some two nice studies that really confirm some of the educational and behavioral stuff that we talked about today and that link to lead exposure. And I think they really pushed the need a little bit in this kind of causal story, which is great. There's a bunch of papers out that are still working papers. There's a paper by Clay, Portnykh, and Severnini that use, that look at the negative effects of lead and lead gasoline on fertility. Feigenbam and Muller had this I think this cool paper that uses variation in the pH of water and leaded pipes to show that if you're exposed more to, if you have water that's more acidic and you get more lead exposure, that we see more homicides.

 

Jennifer [00:50:34] I really like that paper. It's a fun economic history paper for folks out who are into that kind of thing.

 

Stephen [00:50:39] Yeah. Hollingsworth and Rudik have a cool paper that's been circulating around that looks at actually the fact that racetracks banned leaded gasoline a lot later than the 70s. So they only more – like 15, 20 years later, do we see racetracks actually stop using leaded gasoline for their races. And so they do some cool stuff to show that this higher lead exposure reduces test scores and increases elderly mortality. So I like this paper because it actually looks beyond just kids. It's really one of the first paper that looked kind of more, I mean, homicides, yes, but most things have focused on these kind of educational and behavioral stuff. And so the Clay, Portnykh, and Severnini and this paper do some nice health stuff, which I think is important. And one other paper I saw not too long ago was, is a Curci and Masera paper that links lead exposure in the soil and the deleading of gasoline to very inner city violent crime. And they use it as kind of – to test the story of crime in the inner city, pushing residents to the suburbs and understanding suburbanization trends in the U.S. So it's kind of a cool kind of mix of this story. But, you know, there's really I think these are all quite creative papers in thinking about trying to get that elusive causal estimate of lead and all these outcomes. So I'm optimistic that we're getting pretty close to having a more definitive body that can really make that link strongly and make us feel like, hey, we have a good sense of the magnitude. The magnitude is always a bit hard to nail down, too. So not just that we can establish the causal link, but that we know how big it is. The different context, the different ways we get variation. It makes it a little more challenging to do that.

 

Jennifer [00:52:20] Yeah, of course, we'd love to have just a randomized experiment where we know for sure. But there is something really fun and special about a literature where, you know, we know we'll never have the randomized trial. And so everyone just tries to be super creative. And then seeing what they're able to come up with is, it's really fun. Yeah, so putting it all together, the results of this study, as well as all the other studies you just mentioned, what are the policy implications of this work? When policymakers ask you what to do about lead, what do you tell them? 

 

Stephen [00:52:55] So like I've already mentioned, I'll say it again, because I think it's a hopeful paper. So when I read this literature, I get so depressed. I mean, we need hope that because it's so bad, this is such a terrible thing for these kids and so on. So when we think about policy, though, I do think, like the more this literature, all these papers I just mentioned, is really, should be making it very clear to the policymakers that there is something causal going on with lead and all these outcomes. And we need to spend a lot more resources and thinking about these pollutants. And probably beyond just lead. I mean, I have no doubt that we could find lots of other environmental pollutants that are causing some of the same problems. I think lead is just easy, it's easier to  narrow down on because it has very direct sources that we can think about. But this is really important to think about how it impacts our neighborhoods, homes and cities more broadly. And some of the current research I'm doing on is looking at how lead remediation is actually impacting whether neighborhoods improve over time. So I've worked with Kevin Schnepel still and Ludovica Gazze, who we actually go ahead and have funding from HUD, from Housing Urban Development, where we actually look at this stuff, the story and see if we can say something meaningful about this impacting neighborhoods more broadly. I do think that this – so I do think that the studies that gives us some hope that early childhood assistance can help address some of these deficits. You know, one of the things that this paper is that it's in reality is more of a early childhood education paper. And so it contributes to that way, but it does it in a different way than most of the other ones. Most other ones focus on these things, like I said, like Head Start and some of these other programs that are very well defined and focused mostly on education. This is actually focusing a lot on other things. And so, you know, we can start to think about other early childhood interventions that are beyond education, which I think we've found to matter. But there's probably lots of other ways we could help kids that are in low income and kind of disadvantaged households in a way that could lead to lots of long term benefits.

 

Jennifer [00:55:12] Yeah, that's interesting. And actually reminds me of – so I interviewed Chloe Gibbs for the very first episode of this podcast and she noted something that I had not realized about Head Start in the form that it's often evaluated where it was actually much less of an educational intervention at that point and much more of a medical intervention. That's where kids were getting lots of like free dental care and checkups and stuff like that where they wouldn't have gotten otherwise, which then does put it much closer to this paper in that way, which is really interesting to think about. All right. So what's the research frontier here? You're obviously working on some other stuff, but presumably not answering all the questions. So what are the big open questions in this area that still need to be answered?

 

Stephen [00:55:54] I mean, that's always a good question. I think that a couple of questions I came up with in thinking about this story – I think we don't have great evidence on does lead exposure to child impact later life, dementia, other neurological disorders. If lead is really supposed to affect these things, we have really almost no evidence on those outcomes. And that's super challenging kind of area just because it's hard to get that data. But I do think we really should care about that story. And so that's something that those health outcomes, I think, are going to be something we need to spend more time on. We talk about cost benefit analysis and all of these other stories are what is the magnitude of these effects? We often are missing a lot of these long term health stuff that I think is really important. We also have very limited evidence on kind of more the adult exposure. There's a little, like I said, mentioned some papers on fertility, elderly mortality. But I think there's a lot of important stuff on just general labor market, health outcomes, things like earnings, potentially unemployment, things like am I consistently getting sick all the time, having other issues that prevent me from working, being productive? I do think one of the areas I'd love to see a lot also is how does lead coincide or interact with other pollutants like mercury, asbestos and air pollution more broadly? I'm guessing that often if you're exposed to lead, you're probably exposed to lots of other bad things. And so the question is how much of that is kind of bleeding in or being captured in some of these effects of lead? And how much does it interact with them, does it make it worse kind of compound these effects? And, you know, how does the lead rate, you know, if we're the, you know, the EPA or some other government agency, and we're trying to say what's our priority for removing these from society because we know these are bad, which is the worst? We need to have a good sense of where they all rank together.

 

Jennifer [00:57:47] Yeah, that's a really good point. And there is sort of a budding environmental econ in crime literature out there. People are starting to look at these questions about pollution more broadly. But, yeah, I would love to see more on that. Well, my guest today has been Stephen Billings from the University of Colorado at Boulder. Steve, thank you so much for talking with me.

 

Stephen [00:58:08] Thanks for having me.

 

Jennifer [00:58:14] You can find links to all the research we discussed today on our website, probablecausation.com. You can also subscribe to the show there or wherever you get your podcasts to make sure you don't miss a single episode. Big thanks to Emergent Ventures for supporting the show and thanks also to our Patreon subscribers. This show is listener supported, so if you enjoy the podcast, then please consider contributing via Patreon. You can find a link on our website. Our sound engineer is Caroline Hockenbury with production assistance from Elizabeth Pancotti. Our music is by Werner, and our logo is designed by Carrie Throckmorton. Thanks for listening and I'll talk to you in two weeks.