Episode 64: Jason Baron

 

Jason Baron

Jason Baron is an Assistant Professor of Economics at Duke University.

Date: January 4, 2022

Bonus segment on Professor Baron’s career path and life as a researcher.

A transcript of this episode is available here.

Episode Details:

In this episode, we discuss Prof. Baron's work on the effects of foster care on criminal justice involvement:

“Is There a Foster Care-To-Prison Pipeline? Evidence from Quasi-Random Investigator Assignment” by E. Jason Baron and Max Gross. [Working paper available by request from the authors].

 

TRANSCRIPT OF THIS EPISODE:

Jennifer [00:00:08] Hello and welcome to Probable Causation, a show about law, economics and crime. I'm your host, Jennifer Doleac of Texas A&M University, where I'm an Economics Professor and the Director of the Justice Tech Lab. 

 

Jennifer [00:00:18] My guest this week is Jason Baron. Jason is an Assistant Professor of Economics at Duke University. Jason, welcome to the show. 

 

Jason [00:00:26] Hi, Jen. Thank you for having me. I'm excited to be here. 

 

Jennifer [00:00:29] Today, we're going to talk about your research on the foster care to prison pipeline. But before we get into that, could you tell us about your research expertize and how you became interested in this topic? 

 

Jason [00:00:40] Yeah, of course. So I am an applied microeconomist who is interested on topics in child welfare and the economics of education. And I primarily use big data sets, big administrative data sets and quasi experimental methods to answer what I believe are policy relevant questions. So most of my research interests really are shaped by my upbringing in Mexico and really my experience seeing very different cultures with just drastic differences in institutions ranging from, you know, the public education system to the criminal justice system. So seeing these differences firsthand has really always made me wonder how much these institutional differences could be at least partially responsible for some of the stark differences in economic and social outcomes across these two countries. And what would happen if, for example, Mexico adopted some of these institutions? So just to give you an example, I am very interested in the effects of funding for public schools, given the relatively low investments I saw in public education growing up in Mexico. So, for instance, in my job market paper, I examined the causal effects of additional public school funding on children's outcomes. 

 

Jason [00:01:47] And foster care was just another example of this, right. So at least where I grew up in Mexico, I never once heard of, say, a CPS worker knocking on anyone's door. Right. And given just how costly child abuse is, this has made me very interested in understanding the broader impacts of foster care in the child welfare system in our society, and it's causal effects on children's short and long term outcomes. So this is really what drives my interest in the foster care system. 

 

[00:02:13] So your paper is titled "Is there a Foster Care-to-Prison Pipeline? Evidence from Quasi-Random Investigator Assignment," and it's coauthored with Max Gross. In this paper, you consider how placing children in foster care affects their long term outcomes, and you're focused on the Michigan foster care system in particular. So let's start with some basics. Why might children in Michigan be placed into foster care, and who would they typically be placed with? 

 

Jason [00:02:38] Yeah, so a child in Michigan will be placed in foster care if he or she was a victim of child abuse or neglect and if the child faces imminent risk by remaining in the home. Okay, so this abuse is usually carried out by the child's either parents or legal guardians. And in terms of who they are typically placed with, children can be placed in a variety of settings. The most common sort of three broad categories are either relatives. So this is called kinship care, you can think of these as maybe staying with, say, a grandmother. Second category is an unrelated licensed foster family. And the third broad category is a group home or a residential institution. Now, just to give you a sense, in 2015, for instance, the most common placement in Michigan was with an unrelated foster family, followed next by a family member and a much smaller portion of children, though not a trivial portion, about 10 percent are placed in a residential setting or in a group home. 

 

Jennifer [00:03:37] So what is previously known about the effects of foster care? 

 

Jason [00:03:40] Yeah. So we actually know quite little about the causal effects of foster care placement on children's outcomes. Now there is a large, mostly correlational literature that primarily comes from the fields of sociology and social work, showing that foster children tend to have much worse outcomes relative to, say, children who were investigated for abuse or neglect but who were not placed into foster care. And these studies typically show that foster children have worse test scores and other educational outcomes like high school graduation, and are also much more likely to have contact with the juvenile and criminal justice systems. But to my knowledge, for a very long time, the only causal evidence on the effects of foster care came from two really great and groundbreaking studies by Joe Doyle, who's a Professor of eEconomics at MIT. And these two studies one in 2007, published in the American Economic Review and one in 2008, published in the Journal of Political Economy, use a similar strategy to the one that we use in this paper and that I know we'll get to it in just a second year in examining the outcomes of children who were investigated in Illinois between 1990 and the early 2000s. And what Joe found in those papers is that foster care for children at the margin of placement led to substantially worse outcomes. Okay, so just to get more specific. He showed that foster children had arrest, conviction and imprisonment rates as adults that were almost three times higher than those of children who remained at home. And he also showed, you know, similar large negative effects of foster care and other really important outcomes like juvenile delinquency, teen motherhood and employment. And for a very long time, this is really all we knew about the causal effects of foster care. 

 

Jennifer [00:05:32] So why don't we know more than just those two great papers? But - it's always good to have more than two papers. Why is this such a difficult topic to study? Is it mostly about finding the right data, or is it mostly about finding a good natural experiment or is it both of those things? 

 

Jason [00:05:47] Yes, I think it's a mixture of those two challenges. So I would say for this particular topic, I think given that Joe Doyle came up with that sort of great research and he found this really interesting institutional setting that allows us to estimate causal effects in foster care that, again, we'll get to in just a minute here, given that he kind of showed us a really great source of natural or exogenous variation to identify the causal effects of foster care, I think that is less of an issue in terms of finding, you know, finding the right research strategy. Perhaps the biggest issue is the need for high quality administrative data. And you know, so let me unpack that a bit because I think it's very important to understand the type of data set that one would need to get a good estimate of the effects of foster care on children's outcomes. And really, the big issue here is that you need not only individual child welfare records, which are difficult to come by. This is an area that is very sensitive and there are a lot of privacy concerns with these types of data sets. But beyond simply getting child welfare data, you also need to be able to match child welfare data to other administrative data sets if you want to think about the effect of foster care o other outcomes, right. 

 

Jason [00:07:03] So we'll probably talk about my data set and the data set that we've assembled in Michigan later on in the podcast. But if you think about what one needs in order to to carry out this exercise, you need to be able to match, say, child welfare records to adult criminal justice records many, many years later, right when we've had enough time to to kind of observe those children who were once placed in foster care in their adulthood. And so these types of data sets that match across various state agencies, while typical in countries like some of the Scandinavian countries, for instance, they're pretty rare in the U.S. And again, this is primarily due to data privacy concerns that don't allow you to match, you know, a set of administrative records to administrative records from another state agency. So, you know, these where people like Brian Jacob and Joseph Ryan at the University of Michigan have just done such a great job at sort of negotiating these types of data sharing agreements. And, you know, really, really trying to be able to map children to many different outcomes, not just limited to one particular administrative data source. And so, you know, so in short, it's a combination of the need for a source of variation in the probability of placement that is as good as random, though we know methods to do that, but also the need for a high quality individual level administrative dataset. 

 

Jennifer [00:08:26] Yeah. So we will talk more about this amazing data you have shortly. But first, let's talk about strategy, which is also wonderful. So, so you're going to use the as if random assignment of cases to investigators in Michigan as a natural experiment. So let's get into those nitty gritty details of how this process works in Michigan. How does the government find out about potential child maltreatment cases and how are those cases then investigated? 

 

Jason [00:08:52] Yes, the child welfare process in Michigan works pretty much like the one in most other states. And the process really begins when someone calls an intake hotline to report child abuse or neglect. This intake process is the same regardless of who the reporter is. And really, anyone can call the hotline to report suspected maltreatment. But the most frequent reporters are people who are mandated by law to do so. So these groups include, for example, education personnel who are responsible for about a fifth of all calls. But it also includes personnel like legal and law enforcement, pediatricians and other social service workers are also common reporters. And, you know, as a kind of as a quick aside, I think it's these are really important point on its own that teachers are the number one source of maltreatment reports. And you know, this has really broad implications for the well-being of children whenever schools are closed, for instance, due to COVID. Right. So right now, this is a topic of great interest. As, you know, teachers are not able to see children, and many of these children, you know, could be sort of suffering in silence in their homes. Now, after the call is received, a hotline employee who will not further participate in the investigation process is going to transfer any relevant reports to the child's local child welfare office. 

 

Jason [00:10:14] And I think it's, you know, it's really, really important for our strategy to note that the way in which maltreatment investigators are selected for cases. So the way that assignment of cases to an investigator happens is that within local teams, cases are assigned to individual investigators based on a list. Okay, so this is known as a rotational assignment. So each case that comes in is assigned to the next available investigator. Once all investigators have been assigned a case, then what happens is that the investigator at the top of the list is assigned a second case and so on. So while cases are not randomly assigned to investigators explicitly, the result is pretty much the same. Okay, so we are going to refer to this as a quasi random assignment of investigators or the as good as random assignment of investigators to cases given that investigators are not selected to case according on their characteristics or expertize. And so just to give you a sense of kind of the prevalence of these investigations. In Michigan, we calculate that about one in five public school students was the subject of a formal investigation by child welfare by the third grade. Okay, so this is quite a common practice and it's much more prevalent than people realize. 

 

Jennifer [00:11:31] Great. And so what are the possible outcomes for a case in Michigan? 

 

Jason [00:11:36] Yeah. So investigators are going to make sort of two crucial decisions that are going to jointly influence the intensity of the level of child welfare involved. The first decision that the investigator is going to make is whether there is enough evidence to substantiate the allegation. Right. So investigators are going to interview the people involved, you know, examine the home, review any relevant and medical and police reports and then determine whether there is enough evidence that the allegation did in fact occur. This has everything to do with evidence. Nothing to do with risk. Okay, so just to give you a sense, seventy five percent of reports in 2016 nationwide went unsubstantiated, meaning that child welfare officers do not follow up with the families because there is not enough evidence that the allegation did in fact occur. 

 

Jason [00:12:26] Now conditional on substantiation meaning if the investigator substantiates then investigator judgment over the risk that the child faces by remaining in the home will determine the outcome of the investigation. Okay, so there are sort of three outcomes here. If the risk level is low, then the investigator must refer the family to what I referred to as light touch community based services. So these are very minor interventions like referrals to food pantries, support groups or other local non-profits. And these cases require really no further follow up by child welfare after this initial referral. Now, if the risk level is high, the family, in addition to the community services, is also going to receive much more intensive targeted services. So these are services like substance abuse treatment, parenting classes or counseling. And finally, the third potential outcome is that allegations with especially high risk are not only going to trigger the community and targeted services that I just mentioned, but these are also going to require the investigator to file a court petition for child removal. So these are the possible outcomes that an investigation in Michigan can have. 

 

Jennifer [00:13:42] And then once that decision is made, then it also gets handed off to somebody else, right? Then that investigator is not involved anymore. Am I remembering that right? 

 

Jason [00:13:51] That's exactly right. Yeah. So, you know, we can talk about how this how this makes the sort of exclusion restriction, right, the assumption that investigators only influence outcomes through their placement decision. How this makes this assumption more reasonable in our setting. But you're exactly right. Once that decision is made, any other decision, once the child is in foster care, for instance, the type of placement, the length in foster care that's completely outside of the control of the initial investigator. 

 

Jennifer [00:14:19] Okay. Yes, we will talk lots about the exclusion restriction later for those who are listening who love talking about exclusion restrictions. All right. So tell us how exactly you used the assignment of cases to investigators to measure the causal effect of placing children in foster care? 

 

Jason [00:14:36] Yes. So, you know, I think the design is quite intuitive, and I think there are two key insights that I mentioned so far that are going to really illustrate the intuition of our strategy. The first insight is that, as I mentioned, investigators are essentially randomly assigned to families, right, because they are not choosing which families to investigate based on their expertize or on the characteristics of the case. They are basically assigned to these based on who's next in the rotation, you know, which is effectively randomizing families to different investigators. The second insight is that investigators exercised considerable discretion in deciding whether a child should be placed in foster care or not. And for a variety of reasons, such as maybe their own personal experiences or views of the foster system, some investigators are going to be much more likely to recommend placement than others. As a result, if you think about it, children that are assigned to such strict investigators, these investigators that are much more likely to recommend placement. These children are going to be much more likely to be placed in foster care, then had they been assigned to a more lenient investigator. Okay, so an investigator that is much less likely to recommend foster care on average because they say, you know, the investigators personal abuse or experiences. So because this essentially mimics random assignment, you can think about in our empirical strategy in the following, we are simply going to compare the probability that a child assigned to a strict investigator is eventually arrested, convicted or incarcerated as an adult to the probability that a child assigned to a more lenient investigator is arrested, convicted or incarcerated as an adult. Okay, so essentially we are going to attribute any differences in the probability of criminal justice contact to the fact that they were assigned to different investigators given how closely our setting mimics random assignment. 

 

Jennifer [00:16:35] Right. And then you're going to argue that that - the assignment into investigators is really random assignment to placement in foster care. That exclusion restriction we'll talk about later. 

 

Jason [00:16:44] Yes, that's exactly right. 

 

Jennifer [00:16:46] Okay, great. And so what's the counterfactual we should have in mind here. If someone's not removed from the home - so if they're assigned to one of these, if they're assigned to one of these lenient investigators, and so they're less likely to be removed from the home - what happens to that family? What would sort of be the - in these marginal cases would nothing happen or would the family still typically receive some services in this setting? 

 

Jason [00:17:09] That's a great question. So, you know, generally many of the families that are not placed because we are thinking about the margin here and we'll talk a little more about who these children at the margin are and what it means for our estimates. But because our counterfactual are children at the margin who were not placed, this means that these children were probably of similar risk to children who were removed. Right. And so this means that these families are still going to receive community and targeted based services. So these families, while their children will not be removed, they will still be candidates for again, the light touch services that I describe, but also these more targeted services, such as parenting classes, substance abuse treatment and so on. So these are the children that you should think of as being the counterfactual for foster children in our sample. 

 

Jennifer [00:18:02] And what are the mechanisms through which foster care might affect kids long term outcomes in this setting? 

 

Jason [00:18:09] So I think there are multiple mechanisms through which foster placement could impact adult crime. And really, you know, it is unclear before looking at data whether foster care would increase or decrease the probability of a crime. You know, this is part of why there is a big debate about the effects of foster care on adult crime because it is just not quite clear, right. So on the one hand, it could be that foster care makes children safer than they would otherwise be if they remain in their home. As a result, children may be less likely to be abused and as a result could experience improved mental health outcomes and may be less likely to commit crimes as adults that stem from sort of mental health concerns. Similarly, foster care could lead to the improvement of neighborhood sensors. Right. So if foster children move to better neighborhoods or schools as a result of placement, they may be exposed to fewer crime prone people. And so they may be less likely to commit crime. So, you know, there is really good evidence from some of the work by Larry Katz and and the work by Eric Chyn that moving to better neighborhoods can really decrease the probability that youth will have contact with the criminal justice system later on as adults. Right. And so if children are moving to better settings, this could decrease the probability that they commit crimes later on. 

 

Jason [00:19:25] But there's also another story, right. On the other hand, the removal of children from their families and communities could be a very traumatic experience for these children. This may lead youth on a path of disengagement from the system and engagement with the juvenile and the criminal justice systems. Similarly, foster care children may face increased surveillance and increased likelihood of contact with the juvenile and criminal justice system. So there are countless anecdotes of foster families calling the police from minor incidents with foster youth that end up with these foster youth having contact with the juvenile justice system early on. Right. And you know, the type of setting in which children are placed also really matters for thinking about mechanisms. So just to give you an example, if children are placed in a group home, for example, they may be exposed to other children who are themselves victims of previous abuse or neglect and who may have behavioral problems, right. Such negative effects could lead to criminal capital formation while in the group home and lead to increasing the probability that a child will engage with the criminal justice system as an adult. So such negative peer effects have been shown to be very important in settings like juvenile justice facilities, for example in work by my colleague at Baylor. Well, this could also be important in setting such as group homes. So, you know, in some, there are sort of many different channels to which foster care could impact the probability that a child is arrested when the child is in adulthood. And it's not really clear what direction you know those mechanisms go. 

 

Jennifer [00:21:00] All right. Let's talk about the data. Tell us about this amazing data set that you're using to study all of this. 

 

Jason [00:21:06] Yeah. So we have really assembled what I think it's a really nice data set to answer all sorts of policy questions. But what we are doing is we match the universe of public school records in Michigan to child welfare reports, to juvenile justice petitions, and to adult criminal justice records in Michigan. Okay, and so, you know, this is the work of many, many years of data sharing agreements and kind of negotiation by people that I mentioned before, like Brian Jacob and Joe Ryan of the University of Michigan. And I was sort of part of creating these data set during my time in Michigan, during my post-doc last year. And so for this particular paper, we're going to analyze the outcomes of over 120,000 child maltreatment investigations of school age children between 2008 and 2016. And we are going to match these children to their criminal justice records. You know, up until this last year. We are going to be able to measure outcomes such as whether the child was ever arrested, convicted or incarcerated by age 19. And we are also going to be able to look at intermediate outcomes using our education data. So we're going to be able to look at how policemen influence educational outcomes like test scores or grade repetition and also using the child welfare data, we are going to be able to look at whether the child was safe. Right. And so I think that's something very important to keep in mind throughout our conversation that while foster care may have other benefits that we'll discuss later on in terms of criminal justice contact, educational outcomes and so on. The primary goal of foster care is to keep children safe. Right. And so we are going to be able to measure sort of subsequent safety using child welfare reports. So this is the data set that I'll be using in this paper. 

 

Jennifer [00:22:54] And kudos to Michigan for working with these researchers to allow this data linking to happen. 

 

Jason [00:23:00] Yes, they're great. 

 

Jennifer [00:23:01] Yeah. Okay, so tell us a bit about the cases that you have in the sample. What are the characteristics of the kids and families and the alleged maltreatment in both the overall sample and in the sample of kids who were removed from their homes? 

 

Jason [00:23:16] Yeah. So I guess I'll start by describing how the sample of investigated children looked like compared to the overall child population in Michigan. And then I'll talk a little bit about how foster care kind of makes those differences even more pronounced. So children who are investigated are substantially more likely to be black and lower income. They are also more likely to have substantially worse educational outcomes in the year prior to the investigation. So, you know, in the data, we see that they have much lower attendance rates, much more likely to have an individualized education plan, and they're also much more likely to be retained. Now, just to give you a sense of sort of prevalence of adult criminal justice involvement among the sample, 15 percent of children who were investigated for child abuse or neglect will be arrested by age 19. Eight percent will be convicted of a crime and six percent will be incarcerated. Okay, so these are much higher rates then sort of what we would see in just the overall sample in Michigan, which is about a six percent chance of of being arrested by age 19. In terms of the foster care sample, these differences are even more pronounced. Okay, so this sample is even more likely to be mostly black, to be mostly low-Income and to have again, much worse educational outcomes. They are also much more likely to have had previous contact with child welfare. And again, just to give you a sense of how drastic the criminal justice involvement is among this group. One in five foster children placed will be arrested by age 19. And roughly one in 10 will be convicted and incarcerated. Okay, so these are very sad statistics and just kind of confirm the findings of, again, that sociology and social work literature showing that foster children have substantially worse outcomes compared to even children who were investigated for abuse or neglect, but who are not placed. 

 

Jennifer [00:25:08] Right, and so the question - the key question here that we want to know the answer to is, is that a causal effect? And so that's what your strategy allows you to measure. And so let's talk about your results. What do you find is the effect of foster care placement on kids future criminal justice involvement? 

 

Jason [00:25:25] Yeah. So in short, we find that children on the margin of placement are going to be substantially less likely to be arrested, convicted or incarcerated by age 19. Okay, and we find very large effects. So let me get more specific on the point estimates. We find that foster children are twenty five percentage points less likely to be arrested by age 19 relative to children at the margin of placement who are not placed. This represents a 65 percent reduction, okay, relative to the baseline. Now placement also decrease the probability of conviction and incarceration by 29 and 20 percentage points, respectively. And just to give you a sense again of the percent magnitude, this corresponds to a decline of roughly 80 percent relative to the baseline. Now we see similar effects on the probability of being arrested on a misdemeanor and a felony offense, or we don't see much difference there. And we also find large reductions across all types of crimes, including violent, drug and property crimes. But really, the effects are largest and most precise for violent crimes. Okay, so suggesting large sort of reductions in social costs, given how closely violent crimes are. And finally, I want to mention that we see also a large reduction in the probability that a child is ever placed in a juvenile detention center. Okay. So we not only see reductions in adult criminal justice involvement, but we also see reductions in the probability that a youth who was placed has contact with the juvenile justice system as well. 

 

Jennifer [00:26:54] Okay, so foster care, has not just benefits, but huge benefits. Those are huge effects. 

 

Jason [00:26:59] Yes.

 

Jennifer [00:26:59] Which is really amazing. So as we were discussing briefly earlier, these results are going to apply to those kids that are on the margin. So the kids for whom, you know, a different investigator might have made a different decision. So, you know, you're just you're just on the margin of being removed your home or not. So we called that group the compliers. So who are the compliers in your analysis? What types of kids are these results relevant to? 

 

Jason [00:27:25] Yeah. So I'll kind of tell you exactly the characteristics of compliers, but I think this point is just so important to get across to listeners, right. So, you know, our study cannot speak to foster care effects for individuals for which say all investigators would agree that they should be placed. So, for example, because the abuse was so severe. We also cannot speak to the effects for individuals for whom no investigator would want to place. Right. So these are maybe allegations that were so minor that no investigator would place. Nevertheless, we think in our setting, this is exactly the population that we care about, right. So in practice, debates about foster care are really never about black and white cases. So these are most often about these gray cases where it's not so clear whether the investigator should remove or not remove the child. And so we think that our estimates are sort of widely relevant for policy from this standpoint. 

 

Jason [00:28:22] Now, in terms of who these compliers are, these compliers are going to make up about five percent of our sample. They're going to be disproportionately students of color and lower income students, and they're going to be much more likely to be placed due to an investigation that involved at least one allegation of parental substance abuse. And we'll come back to why, you know, this is so important for thinking about the mechanisms of why foster care impacts long-term outcomes. But compliers are also much more likely to have previous child welfare involvement and to have much more stable and shorter foster care placements. Finally, they're much more likely to be reunified with their birth parents. Okay, so we find that upwards of 70 percent of compliance in our sample are reunified with their birth parents after about 17 months. Okay, so foster care is a relatively short intervention for a complier children, and these children are also much more likely to come back to their birth parents. 

 

Jennifer [00:29:20] And you're able to look at some other outcomes in order to dig into potential mechanisms a bit. So what do you find when you do that? 

 

Jason [00:29:27] Yes. You know, I've been alluding to this through kind of the description of the compliers. But we find strong evidence in our study that parental improvement, while children were temporarily away from their homes, is a likely explanation for the observed improvement in children's long term outcomes. Okay, so I want to upack this a little bit because I think it's so important to get at mechanisms here. So we're going to base this claim on a few key findings. The first I just mentioned, right. So compliers in our sample, we're in the foster system for about 17 to 19 months on average, after which the vast majority of them were reunified with their birth parents. Now what are we going to do next is we're going to use our matched education child welfare data to think about how foster care impacts key intermediate outcomes, such as safety and educational outcomes, such as test scores. In examining the time patterns of these intermediate outcomes, shows us that differences in these outcomes we're actually quite small between children placed in those, at least for about the first two years following the investigation. But foster children experienced large gains in safety and educational outcomes, such as test scores in the years following reunification. Okay, and so we think that a likely explanation for this time pattern of the effects is that birth parents who work closely with social workers and the court's following child removal improve their parenting skills and address concerns in their own lives, such as substance abuse. 

 

Jason [00:30:56] Okay, and there are actually several institutional features in Michigan that are going to support this change. So first, after their children are removed, birth parents work closely with social workers to address challenges in their own lives. Right. So birth parents are going to receive fully funded services to help with these challenges, such as substance abuse treatment, parenting classes or counseling. Now, the fact that compliers are disproportionately likely to be referred for cases where there is a parental substance abuse allegation further supports this trend. Second, a judge in Michigan actually needs to approve that it is safe for children to return home before they can be reunified with their birth parents. And so we think that is really incentivize parents to comply with these services. And finally, we actually are going to show in the paper directly that there was sort of statistical evidence of parental improvement. So we show that birth parents were much less likely to abuse or neglect other children even years later if their initial child victim entered foster care. We are going to find much less evidence for some of the other explanations that I put forth earlier in the podcast, such as moving to better neighborhoods or better schools, given this sort of temporary intervention of nature of foster care. 

 

Jason [00:32:12] And so I just want to summarize by saying that many people do not realize that foster care is sort of a family oriented intervention and it's also a temporary intervention. Right. So this is, I think, really important to get across in our study. 

 

Jennifer [00:32:26] Yeah, I think it's really interesting in light of what we were discussing earlier about kind of what the counterfactual is, what this - what these families would receive if they were on the margin, but had a stricter investigator who removes the kid versus what they would have received if they had a more lenient investigator and the kid was not removed. So these families would have received, in all likelihood, all the same types of services. But it really does seem like removing the kid does provide either the space and time for the parents to really focus on their own recovery or the incentive to really do it. Is that how you're interpreting those results? 

 

Jason [00:33:06] That is exactly right. And so, you know, we think this is just so important because it suggests that, you know, if we can find ways to incentivize the same type of sort of compliance in these services. And if we can find ways in which we can kind of help parents work through some of the challenges in their own lives, then we might be able to safely reduce the use of foster care, right, which is, we'll come back to this if we talk about policy implications. But there's a big push to reduce foster care, you know, across the field, by practitioners, by scholars, most recently by the federal government and really understanding how we can keep children safe at home, you know, without foster care, I think it's a crucial next step in this research. 

 

Jennifer [00:33:55] Yeah. Okay, so this is an instrumental variable analysis, and so researchers listening to this, they're probably wondering about the exclusion restriction. So a key assumption here is that your treatment variable in this case and investigators tendency to place children in foster care is correlated with the outcome only through its effect on whether a child is placed in foster care. That's the exclusion restriction. So since you have random assignment of investigators to cases, we're not worried that investigators preferences or skills are correlated with case characteristics. That would be the usual big problem you'd have to overcome in a correlational analysis. But it's still possible that the investigators influence the cases they handle in other ways than just that foster care decision. So you investigate this a bit. So tell us what you do and how you convince yourselves that this exclusion restriction is plausible in this setting. 

 

Jason [00:34:46] Yeah. So we do our best in the paper to really, really get at this assumption. But I actually think that this assumption is quite reasonable in our setting. And so let me give you an example. A potential concern here would be that, for instance, investigators who vary in their tendency to remove might also influence children's experiences while in foster care. So as an example, if a particularly strict investigator is also better at, say, getting children to reunified with their with their parents faster, then this would violate the exclusion restriction. A setting of Michigan that is very useful for us, however, is that investigators do not work with children after the investigation, so cases that end up in foster care are actually transferred to other child welfare caseworkers known as foster care workers in Michigan. So investigators are actually not involved in determining where children are placed, how long they're going to remain in foster care or even the stability of their placements. 

 

Jason [00:35:41] Right. So in the paper, we show kind of statistical tests showing that investigator tendencies are not correlated with indicators of children's experiences in foster care, such as, you know, length in foster care, the type of placement and so on. You know, I think Max and I always like to be really upfront about other assumptions that we cannot test, right. And so inherently, the exclusion restriction is untestable. And so we rely on these statistical tests and kind of arguments that we believe it is. But you know, it is plausible that investigators may affect children during the investigation in ways that could potentially impact outcomes. So let me give you an example. So relatively strict investigators, for instance, could vary in their say sensitivity to a family's schedule or in the way in which they conduct themselves during the investigation process and this could impact children's outcomes. Right. So even with detailed survey data on family experiences, we really would not be able to address all of the potential channels through which investigators could impact children. So we think that this assumption is plausible in our setting. You know, we do our best, but at the end of the day, I think it's it's very useful to be honest about what we can and cannot rule out in this setting. So, you know, in short, we do our best to probe it. It seems like it's reasonable in our setting and we pass all of the usual statistical tests. But you know, there are some things that we are not able to empirically show in the paper. 

 

Jennifer [00:37:08] Yeah, that's I think it's a great caveat. This is an assumption that one can never test directly. 

 

Jason [00:37:14] Exactly.

 

Jennifer [00:37:15] Yeah. I mean, I think just on your last point that there might be something about the way that the type of investigator who is strict versus lenient, you know, interacts with the family that could be different, that happens to be correlated with that foster care placement propensity. I think then you wouldn't expect to see the time pattern in the results. I think that you see right, the fact that it's it seems to be after the kids are reunited with the families that you really see the big benefits. You kind of expect to see the effects immediately if it's something about the interaction rather than the foster care placement, I think. 

 

Jason [00:37:46] Yes. Yeah. I mean, we think that's exactly right. 

 

Jennifer [00:37:49] Yeah.

 

Jason [00:37:49] You know, it was very difficult to kind of empirically test. 

 

Jennifer [00:37:53] Sure. Yeah. 

 

Jason [00:37:53] But I agree with you. I think the time pattern of the results is kind of really revealing about, you know, which are plausible mechanisms in our setting and which are not and parental improvement, really from everything we've seen really seems to be the main driver of our results. 

 

Jennifer [00:38:11] Yeah. Okay, let's go back to the Joe Doyle papers that you mentioned previously, which really for a long time had been, you know, viewed as sort of, you know, what we know about the impact of foster care. So it used basically an identical empirical strategy to what you all do, this random assignment of cases to caseworkers to measure the effect of foster care in Illinois between 1990 and 2003. And as you said before, that paper found that placement in foster care increased the probability of a future arrest. So basically the opposite of what you're finding in this paper, and I really liked that in your paper, you did a bunch of work to try to understand why these studies are reaching such different conclusions. So walk us through what the possible reasons for these differences are and how you determine which are the most important. 

 

Jason [00:38:59] Yes. So, you know, we've been thinking about this for a long time about, you know, the Doyle papers are excellent. And so it was very surprising to sort of find such drastic results. Right, not only does Joe Doyle find, you know, negative effects of foster care, he finds extremely large negative effects of foster care. And we find, like you said, just the opposite very large positive effects of foster care. And so I think one of the challenges of our work in the social sciences right is that estimates can vary so drastically by - so, you know, there's so many possible reasons why our findings in Michigan in the last decade could vary from, you know, the findings of Illinois in the late 1990s. And in the paper, we are going to kind of think about a bunch of possible different reasons and try to do our best to give the reader a sense of what we believe is going. And so we are going to start by asking sort of more simple questions like is the sample composition different across two studies or whether compliers may be there. Right. So Joe Doyle has these really nice tables in his paper where he has sort of the characteristics of compliers and how they compare to his overall sample. We have similar tables in our paper. And so we're really able to take a look at whether differences in the study may be driven by differences in sample construction or complier characteristics. We don't think that's the case. So our compliers tend to look quite similar to those of Doyle. Our sample construction tends to be quite similar. We even do an exercise where we try to construct our sample even more similarly to the way in which Joe constructs his and our results look similar to those in our main analysis. And so, you know, we kind of rule these two explanations out the best that we can. 

 

Jason [00:40:43] What we believe is going on here is that we think there are very large differences across the foster care systems, across the two states and two study periods. Okay, so let me on unpack this a bit. I think this is very, very important for kind of future research and the way in which we can view this literature. We show in the paper that the foster care system in Illinois at the time that Joe studied the system was not very representative of the average foster system at the time. So specifically, placements in Illinois were considerably longer and less stable than pretty much any other state at the time. Meanwhile, Michigan is actually a quite representative state in terms of these two key foster care indicators, right. So Michigan looks very similar to the average state in terms of placement length and stability. And so it is possible that some of the differences in our results are driven by the stark institutional differences across the two settings. So again, the fact that placements are longer and less stable and the fact that we find that most of our positive effects could come from the fact that foster care is actually a temporary intervention while parents sort of improve, you know, this could be largely responsible for the differing results. 

 

Jason [00:41:56] But it is also possible that legislative and cultural changes to child welfare practice over time, such as shorter stays, increased placement with relatives improve foster systems nationwide. Right. So there has been a drastic shift in nationwide foster care policy to move away from placing children in group homes settings, other congregate care settings and to place them with relatives instead. And also, there's been a really large shift toward achieving reunification faster toward making placements more stable. So to us, what looks like the main driver of the differences in our results is sort of these stark differences across institutional settings. Right, so we're examining a setting in which placements are shorter, placements are more stable, group home placements are less likely. And we just think that given how representative Michigan is of sort of other states right now, that this indicates that in a typical foster care system, one might not see the large negative effects that Joe documented in Illinois in the late 1990s. 

 

Jennifer [00:43:00] So one quick clarification so by more stable placements, you mean the kids aren't bouncing around between foster homes? Is that right? 

 

Jason [00:43:07] That's exactly right. Yeah. So we measure kind of placement stability as the number of placements that a child may have while in foster care, right. So the more placements, the less stable. 

 

Jennifer [00:43:17] Yeah, Okay. So, yeah, so stays are shorter and more stable now. So I mean, it strikes me that, you know, all of those changes could be due in large part because of the research that Joe Doyle did right? Those were very influential papers that I think made a lot of people realize like, wow, our foster care system is really not working. So even if it was unrepresentative at the time, it still was kind of helped push the shift, perhaps. And so one way to interpret this difference is it worked right. Like the changes that that research and other research on foster care had been pushed for or the changes that the research led us to to want in large part has been achieved. And now the foster care system is working much better than it had in the past. Does that sound right? 

 

Jason [00:44:04] Yeah, I think that's absolutely right. The Doyle papers, I mean, we talked to a lot of sociologists and social work scholars at Michigan about this project, and everyone knows the Doyle papers. They're very well-regarded. And I think what has made me very interested in this topic also is these are setting where policy actually, you know, does listen to a lot of what researchers have to do. So of course, we know that's not true of every kind of policy decision. But you know, we've been part of numerous conversations with policymakers in Michigan. And, you know, sociologists are often part of of numerous conversations with policymakers thinking about how to make our foster care system better and more fair to and more effective for at-risk children. So I agree with you. I think the Doyle papers were excellent and they've been, you know, very influential in the field. But what I do think it's it's important to sort of evaluate the way foster care looks now and, you know, sort of get rid of some of the typical misconceptions about how bad our foster care system is, right? 

 

Jennifer [00:45:11] Yeah. And it almost seems like maybe we've gone too far based on your results and should be shifting back a little bit - like those kids on the margin, it seems like we'd actually want investigators to be like a little bit more like strict investigators that remove kids more often because those kids on the margin are doing so much better than those that are not removed. So we can talk more about that. Before we get into policy implications, have any other papers related to this topic come out since you and Max first started working on the study? 

 

Jason [00:45:38] Yeah. So there's you know, in the last five years, there have been so few studies seeking to reexamine the causal effects of foster care. So there's a really nice paper in Rhode Island by Anthony Bald and Eric Chyn and other coauthors that finds that foster care placement increases test scores and reduces rate repetition for young girls, but no discernible impact for young boys. So, you know, I think that heterogeneity is really, really interesting. That's a very nice paper by the authors. There is also a paper by Kelsey Roberts, who is a Ph.D. student in economics at Clemson. And this was kind of her job market paper, and she actually found similar positive effects on grade repetition. And finally, you know, our own work in Michigan looking at the effect of foster care placement on short term outcomes, showed that foster care can have kind of positive effects on educational outcomes like test scores, attendance and so on. Our paper is forthcoming in the AEJ Applied. And so all three of these papers use the same research design. There are sort of various differences that we detail in the paper about data availability and the context. But these all seem to point to a more positive effect of foster care than the early Doyle papers had. And so again, I think this kind of reinforces, Jen, what we were just discussing that, you know, it could be that, you know, the foster care system has improved drastically over time. 

 

Jennifer [00:47:01] Yeah. So what should policymakers and practitioners take away from all this. What are the policy implications here? 

 

Jason [00:47:08] Yeah. So we are very careful here in policy implications. I don't think our results mean sort of go out and remove more children. Right. So I don't think anyone is sort of in favor of the traumatic experience that family separation can bring. But I think there is a big implication here that there is this big push in the field to reduce the use of foster care. So these across practitioners, scholars and as I mentioned, even the federal government just passed a law. The Family First Prevention Services Act, which took effect in 2019 that literally incentivizes states to move away from out-of-home placement and to instead focus on prevention services that are focused on family preservation. 

 

Jason [00:47:51] So the way we view our results is that, well, this current prevention services are falling quite short. Right. And so because we are comparing children at the marginal placement who placed to those who were not placed, the way we view our results is that children who remain in the home are much more likely to be abused again, to have worse educational outcomes and to commit crimes as adults. Right. So if we want to reduce the use of foster care. Then we need to make sure that we invest and that we rigorously evaluate prevention services, right. So reducing foster care without adequate in-home abuse prevention is just too costly. Right. And we just cannot afford to do that for the well-being of children. And, you know, even beyond the well-being of children, the enormous cost that you know, a future crime, and, you know, just broad criminal justice involvement brings to our society. So, you know, I think this is the big takeaway from our paper that we really need to invest in finding out which in-home prevention services are most effective at sort of keeping kids safe and thriving in the home. If there is going to be this big federal push to reduce out of home placement. 

 

Jennifer [00:49:06] Right. And as we were talking about earlier since the families on the margin where the kids weren't removed were still receiving all the same services that the families whose kids were removed had gotten, you know, in some ways you are testing that directly here, at least in terms of the status quo services and all of that. If it is something about, you know, the parents having the time and incentive to make use of those services, then yeah, we're going to need a lot more experimentation and investment to figure out what works there because the status quo doesn't seem to be effective. 

 

Jason [00:49:42] That's exactly right. 

 

Jennifer [00:49:44] Yeah. If your goal is to keep kids in the home, which I agree is a worthy goal for many reasons. 

 

Jason [00:49:48] Exactly.

 

Jennifer [00:49:49] Okay. Well, so what's the research frontier then? I guess that's one big one. What are the next big questions that you have in mind for what researchers in this area need to be thinking about in the years ahead? 

 

Jason [00:50:00] Yeah. So one of the kind of extensions of our paper that we are currently working on that we are hoping to add to the paper is thinking about which types of foster care are most effective. Right. So I think there's a lot of nuance in foster care, and it is a complicated, very broad intervention. So it's not fair to kind of say, is foster care good or bad for children, right. So foster care can mean anything from the child is staying with his or her grandmother for six months to the child is in a group home for two years, right, with other children who were previously abused or neglected. And there is actually very little evidence on the relative effectiveness of distinct placement types and on other things like the effects of placement length, for example. So our study can really only speak to the average effect of foster care placement, but one can imagine that placement in a group home is going to have very different effects from placement with a family member. Right. So unfortunately, our research design doesn't allow us to kind of readily incredibly bring evidence to this question because, you know, placement happens after the random assignment of investigators. Well, we are currently thinking about ways in which we could credibly estimate, you know, heterogeneous effects by placement types, and I think this is just such an important question and something that we have such limited evidence on. And I really think there are big returns to figuring this out. So just to give you an example, the Family First Act that I just mentioned a moment ago actually incentivizes a reduction of group home placements. And this is sort of generally accepted by the field to be a good practice. But there is little to no empirical evidence on this. Right. And so bringing some empirical evidence to these questions, I think, could have large returns for the field of child welfare policy. 

 

Jennifer [00:51:51] My guest today has been Jason Baron from Duke University. Jason, thank you so much for talking with me. 

 

Jason [00:51:57] Thank you, Jen. This was great. 

 

Jennifer [00:52:04] You can find links to all the research we discussed today on our website probablecausation.com. You can also subscribe to the show there or wherever you get your podcasts to make sure you don't miss a single episode. Big thanks to Emergent Ventures for supporting the show, and thanks also to our Patreon subscribers and other contributors. Probable causation is now part of Doleac Initiatives, a 501(c)(3) nonprofit, so all contributions are tax deductible. If you enjoy the podcast, please consider supporting us via Patreon or with a one time donation. You can find links in our website. Please also consider leaving us a rating and review on Apple Podcasts. This helps others find the show, which we very much appreciate. Our sound engineer is Jon Keur with production assistance from Haley Grieshaber. Our music is by Werner and our logo was designed by Carrie Throckmorton. Thanks for listening, and I'll talk to you in two weeks. 

 

Jennifer DoleacInstrumental-Variables, Health, Courts