Episode 78: Sandip Sukhtankar

 

Sandip Sukhtankar

Sandip Sukhtankar is an Associate Professor of Economics at the University of Virginia.

Date: August 16, 2022

A transcript of this episode is available here.

Episode Details:

In this episode, we discuss Prof. Sukhtankar's work on Women's Help Desks in police stations in India:

“Policing in Patriarchy: An Experimental Evaluation of Reforms to Improve Police Responsiveness to Women in India” by Gabrielle Kruks-Wisner, Akshay Mangla, and Sandip Sukhtankar.

OTHER RESEARCH WE DISCUSS IN THIS EPISODE:

 

TRANSCRIPT OF THIS EPISODE:

Jen [00:00:08] Hello and welcome to Probable Causation, a show about law, economics and crime. I'm your host, Jennifer Doleac at Texas A&M University, where I'm an economics professor and the director of the Justice Tech Lab. My guest this week is Sandip Sukhtankar. Sandip is an associate professor of economics at the University of Virginia. Sandip, welcome to the show.

 

Sandip [00:00:27] Thanks, Jen. You already sound really, really professional.

 

Jen [00:00:32] This is my job. Today, we're going to talk about your research on women's help desks in police stations in India. But before we get into that, could you tell us about your research expertize and how you became interested in this topic?

 

Sandip [00:00:45] Sure. Yeah. So I actually I'm more generally interested in governance and public service delivery. So I came at this issue through this angle rather than, I guess, the crime angle. I mean, the police are, of course, one of the largest departments in the government. And in India, they tend to be like really, really under-resourced. I know in the U.S., people talk about things like defunding the police and they have fancy military grade equipment in India many frontline officers don't even have simple weapons like guns, for example. They have these lathis, which is basically like a fancy word for sticks. So in this context, I became interested in working with the police as a public service agency and one of the biggest issues that they are facing and they are dealing with is crimes against women. This has obviously gotten a lot of attention in the media, even in the international press and India is, of course, one of the places in the world with really high gender inequality. They are there are really high rates of crimes against women and it's not like the police are just sort of sitting there twiddling their thumbs.

 

Sandip [00:02:04] They want to do something about this senior officials desperately wanted to do something, but of course, this is a really complicated problem there's societal norms, there is sort of deep rooted patriarchal attitudes. So when I started working with them, my coauthors and I, we sort of started to ask them, okay, what is it that you've done before? What do you think might work? And for us, this idea, this academic angle of sort of whether or not representative bureaucracy works was was really interesting thinking about whether female officers can play a big role in helping to address this issue against crimes against women. So that's basically how I got into this topic.

 

Jen [00:02:50] Your paper is titled "Policing and Patriarchy: An Experimental Evaluation of Reforms to Improve Police Responsiveness to Women in India." It's coauthored with Gabrielle Kruks-Wisner and Akshay Mangla, and it was recently published in Science. Congratulations.

 

Sandip [00:03:05] Thank you.

 

Jen [00:03:06] It describes the results of a field experiment you ran in India, putting designated spaces that are mandated to respond to women's cases in police stations. So say a little bit more about the problem that you were hoping this intervention would address.

 

Sandip [00:03:20] Yeah. So one of the main things that we were hoping that it would address is underreporting. Now, of course, you probably know that this is an issue that exists all around the world. Underreporting of of crimes against women. Women don't approach the police don't want to talk about it, which is fair and I mean, obviously, there's a lot of reasons for that. In India, though, this problem is is just really, really severe. So they're obviously hard to estimate, but but the best estimates suggest that anywhere between 90-95-99% of cases of violence against women go unreported and that's for good reason.

 

Sandip [00:04:02] So there was a really horrible case recently in the news all over India where there was a 13 year old girl who went to the police after she was abducted and gang raped and then the head police officer that she went to complain to actually raped her in the police station. I guess maybe we should have some trigger warnings or something with this because it's obviously absolutely horrifying. Now, clearly, this is an extreme case, but you can imagine the kind of sort of patriarchal attitudes that exist among the police. They really need some kind of gender sensitization a lot of police sort of don't quite know what to do when women come in to ask for help.

 

Sandip [00:04:44] So, you know, we thought we would start with the simple things. So we worked with the police, a lawyer in Delhi High Court who has sort of experience working with the police and civil society organizations to sort of distill a few components that could be implemented, even with sort of all the under-resourcing and all the constraints that the police face. So the simple idea is to try and make the police more approachable to women. One other thing that's important to note about the context is women make up a really, really, really small proportion of the police forces so 7% of the police are women in the state that we are working with, I think that's roughly true of India as well and that that was true at least when we started.

 

Jen [00:05:29] And so before this experiment and the intervention you all were trying what other types of interventions had been tried before? And what did we know about whether they worked?

 

Sandip [00:05:40] Yes. So, I guess one sort of way to to categorize these interventions are whether they're gender segregated or mainstream.

 

Sandip [00:05:49] So there's a number of type of gender segregated interventions that had been tried. So one famous example is this all women's police stations. So these are police stations that are staffed entirely by women, and they're only for women. Now, there's a history of this type of segregated intervention to address issues of gender based violence. So, for example, one common thing, if you've ever been to India or traveled in some of the big cities, there are women only train cars in the Delhi metro or the famous Mumbai local trains, for example, have these women on the train cars. So, of course, the issue with these all women police stations is they're really hard to access. When we started our experiment, there were only 11 of these serving the entire state, which is 80- 90 million people even now, they've expanded some of the some of them, but even now, at most, there's about one police station per district, which is about 2 million people.

 

Sandip [00:06:51] So if you, for example, are a woman who doesn't live close to this all women's police station, it's going to be really, really difficult to get to and really, really difficult to get help. Now, the problem, and this is some research has actually shown this, is that what happens then is that if you as a woman go to your local regular police station, the police there say, hey, wait a minute, this is actually a men's station you have to go to the women's station to get help. Right, which, of course, is really, really difficult it's hard to get to your it's obviously they're constrained there because they don't have enough staff to to handle everybody that comes. So this is one of the types of interventions that they've tried, which, you know, there's some evidence that that this may have led to more cases being registered, but there's also some evidence that that this displaces a lot of cases and and then it segregates women's policing into this.

 

Sandip [00:07:45] Okay. You can only access help from these old women's stations. Now, previously, the [00:07:52]Amati [0.0s] British police that we were working with had tried the women's help desk intervention, but it was just sort of this one page circular government order, they call it, that says, hey, go established these desks without any kind of support or explanation, but this time they were hopeful that, look, if we do this in a more systematic way, which we actually explain to people what has to be done, trained people, put in some resources, this might be helpful.

 

Sandip [00:08:22] Now, there's other types of interventions that people have tried as well that include these all women's police cars, their patrol cars that go around. This is obviously, as you can imagine, it's a little bit hard to study because they're they're mobile. You'd have to figure out how exactly to define your sort of treatment and control areas of treatment and control groups. So establishing any kind of causal impact of something like this is going to be very difficult. So that's sort of the broad state of the world when we started.

 

Jen [00:08:57] And so why don't we know more than we do about how to solve this problem? Already, you mentioned, you know, part of the issue about figuring out what's a treatment and comparison group in some settings. Walk us through the main challenges that researchers need to overcome in order to figure out how to increase reporting and reduce the incidence of gender based violence.

 

Sandip [00:09:18] Yes, I mean, there's just so many hurdles. So one one, as I just mentioned, is that some of the interventions don't necessarily lend themselves to a clean separation of treatment and control.

 

Sandip [00:09:31] Some of them are implemented somewhat haphazardly, or they may be implemented across the entire state or across the entire country, even at once. So, for example, there's a lot of different states have these and they're called Nirbhaya vehicles that are these were women driven cars that sort of go around these kind of hotspots to deter crimes or things like catcalling, that kind of stuff, so that's one issue one sort of broad set of issues. The second sort of broad set of issues is the police themselves tend to be extremely wary of researchers. They don't want to share data or access. So we were very lucky. We had a very responsive and open counterparts in the police who actually wanted to do this, knowing that there was a possibility that the experiment would show that nothing happened or bad things happened, but they were willing to do that, but in general, police don't tend to be that. They're extremely cautious. They don't want to be made to look bad. They don't want to share data because it might make them look bad. So that's one sort of big issue.

 

Sandip [00:10:42] And then finally, I think one really particular issue in this area is that you're going to have to collect a lot of your own data. Right. So so as I mentioned previously, there's this huge underreporting problem. So you can't you're not going to be able to rely on administrative data on its own to see what's happening to crime rates, for example. And again, I'm sure this is an issue and other crime research as well, but for us, for me, I guess somewhat new and in the sense that the gaps are just so large that there's really not very much a meaning in the registration data in terms of what's actually going on with crime.

 

Sandip [00:11:24] So you're going to have to collect data on at various stages, right? So are people reporting more off the cases that are reported? Is registration actually going up or not? And then eventually, of course, and all of this changes is actual crime rates. So collecting all of this in a representative manner is is obviously very difficult. And so that's that's another sort of big, big challenge.

 

Jen [00:11:51] Lots of challenges. Okay. So in this context, what is the process that police are supposed to follow when a woman reports gender based violence? And what did you know going in about how often that process was actually followed?

 

Sandip [00:12:05] So, yes. So in this context, the police can do a number of things so they can find two types of cases. So one is sort of the regular bread and butter criminal cases or first information reports or FIRs, as we often use the acronym in the paper. Those are based on the Indian penal code and those are sort of classic cases of violence in India. Those also include things related to dowries. So pressuring someone to to pay a dowry is a crime. There's a lot of harassment of women related to dowries and so those can be crimes as well. And then there's all the other types of violence against women that we might see in other places, too. So those are criminal cases. Now, you can also file civil cases of domestic violence. This is very interesting because this is somewhat new. In fact, that's one of the things that we're going to see in our data is that before the experiment, basically there were zero cases of civil cases of domestic violence that were reported. These are called domestic incident reports or DIRs.

 

Sandip [00:13:17] Now the police really should be able to know to file them and the interesting thing about this is that it sparks a bunch of inter-agency coordination. And it if you're filing this kind of DIR, lets women access a lot of resources so they get resources from the women and child's department, women and children's department. They can get access to legal help. They can get access to shelter. So it's actually really useful, even if the police are not filing FIRs, to file these domestic incident reports, which will then allow women to access a lot of these resources. So for many, many reasons, we knew that these processes and these registrations were not happening. So with that domestic incident reports, the simplest thing was that people just didn't even know that this existed. Even though the law was passed in 2005, not many people are aware of it and not very many people know how to do it. It's slightly complicated in that like you can't literally file it at the station. You have to bring it in front of a magistrate, but you can do most of the paperwork in advance.

 

Sandip [00:14:27] So this was just a sort of knowledge gap. Then the big issue with the criminal cases, the FIRs is that even though you really shouldn't be using these to measure crime rates, the reality is that people do. And so if you register more FIRs, you might get a media report that says, Oh, hey, crime is going up in this area, even though crime actually hasn't gone up. It's just that more of it is being registered. So the police are really, you know, unwilling to register because it makes them look bad. It makes them look like crime is is going up. It creates extra work for them because there's some legal obligations with once you file an FIR, you have to investigate.

 

Sandip [00:15:13] It also allows you to make arrests. So so in a way, it's it's a gating step and but it's obviously really, really important because without filing an FIR, you cannot access justice. You cannot have a court case, really a criminal court case without an FIR. So it's this sort of key step very, very hard to actually register and, you know, obviously once it's registered 98 to 99% of cases, then actually go in front of a court so there's a chargesheet filed and a court case happens. Basically then it's the court system and whether or not you actually get justice, there's lots of other things involved, but without this critical first step, you can't even reach the core straight. So that's sort of where we were.

 

Jen [00:16:03] Yeah. I'm curious. So with the FIRs, did you get the sense that that sort of incentive to not file a report on the police side, did that affect reporting for other types of crimes, too, or is there something about the female victims that sort of made them more vulnerable to this?

 

Sandip [00:16:20] No, it absolutely affects all other kinds of crime as well. And so there's papers written about this. In fact, there's papers not just in India, but in Pakistan as well, written about this, where politicians literally pressure the police not to file these cases so that they don't they don't make themselves look back. Now, it can go the other way, too. The politicians can pressure police to file cases against their opponents or to file cases when they want the cases to be filed, but for the most part, if you want to look at sort of the overall numbers, yes, there is a lot of pressure to not register and an FIR, in fact, some of the horrific cases that we've read about recently one of the things that came out was that even though these were such horribly egregious cases, FIRs were not filed until days later.

 

Jen [00:17:12] Wow. Okay. So let's talk about how you're going to solve all these problems. So tell us about the specific intervention you implemented. What are women's help desks?

 

Sandip [00:17:23] Yes. So women's health does have four simple components. We wanted these to be understandable and implementable, even across the the very large expanse. That is what they produce so Madhya Pradesh, is a huge state you know about 85- 90 million people, very large. You should think of this as basically France or Germany. It's sort of that big and so the four components were dedicated physical spaces. It could be a room or most likely just a cubicle, sometimes just literally a desk within a police station. So the idea is to just have a place where women can be a little bit more comfortable discussing what could be sensitive details. Right. And so that was very simple, very straightforward. People thought that this was kind of a a simple but no brainer thing to do.

 

Sandip [00:18:19] A second sort of simple, no brainer thing was was standard operating procedures, too, so that people the police know well, what is it that you should do when a woman approaches the desk. So this was in the form of simple checklists. This was in the form of cheat sheets. This was in the form of a tick manual for reference. And then part of this component was also training on these standard operating procedures and these guides and manuals. And that was done at various different levels for the officers that ran the desks, for the head of the police station and at the station itself, for everyone else, all the other police officers in the station.

 

Sandip [00:19:08] So so a number of different trainings on what to do. And so this, for example, would would help with, okay, what is a domestic incident report how does one file it so very simple things like that. As well as very complicated things about well is it better to file under IPC code 371 A versus 371, B, etc., etc. So that's component to a third component which was seemed really straightforward as well was, was just outreach to the community, to local women's networks just to let them know that, hey, we have this resource, please come to us and we will help you. So this is about some outreach events. You know, there's actually a lot of these events held. We have tons and tons of different photos, really, really fun, interesting photos of these types of events.

 

Sandip [00:20:00] Unfortunately, of course, the police stations also serve enormous catchment areas rates on on average one police station is serving about 130,000 people and so it's still hard to get the word out. So that's component three and then component four, which I'll talk about in a little bit when we discuss the actual evaluation, this was sort of cross randomized. So not all police stations that got a women's helpdesk got this component, which was a female officer assigned to run the Help Desk. So half, roughly half of the treatment group was basically assigned to have a female officer running the Help Desk. This is really interesting because even sort of the senior most woman officer who supported the intervention and the evaluation throughout herself was not sure about, well, whether women police officers are going to be responsive, are going to be sensitive to women victims and complainants, you know, given the huge sort of selection effects. Right.

 

Sandip [00:21:09] There's only 7% of officers that are women and so often the female officers face a lot of pressure to sort of behave like one of the guys act tougher than than the men. And so it's not necessarily true that these officers might be especially more sensitive to two other other women. So those were sort of the main components of the intervention.

 

Jen [00:21:35] So say a little bit more about who staffed these desks, how were the officers selected?

 

Sandip [00:21:39] Yeah. So these were you have to be a specific rank. So this is an additional subinspector or above. So mostly it was additional subinspector. So there's the lowest rank is the constables and you can become a head constable and then you have the ASI, the additional subinspector. So you have to be at least an additional subinspector and most for the men who are staffing the desk, these were basically you would have of the pool of ASIs that were in the station, but the head of the station had had a lot of basically discretion on, on who they chose to be the the helpdesk officer for the helpdesk that were assigned to have a female officer around the station not all stations necessarily had a female ASI already assigned to that station.

 

Sandip [00:22:37] Right. So, so they may have to get someone posted there so that woman could then run the help desk. And again, most of the time these were men would come from what we know from the district headquarters. So maybe you were sort of running more of a desk job at the headquarters and now you were assigned to the frontlines is sort of a very broad way of thinking about this. Of course, there's exceptions, but the one thing that we did not want to do is to have women be reassigned from control police stations to treatment police stations, since, of course, that would violate [00:23:18]Sati [0.0s] and all kinds of other good assumptions that we have for experiments.

 

Jen [00:23:23] In that case, the control stations would be treated to.

 

Sandip [00:23:26] Exactly, yeah.

 

Jen [00:23:27] Okay. So what are the various channels here that we should have in mind through which these help desks might affect officers behavior or victims behavior? What are the mechanisms here?

 

Sandip [00:23:40] Yeah. So I think, you know, we can go component by component. The space was just to sort of increase the comfort for the women in narrating their stories and once women know that the hope was that more women would come in to the police station so that now, again, they wouldn't have to be sort of narrating intimate details in front of absolutely everybody. So that was very simple in terms of the space. The training, again, was really straightforward and the hope is just filling in some of the knowledge gaps. Checklists has been shown to be effective in various different settings. So at least just going through it when someone comes in like, okay, ask them about X, Y, Z, have this, you know, ABC type of information on hand, here's the register, here's the manual, those kinds of things.

 

Sandip [00:24:35] We hope that it would it would make the police's job easier instead of sort of adding more paperwork, just streamline the process. So that was the hope with the with the training and, and the standard operating procedures. The outreach again, the idea was to let women know that there is help that's available for you here. There's some specialized services you can go and access this desk. You don't have to go through all the procedures in the main police station. There's some sort of set aside for you to get help. And again, now with the female officers as I said, it really wasn't clear, but the hope was that that women victims might feel slightly more comfortable.

 

Jen [00:25:21] And it was possible that the women officers might be more responsive. So this is, you know, goes back to broader ideas about about representative bureaucracy that have been discussed in the political economy literature for a long time. So not just in the case of the police, but other bureaucrats as well. Are you going to be more responsive to someone from your own end group? And so that's sort of the broad question. There's a lot of research, there's some mixed evidence and so we thought this would be very interesting to try and look at.

 

Sandip [00:25:53] Okay. So as you mentioned, you ran a huge randomized controlled trial. I guess you didn't mention how big it was, but it was a huge randomized controlled trial in order to measure the effects of these women's help desks. So where and when was this RCt conducted and how many police stations were included?

 

Jen [00:26:10] Okay. Yeah. So this was conducted in the state of Madhya Pradesh. So about 12 districts and in the states sort of those were not randomly selected. Those were picked by the police to basically the idea was to spread them out. So if you look at a map, they kind of picked them from all around the state. One thing that they did was to pick the four districts with the four largest cities. So this was definitely an urban skewed sample because their idea was that there's more women that come to these stations, more footfall in general to be able to justify having a sort of dedicated help desk at at a police station. So other than those for the other districts are roughly representative of the state. And within those districts, basically we had a sample of 180 police stations. The catchment areas of these are pretty large.

 

Sandip [00:27:08] So these 180 police stations serve about 23 million people, which is, you know, obviously huge and one way of thinking about the experiment is is at least 23 million people, if you expect that both men and women are going to be affected by this intervention or potentially affected. And then the timeline is, you know, this is a long sort of we can talk about the back story maybe a little later, but we started piloting and looking at this and starting the conversations with the police way back in 2017. We did a dedicated pilot in 2018. The training and all the sort of background resources started in early 2019, and we sort of consider that the actual intervention that fully started in May 2019 and then we were lucky enough to get in most of our data collection before the pandemic hit and that the lockdown happened in India at the end of March 2020. So we had a good ten months where the intervention was sort of fully operational. Of course, after after the lockdowns, things start getting a little bit messy. Of course, nobody was actually going to the police stations since you were not allowed to do that, that's starting about April 2020.

 

Jen [00:28:35] Yeah, that was very lucky. Like so many researchers had their RCTs completely thrown off due to the pandemic. So this is well timed. So, yeah, tell us a little bit more of the backstory here. You mentioned, you know, you were able to find these friendly police practitioners who were interested in these questions that open to research. How did this partnership come about?

 

Sandip [00:28:57] So I think it really worked because the police approached us. By us, I mean JPAL rather than us approaching them and so there was an officer, Vineet Kapoor, in the police who was not of not only an IPS officer, which is sort of the highest rank in the police, the broad rank, but also has a Ph.D. and so he was interested in research.

 

Sandip [00:29:24] He was interested in evidence based decision making and trying to introduce at least these ideas into the police in India, because he was of the opinion along with other people and other people that were his counterparts there, that the police, you know, make a lot of policy based on on anecdotes and experience and don't really look at the data as much and so he found a very supportive senior. So the director general of police of the state of Madhya Pradesh said this at the time was Rishi Shukla. He actually later went on to become the director of the Central Bureau of Investigation in India or the CBI, which is basically the equivalent of the FBI and then we had a very supportive additional director general called Anuradha Shanker, and she was the sort of the director of administration at that point, which is a very powerful post.

 

Sandip [00:30:24] Then became the director of training, which also helped in the intervention basically being able to run a lot of the trainings, etc.. So all of these officers were very supportive. They wanted to work with us. We identified a few areas of overlap. So the violence against women was not the only sort of area we they were interested in, in some community policing type interventions. They were interested in some organizational changes as well. So we tried to work with them, try to pilot all of these things, trying to come up with something that was of interest to both us and them and something that they thought they wanted to try, as well as something that that really ex ante wasn't obvious and none of this was was clear exactly in terms of, okay, this is going to work or this is not going to work or the female officers are going to work or the other desks are going to work or not.

 

Sandip [00:31:21] So. So that's I guess it's a good sort of juncture to run run an experiment when you're not sure what you're actually going to find. So then we piloted the intervention, so it looked like it was going to be worth studying. We agreed on an RCT, we signed an MOU, a memorandum of understanding with the police and JPAL and yeah, then that's it. So all of a sudden, one day they were like, give us the randomized list. Right now just sitting in a coffee shop, running data codes coffee shop at a hotel in Bhopal coming up with this code, of course, that, you know, it took a long time to get to that point and then all of a sudden they're like, do it today.

 

Jen [00:32:06] Let's go.

 

Sandip [00:32:07] Let's go out. And they had to come up with this hoping that there were no weird imbalances and it was okay. And then, of course, from that point, then it took them another six, seven months to actually ramp up the intervention, but.

 

Jen [00:32:21] Sure, sure. And then you wait on the research side.

 

Sandip [00:32:24] Then we just wait.

 

Jen [00:32:27] Yeah, this seems like a great opportunity to plug JPAL more broadly. So for those who are not familiar with that organization, there's JPAL that works in countries around the world and there's JPAL North Amerrica that does a lot of stuff, particularly in the U.S. and yeah, I mean, my experiences with them, they often fund RCTs teams like this one, but they also have an amazing staff that spends a lot of time talking with practitioners and policymakers that are interested in doing research, just basically trying to figure out like, is this feasible and is everyone on the same page about what this might look like. And then at some point, if everything sounds good, they loop in researchers like you or me to kind of go forward. Am I describing the process the way that it has worked for you as well?

 

Sandip [00:33:14] Yeah, absolutely. And in fact, just a little bit more on that back story. So Vineeth approached Iqbal Dhaliwal, who was one of the directors at JPL who used to be an Indian administrative service officer, and he was quite famous in India. And so Vinita knew him because of that and approached him and him and Abhishek Banerjee then sort of convinced me that I should at least go to Madhya Pradesh and talk to the police there, given what they had seen in terms of the interest of the police in doing some work with with JPAL, as well as just the fascinating, you know, issues related to the police. Right. So everybody in India, for example, really blames the police for a lot of things, but they are responsible for so much with so few resources. It's it's absolutely staggering. You know, people come to them for absolutely everything and anything and they're supposed to to resolve bizarre lots of it.

 

Sandip [00:34:12] I don't know if you have time for a funny story.

 

Jen [00:34:14] Sure.

 

Sandip [00:34:15] So one funny story when I was doing some fieldwork was this man comes in and I had been talking to the head of the police station and and he finally tells me, look, this guy's been waiting for a long time I really need to see what his issue is. I said, okay, that's fine I said, "Do you mind if I stay?" And he says, "No, that's fine." So the guy comes in and he tells the police officer, "Sir, my wife ran away." And when the police officer says, okay, starts asking some questions. It turns out that this guy's wife had run away with another man.

 

Sandip [00:34:48] Now, obviously it's sad for this person, but the issue is, what are the police supposed to be doing about this? I just think you're sitting there thinking, I'm really sorry for you dude I feel bad for you, but.

 

Jen [00:35:03] Not a crime.

 

Sandip [00:35:04] Not a crime. There are two consenting adults.

 

Jen [00:35:07] Right.

 

Sandip [00:35:08] And and then he has a lot of time listening to all his his history. And then he brings in his mother in law. And the mother in law comes in and backs up his story and says, "yes, my daughter ran away with another man." And again, I'm thinking, I'm really sorry for all of you, but like, how are you taking up the time off these guys have so much work to do with this case. And so this you know, this station house officer (SHO) had to spend a bunch of time sort of mollifying them and saying, oh, we can see what we can do, etc., etc.. In reality, he knew that there's really not much that can be done.

 

Jen [00:35:48] Yeah.

 

Sandip [00:35:48] But he has to at least talk to them. Right. So, so yeah. That's, that's, sorry, that was the funny side story. But yeah, this gives you a sense of sort of how people both blame the police for everything, but at the same time, they expect them to solve every problem they have.

 

Jen [00:36:05] Right. Just sort of like the level of responsibility or perceived responsibility on their plates. Okay. All right. Well, let's keep moving here. We want to eventually get to results. I imagine people want to hear what you found, but before we get there, what data are you able to use for this project?

 

Sandip [00:36:25] So yeah, so we wanted to be sure we collected everything possible given, as I mentioned, these issues with underreporting, etc..

 

Sandip [00:36:34] So obviously first and foremost was the administrative data on crimes registered. Both the criminal cases or FIRs and the civil cases or DIRs. So luckily, thankfully, since we were working with the police, sometimes this data can be hard to get, but they were able to directly give us these sort of de-identified, of course, cases aggregated at the police station level based on based on the type of crime, etc.. So we got that data then to see whether women were coming to the police station, were visiting the police station. Obviously you won't we wouldn't have been able to do that just from the registry registered cases. So we had one sort of clever way and it it being slightly less clever than that that we help for was to just use that.

 

Sandip [00:37:28] Every police station in India has a CCTV camera at the front entrance of the police station, so we thought we could just recaptured video feeds for about a week of the data, so they're already capturing those. So all we did was say, Hey, give us your feeds, and we are going to just run it through a machine learning algorithm where we teach it, Hey, this is what a man looks like and this is what a woman looks like. Thankfully, in India there is, you know, the clothes that men and women wear are very different. And so it's really fairly easy to identify. And so we use that to just see, hey, are more women coming to the police station? So that was one said it turned out to be way more complicated than we planned. Yes. So, yes, in fact, I still have yeah, I will still have to deal with some of the storage issues related to like terabytes and terabytes of video data that you get.

 

Sandip [00:38:30] So that was one. Then at the end line only we did just a quick user survey. So they think of this as kind of an exit poll. Hey, you went to the police station. You did just coming out. How was your experience? Were you satisfied? Where are you comfortable? Just two or three quick minutes where we didn't even collect people's names. Just just sort of a kind of quick user experience survey. Then we had some two very large surveys, one of police themselves at both control and treatment stations, where we asked them about their attitudes, their experiences, etc., etc.. And then a citizen survey, which, you know, it's interesting because none of our our main outcomes that we preregistered were really from the citizen survey what we absolutely needed it to be able to be sure that any changes in registration or reporting were not because of underlying changes and crime rates themselves.

 

Sandip [00:39:31] So that was sort of the main purpose of the citizen survey is to get a representative data on actual crime rates. Of course, this is easier said than done. It's it's hard to get these sensitive data, but we were hoping that at the very least we have a baseline. At the very least, we know that this is not going to be any different between treatment and control. So at least we have a sense of what the underlying crime rates were and how they changed over time.

 

Jen [00:40:02] Okay. And so what? Or the main outcome measures you're most interested in here.

 

Sandip [00:40:08] So the main outcome measures are sort of registrations of cases, whether more people are coming in to report cases. We had user satisfaction from the quick user survey. We are interested in some main outcomes on police attitudes to see if those were affected at all.

 

Sandip [00:40:30] And then as as a check, we want to want to see whether our crime rates are changing or not. So those are sort of roughly corresponding to the data sources. Those are the things that we were trying to get at.

 

Jen [00:40:43] Okay. The downside of running an RCT is it's a lot of work at the upside is it's easy to analyze the data, say here to say compare the treatment and control groups and look at the outcome measures across the two groups to see and the difference between them to see what the effects are. So what do you find is the effect of women's health desks on the registration of women's cases?

 

Sandip [00:41:04] Well, there's a couple of complicated things that we won't get into here. But one thing that I should mention is there's there are two ways to cut the data, right. So again, we thankfully with there's certain standards on how you preregister everything. So you're not don't have too much discretion exposed to change things around so so that's what we did we had everything pre-registered, but because there's three groups and there's two treatment groups and a control group, you can look at the treatment groups together or you can look at them separately. So for a throughout the paper, we report them both together and separately just to see sort of which of the groups was driving the change, if any. So in terms of registration, what we see, whether we look at the two treatment groups together or separately, we see a sort of dramatic increase in the registration of civil cases of domestic violence. These domestic incident reports or DIRs. So they're basically zero before the intervention and they're basically zero in the control group. And now there's about 1.5 cases per month per police station.

 

Sandip [00:42:13] Now the raw number might seem very small, but that's like a 1500% increase and it reflects about 2000 additional cases over the sort of ten, 11 months of of our study. Right. So that's that's a sort of big, big increase. And again, the rates and that the coefficients are very similar, whether it's the treatment group with the sort of regular women's help desks or it's the treatment group that had the woman run Women's Help Desk. So that was the that was sort of that the first dramatic. If you look at some of the graphs in the paper, it's just so nice and clean and it's just this huge sort of jump that you can easily see.

 

Sandip [00:42:57] You don't really need to see any sort of tables or regressions once you see those those figures. So now if you look at the registration of criminal cases, the FIRs, which is sort of the sort of standard bread and butter policing, so those also increased significantly when you look at the treatment groups together. So both considering treatment to be that you had any kind of desk. Okay. And so that's an increase of about 14% or three and a half thousand more FIRs filed. And now if you break down the treatment groups, so you look at them separately, this change is almost entirely driven by the treatment arm, which had the female officers assigned to run the help desk. So these women are really the ones who are driving the increase registration of criminal cases, which we thought was just absolutely staggering. And that was sort of the thing that probably surprised us the most in this entire experiment.

 

Jen [00:44:01] Yeah, that's really interesting. And so was there any effect using that camera footage? Was there any effect on the number of women who came to the police departments to report crimes or using your your survey measure? Was there any effect on the incidents of gender based violence?

 

Sandip [00:44:17] Yeah.

 

Sandip [00:44:18] So absolutely nothing there. Now, some of the CCTV footage, as I mentioned, was a little bit hard. We also brought it up against the the beginning of the lockdown just when we were about to collect the footage. So, you know, some of it is missing. Some of the video quality is not great. So although in theory it sounds fantastic, practically, it's just an enormous pain, enormous pain. We had to use the supercomputer here at the UVA to run the analysis because obviously you got processed that much video on a regular laptop. So we had to buy supercomputing space and processing units and all of that and after all that, luckily we're well ehat was interesting is that there is nothing that we see in terms of more women coming to the police stations.

 

Sandip [00:45:12] Now, you might think all but maybe just the data is crap, but it's actually not what's really interesting is, you know, there's correlation. If you just look at the correlations, the baseline data is correlated with the online data. And if you just look across police stations, the police stations that have more female officers overall as part of the station, we can actually see that in our CCTV data. Right, because we see more women entering in and out of those police stations. Obviously because there's more female officers, they're going to be coming in and out a lot and so that is correlated. So our data is informative, but it's telling us that no more women are coming to treatment stations. So that's actually not that surprising. But because in our citizen survey, if we ask people, Hey, have you heard of this great new intervention in Hindi, it was called the URJA Intervention. The intervention URJA is one of the clever acronyms that people in India love to use that stands for Urgent Relief and Just Action as the acronym.

 

Sandip [00:46:16] But URJA, the word URJA means energy in Hindi. So so that they had all these posters everywhere, these really nice sort of large posters and, and banners and all this. But only 10% of the women in our Citizen Survey report, even having heard of it and probably that's even an overestimate because I think they may be sort of possibly confusing it with other initiatives, but still so there and there was no difference between treatment and control groups on that measure either. So it's clear that women didn't really know about this as much as we would have expected and so of course, more of them are not going to come to the station. And again, if you look at the citizen survey, there was no actual increase in crime that we can detect, which also makes sense. And so more women are not coming to report these cases. All the action is taking place on off the women who who come to report more of their cases are being registered.

 

Jen [00:47:19] And then using the data from your exit interviews, you consider the satisfaction of the women who visited those police stations. What you find there?

 

Sandip [00:47:28] Yeah. So we find some, you know, everything is in the right direction. The sort of statistical significance levels are are not as high and as clean as one might expect in general if you look at it overall, it looks like there are sort of modest increases in reported comfort that the women were more comfortable in visiting during their visits what was really most interesting here is the heterogeneity. So we captured and again pre-registered measures of of training and implementation quality and what's really clean and significant is that all the stations, the better training and the better implementation quality overall in your police station, satisfaction rates are higher. So that's that's that's really interesting for us and that's sort of underlines the importance of the training modules in this intervention.

 

Jen [00:48:26] Yeah. And then finally, you consider whether police officers attitudes changed at all, did they?

 

Sandip [00:48:33] Simple big picture answer is no. And that's absolutely not surprising to us because as you can imagine, these are attitudes that are sort of deep rooted and so it's it's going to be hard to think that sort of ten month, 11 month intervention and maybe a year of training or so some some, you know, sporadic training camps, etc., is really going to change these sort of deep rooted attitudes, but there are something that's very, very, very interesting. The sort of exception to that rule is super informative. The one place where we do see changes in attitudes have to do with the female officers and in a very particular way, which is that they are now more likely to believe rather than dismiss cases of women. So specifically, the question related to this idea that women come in and file false cases. False cases is this terminology like people use the English term, even when speaking in Hindi, they've always false cases, false cases right and this is again true all over the world. People don't believe women, but again, in India and in the police force, we find like 40% of officers are willing to say that, oh, generally women come in to file false cases. Now, in our baseline, women police officers are even more likely than male police officers to say that the women victims who come in are coming in to file false cases, but that drops after the intervention. So that's the one area where we see this fairly decent sized changed is that women police officers in treatment groups are now a much less likely to say that women come in to file false cases than their counterparts in the control group.

 

Jen [00:50:31] That is so interesting. Okay. So what are the policy implications of all of this? What should policymakers and practitioners take away from your results?

 

Sandip [00:50:41] So I guess the most important part is immigration and it's highly relevant because now many states have this rule that 30% of all new recruits into the police force have to be women. And a thing that you could do is, okay, fine, we'll hire all these women, but then we'll basically have them push paper and police headquarters, right? So take on these desk positions, these desk roles, not be at the front lines and what I think we find and what our results really strongly suggest is that should not happen, that the new recruits really should be at the front lines. They make a pretty big difference in case registration. They make a pretty big difference in and responsive policing. Now, of course, that doesn't mean that, okay, all you do is just put these women in the police stations and have them fend for themselves.

 

Sandip [00:51:34] You know, this is a bundled intervention. And it's clear that the training, the support, the standard operating procedures, all of those were very important as well as this sort of it's this is something that we're exploring a little bit more with our qualitative data and in a separate paper so I'm not going to be able to say too much more about this, but basically this idea that there is space created for women officers to respond to women citizens, really, and creating that space is is also really, really, really important. So I think those are the sort of big takeaways in the in the Indian context. I think one other takeaway is, is just to really stop equating crime rates with crime registrations because the gap is just so, so large.

 

Sandip [00:52:26] Maybe there's some correlation over time and, you know, generally across space, but, you know, just because more crimes are registered doesn't necessarily mean that more crimes are actually happening. So one one sort of spectacular evidence of this is it's what we see, not us. Other people collected data on these DIRs, these domestic incidents cases registered all across India during the time that our intervention was in place and the state of Madhya Pradesh is responsible for 41% of all the DIRs filed all over India, and that's only has about 8% of the population. So clearly it's not the case that there's a huge surge in domestic violence.

 

Jen [00:53:14] Right.

 

Sandip [00:53:15] And nor that MP has a really, really high rates of domestic violence when compared to other states. It's literally our intervention that you can see in the national data that's going on. And it's again, it's really important because registering these cases can help women, right. They can then access these resources and so, you know, it's not that you should make the headlines should absolutely not be, oh, look, suddenly domestic violence has increased in Madhya Pradesh. The headlines should be, oh, look, a lot more women victims are being assisted in Madhya Pradesh.

 

Jen [00:53:53] Yeah, that's amazing. Do you all have ambitions of getting is it possible to get data on kind of what these case outcomes were or, you know, what happened to the women after this intervention?

 

Sandip [00:54:05] So we just it took us a while to get the whole thing across IRB of course, and we were very, very careful to say and again, even with the police as well, they, they really didn't want us to have any sort of details of the cases themselves. So we did not follow up. We do know a couple of things.

 

Sandip [00:54:28] So we know that there's no sort of big change in arrests and so that's perhaps not that surprising, given that that there's limited resources. It's basically sort of in line with the number of if you look at the sort of standard errors it's in line with the number of increases in cases registered. So it's not that there are finding these cases, but arresting a lot fewer people. It's just that they're we can't really see this big bump in arrests. We thought we would look at whether the cases were reaching the court system and where there's a difference in that, that was not as helpful because as I mentioned earlier, what we found is that nearly all cases that are registered as a criminal case make it to the court. So there's not going to be a big impact on that per say from the intervention itself, because basically if you file it, you're going to get into court. So that rate didn't did change because it's already so, so, so high. That's about as far as we can say about post registration outcomes.

 

Jen [00:55:39] Got it. Okay. Are there any other papers related to this topic that have come out since you all first started working on the study?

 

Sandip [00:55:47] So I think there's a couple of papers, but I think I mentioned them in the sense that are these papers on the all women's police stations. So there's a paper that suggests that there is a lot of displacement and there's another paper that also finds some displacement, but suggests that maybe there's an increase in registration in as a result of these stations. And so there's been some work because we know there's different there's some variation to exploit in terms of when these police stations are established, etc..

 

Sandip [00:56:18] So it's they're not RCTs, but they have, I think, reasonable sources of variation. So those, I think, are the main papers that are coming out. There's some interesting new potential work that people are doing that I think could be could be really fascinating. And tell us a lot about, you know, what might happen in this in this area.

 

Jen [00:56:41] And what's the research frontier? What are the next big questions in the space that you and some of these other people you mentioned will be thinking about going forward?

 

Sandip [00:56:50] Yeah, so so I think one, this is a interesting new study is by Nishith Prakash and Maria Micaela Sviatschi and coauthors and what they're looking at I think is super fascinating, which is exactly one of the things that we didn't see in this paper, which was the male officers attitudes. And they're doing an intervention where they're directly targeting the male officers attitudes, using a variety of different types of interventions, including like sort of skits and sort of playacting and this kind of stuff, which I think is fascinating because, you know, obviously the male officers are such a huge part of the police force and without their attitudes changing, it's going to be really difficult to have long term change.

 

Sandip [00:57:37] So I think trying to see if those there's any traction there at all, I think is really interesting and fascinating, and I think that's super interesting. The second thing that I think is interesting in the space, which, you know, we're hoping to to look at, maybe we don't really have a great research question quite yet, but it's the broad area of the female officers themselves and their recruitment in the context of India and its falling labor force participation. A female labor force participation rates, which as a lot of papers have been written about this and lots of people have been looking at why and why not.

 

Sandip [00:58:18] And whether it's an issue or whether it's not an issue, but it's what's true is it's very clear that female labor force participation rates have been falling across India. And now trying to understand how this interacts with women, police officers, their recruitment, this extra, you know, 30% of new recruits have to be women type of policies I think is really fascinating and understanding what exactly was going on there, whether there's going to be huge problems in recruiting, what strategies could be used to recruit, could they be successful elsewhere. I think that that's that's another really fascinating topic. And then I think there's a paper actually in science on community policing in many different contexts around the world and how it really doesn't seem to work and I'm obviously really, really shortening their their punch line.

 

Jen [00:59:21] But I think that's the punch line.

 

Sandip [00:59:23] Yeah. But looking at some of the results more carefully, it seemed like often it was kind of like the community outreach in our intervention, right? So they tried, but they just did not have the resources to do it. Right. And so I'm sort of wondering whether, look, is it just that, you know, it doesn't work or you just cannot get it to work because you don't quite have the money and human resources or not. So I think that's that would be really interesting to try and get it.

 

Jen [00:59:51] Yeah, it's so interesting how, how many of these questions despite in some ways the very different contexts right across India and other parts of the world, the US, so many of the questions that we're all grappling with are the same, hiring more women officers, how to increase reporting of gender based violence all of that stuff.

 

Sandip [01:00:11] Absolutely.

 

Jen [01:00:12] Yeah. My guest today has been Sandip Sukhtankar from the University of Virginia.

 

Jen [01:00:16] Sandip, thank you so much for talking with me. 

 

Sandip [01:00:19] Thank you, Jen.

 

Jen [01:00:25] You can find links to all the research we discussed today on our website probablecausation.com. You can also subscribe to the show there or wherever you get your podcasts to make sure you don't miss a single episode. Big thanks to Emergent Ventures for supporting the show and thanks also to our Patreon subscribers and other contributors. Probable causation is produced by Doleac Initiatives, a 501(c)3 nonprofit, so all contributions are tax deductible. If you enjoy the podcast, please consider supporting us via Patreon or with a one time donation on our website. Please also consider leaving us a rating and review on Apple Podcasts. This helps others find the show, which we very much appreciate. Our sound engineer is Jon Keur with production assistance from Nefertari Elshiekh. Our music is by Werner and our logo was designed by Carrie Throckmorton. Thanks for listening and I'll talk to you in two weeks.

 

Jennifer DoleacRCT, Policing, Domestic-Violence