15: What are the challenges with EBP?

This chapter explores various common concerns with the growth of evidence-based policing. These include that police experience and craft are ignored, it is not suitable for frontline officers, the discipline is too purist, policing research in general supports the status quo of policing, that it is still vulnerable to problems of publication bias and lack of replication that plague other sciences, and that police rank can stifle adoption of implementation. The chapter concludes with a discussion of the ethics of randomized controlled trials.

Glossary terms in this chapter

Optimism bias: The inclination to overestimate the likelihood of experiencing positive outcomes and underestimate the chances of experiencing negative events.

Pracademics: Police officers with not only professional expertise but also academic training capable of examining research problems.

Principle of proportionality: Involves balancing the seriousness of the intrusion into the privacy of the subject of an operation against the need for the activity in investigative and operational terms.

Replication: The process of repeating a study or experiment to see if results similar to the original study can be reproduced.

Publication bias: The incentive to publish novel research that will garner attention and attract interest and citations, and are biased against reproductions of existing research as they are less interesting.

Research ethics: Research ethics are the values, norms and institutional arrangements that are the fundamental values of research practice, and they regulate and guide scientific work based on the general ethics of morality in science and the community.

Additional information and links

The craft of policing

Jim Willis's discussion titled Improving Police: What’s Craft Got to Do with It? is a thoughtful take on the craft of policing. It is from the Ideas in American Policing series, which highlights commentary and insight from leading criminologists on issues of interest to scholars, practitioners, and policy makers. The papers published in the series are from the Police Foundation lecture series of the same name. I have in the past given on of the lectures on harm-focused policing. Jim Willis does a nice job of laying out the challenges of finding a balance of craft and science within policing.

The man who saved the world

This is the title of a 2013 feature-length Danish documentary film by filmmaker Peter Anthony about Stanislav Petrov, featuring Kevin Costner.

Evidence-based policing isn’t for street cops

In this section I highlight the original research conducted by Larry Sherman and colleagues on the The Minneapolis Domestic Violence Experiment. That original, and groundbreaking study, was published by what is today called the National Policing Institute. Their research brief on the study from 1984 nicely summarizes the study.

Pracademics

In the section titled Evidence-based scholars do not have a dog in the fight, I discuss pracademics. A good discussion of pracademics and internal police research opportunities from Eric Piza raises the interesting question of whether we are in a world of police-led science or science-led policing. He goes on, with co-editor Brandon Welsh, to discuss the expansion of evidence-based policing (in an open access article titled Evidence-Based Policing Is Here to Stay: Innovative Research, Meaningful Practice, and Global Reach) which includes chapters from many people in and around policing, including pracademics.

Evidence-based policing is too purist

While not available to read online, one book that is available to purchase is Critical Reflections on Evidence-Based Policing. In Peter Neyroud's review of the book that it is, in places, a polemic attack on the Sherman perspective of evidence-based policing. Neyroud concludes, "The editors might have done better to have critically reflected themselves that EBP is a much larger and more diverse movement than some of the chapter’s managed to describe. That wider focus and a little more grace in accepting some of gains of science and evidence in transforming policing would have provided a more significant contribution". I include the link in case you are looking for something more critical of evidence-based policing than I provide in the EBP: The Basics book.

Police abolition

In the section titled Policing research supports the status quo, I do not discuss police abolition. While some claim that discussions of abolition are really discussions around defunding and reform, there are others who are adamant that abolition means abolition. The book chapter has a number of references you can pursue, and this more recent work by Morgan Williams and his colleagues, finds "that, in the average city, larger police forces result in Black lives saved at about twice the rate of white lives saved (relative to their percentage of the population)."

Publication bias

Publication bias is real. This graph below is of Z-values extracted from confidence intervals in Medline journal articles between 1976 and 2019. If you are unfamiliar with z-values, you should know that the darker red values to the left of the vertical dashed line indicate studies where there is a high level of confidence the intervention made things worse. Conversely, darker red lines to the right of the right-hand dashed line are those where the intervention likely made things better. We should expect a normal distribution if publication bias were not an issue, however, it is clear there is a significant problem with getting non-significant studies published. Adrian Barnett's blog post is the source of the graphic and has more information.

Related Reducing Crime podcast episode

This episode is useful because Jason discusses some of the common concerns with randomized controlled trials and how there are considered to be other sources of knowledge that can aid policing and public safety.

#44 (Jason Roach)

Dr. Jason Roach is a chartered psychologist, Professor of Psychology and Policing, and Director of the University of Huddersfield's Secure Societies Research Institute. In the podcast, we talk about some of his projects that have explored offender self-selection and trigger crimes, criminal decision-making, nudging and influencing crime prevention, and learning from offenders.