11: How do randomized experiments work?

This chapter covers the main study design limitations that are addressed by using randomized controlled trials, or RCTs—also known as randomized controlled experiments. These include the capacity to overcome pre-treatment differences and confounding. The chapter then goes on to explain how randomization works and two methods of randomization (simple and block). The chapter concludes with a discussion of some issues to consider with randomization, given they are not well understood in policing, and ethical concerns around their use. A handy box lists several tips for running a randomized trial.

Glossary terms in this chapter

Randomized controlled trials (RCTs): A type of study design that randomly assigns individuals or areas into either an experimental group or a control group.
Randomized controlled experiments: See Randomized controlled trials.

Pre-treatment differences: Differences that exist between a treatment group and control group that exist outside of the study causing them to not be reliably comparable.
Selection bias: See Selection effects: This is a problem if any effects observed reflect differences that already existed between the people or places receiving the treatment and the comparison areas.

Confounding: When an effect was caused by some other event occurring at the same time as the treatment intervention.

Confounding variable: A variable that causes confounding: See Confounding.

Systematic bias: This bias can be inadvertently introduced by the research design and encourage one outcome over another.

Substitution bias: Occurs when individuals in a control group independently seek to replicate the treatment activity or try and find alternatives.

Additional information and links

Are parachutes effective?

This article in Perspectives in Clinical Research, titled Evolution of Clinical Research: A History Before and Beyond James Lind, provides a useful summary of the history of clinical trials, including description of the first ever use of randomization in the 1946 UK study of the use of Streptomycin to treat tuberculosis. Prior to this, the preferred method had been allocation, where study patients were assigned to a treatment or control based on strict alternate allocation as per a schedule. There is also a marvelous first-person account of the first medical trial using randomization, The MRC randomized trial of streptomycin and its legacy: a view from the clinical front line written by John Crofton.

A refresher on randomized trials

There is a readable refresher on randomized, controlled trials by Amy Gallo over at the Harvard Business Review. She covers the terminology, the purpose and value of RCTs and even provides a nice quick primer on doing one. Handy review if you want to give someone a link. Though, you should really just buy them a copy of this book.

Confounding

Confounding often confuses or confounds students. There is a basic primer on confounding here, if you want to read something in addition to the book. There is also a useful and relatively short (a bit more than 4 minutes) below. They come up with a nice analogy with smoking and matches, though I will advise that they use all of the technical language. But take a listen - you will get the hang of it.

Related Reducing Crime podcast episode

This episode starts with Alex Murray discussing an offender outreach program that he initiated in London, based around randomizing who would get a visit. While much of my own research revolves around randomizing places to improve internal validity, it is good to see that it can be done with people as well. If you want another insightful example, see episode $47 with Jackie Sebire's research on domestic violence court advocates.

#31 (Alex Murray)

Now a Deputy Commissioner in West Mercia police, at the time of the recording Alex Murray was a Commander with the London Metropolitan Police working as their lead for trafficking, online child abuse, the flying squad, cyber crime, and major crime. He is a firearms, counter-terrorism and public order commander, and has previously work in local policing, CID and counter-terrorism. Prior to joining the Met in 2020, he was temporary assistant chief constable for crime with West Midlands police. In 2017 he was awarded an OBE in part for his contributions to evidence-based policing and founding the Society of Evidence-Based Policing. We discuss offender management opportunities during COVID-19, what to look for in an academic that can support policing advancement, and the important evidence-based policing lessons for police leadership.