In July 2014, the Lancet, one of the world’s best known medical journals, devoted a special issue to sex work and HIV. The publication was timed to coincide with the 20th International AIDS Conference in Melbourne, Australia. One of the articles, led by researcher Kate Shannon of the University of British Columbia, caught Senior Abt Associate Michael Shively’s eye.
The article, which Shively refers to as “the Shannon study,” used forecast modeling to test HIV prevention scenarios and estimate their impact on transmission rates among female sex workers and their clients in three very different cities: Vancouver, Canada; Mombasa, Kenya; and Bellary, India. The authors plugged independent variables like “safer work environment” and “elimination of inconsistent condom use due to any client violence” into their model and estimated their effect on averting HIV infections over ten years. They found that the decriminalization of sex work and its resulting benefits would have the potential to avert 33-46% of infections over a decade.
This finding made international headlines. “Decriminalization of sex work could reduce HIV infections,” declared the Washington Post. “The evidence is in: Decriminalizing sex work is critical to public health,” affirmed the Center for HIV Law & Policy. The authors circulated their findings to influence Amnesty International’s policy on the decriminalization of consensual sex work, which the organization released in May 2016.
“This is a very consequential study,” says Shively. “This is presented as the state-of-the-art, capstone event of decades of research and it was received that way and it was presented that way by the authors. They’re basically saying, the evidence is in. We’re done here. We figured this out.”
A self-proclaimed “certified nerd” and long-time criminal justice policy researcher working with Abt and the U.S. Department of Justice, Shively says he was excited that a policy measure like decriminalization could have such a positive impact on the health of sex workers. So he decided to “look under the hood” to see how Shannon and colleagues got to their influential conclusion. What he found, however, was significantly less conclusive than the study seemed to suggest and, perhaps more importantly, than reporters and activists made it out to be.
Shively scrutinized the (more than 300) citations used in the study and its adjoining appendices, and recruited statisticians at Abt to validate the modeling. Based on their critical analyses, Shively says there are several ways in which the study lacks credibility.
A major limitation he cites is the “assumption used to simulate [the] impact of decriminalization on HIV.” For one, the model assumes that decriminalization of sex work will immediately result in several overly optimistic outcomes, for instance, 0% violence toward sex workers, 0% police harassment, 100% access to prevention and treatment services, and maximum condom usage. It also assumes that decriminalization will result in all all of these changes overnight, regardless of context or details of the legislation.
Shively presented his findings and concerns at a Commercial Sex Symposium in November 2017, arguing, “there is no evidence that fully decriminalizing prostitution produces any of the outcomes assumed by Shannon.”
Of course, this type of modeling can be useful, but only if a variety of reasonable assumptions are used to determine a realistic range of outcomes following decriminalization. “What Shannon really found,” according to Shively, “was that HIV transmission would be reduced if violence, harassment, inconsistent condom use, and barriers to healthcare were eliminated.”
The authors do not qualify their findings, and, instead, present their work as unequivocal. Shively says he is not necessarily arguing that the study is wrong, but that the data is not sound enough to base international policy decisions on. When Shively attempted to confront the co-authors of the study, he was not well-received.
“When you say that you’ve got hard science that proves something, it needs to be challenged [and] scrutinized, and when it doesn’t hold up, they should be welcoming scrutiny,” Shively says. “I don’t expect people to just accept what I’ve said, I want—I should get challenged on it.”
Shively doesn’t know what will happen next. He has plans to publish his critique of the Shannon study and says, “I don’t think this is going to end well for me.” Point in fact, a reporter who attended the November symposium castigated him during the question and answer session and was more or less given the last word. Shively hasn’t heard from her since, but said he would have been glad to address her concerns.
For anyone writing about research, as we do every day at Public Health Post, Shively’s quest to “advance knowledge, build science, [and] get closer to the truth” is an important reminder. It is easy to gloss over the nuanced methods and results of academic papers and focus on the data that support our positions. Shively set out to understand the Shannon study and replicate it—a standard scientific practice. By asking unpopular questions and challenging overly optimistic conclusions, researchers can be sure they are producing the most thorough, transparent, and accurate evidence possible.
Feature image: Michael Shively, speaking at the Boston University School of Public Health Dean’s Symposia “Understanding Commercial Sex Policy: A Global and US Perspective“
