In response to the opioid epidemic, several cities, from New York City to Seattle, are considering a controversial policy: allowing spaces where people can, under supervision, inject heroin and use other drugs. The idea is that if people are going to use drugs anyway, there might as well be places where those drug users can be supervised in case something goes wrong.
“After a rigorous review of similar efforts across the world, and after careful consideration of public health and safety expert views, we believe overdose prevention centers will save lives and get more New Yorkers into the treatment they need to beat this deadly addiction,” New York City Mayor Bill de Blasio said in a statement earlier this year.
But a new study has found that these places, known as supervised drug consumption sites, safe injection sites, and many other names, may not be as effective at preventing overdose deaths and other drug-related problems as once thought. According to a new review of the research published in the International Journal of Drug Policy, safe consumption sites appear to have only a small positive relation to drug-related crimes but no significant effect on several other outcomes, including overdose mortality and syringe sharing.
“The contrast between the claims that are being made and what the evidence actually says” stuck out to Keith Humphreys, a drug policy expert at Stanford University who was not involved in the review. The new research review’s results, he said, “are fairly disappointing.”
In the past, experts, advocates, and journalists (including myself) have said that supervised consumption sites have a lot of evidence supporting them — pointing to past reviews of the research that concluded the sites are effective in several areas. But this latest review of the research is more rigorous than those done before it, and it detected little to no effect from supervised consumption sites in the best studies the researchers could find.
That is not to definitively say that supervised consumption sites don’t work; it’s more that we simply don’t know yet. One of the problems the review found is that the research is seriously lacking in this area. Out of the dozens of studies on the topic they found, the researchers concluded that only eight were rigorous and transparent enough to include in the review. With such a small pool of studies included, it’s possible — maybe even likely — that these few studies were in some ways biased, so future research could produce entirely different findings.
As a result, several experts who support supervised consumption sites said that the new review of the research is fundamentally flawed. “They excluded, almost systematically, a lot of the studies that had demonstrated benefits on the metrics that they have selected,” Leo Beletsky, a professor of law and health sciences at Northeastern University, told me.
The review does not show that supervised consumption sites lead to, as detractors claim, more drug use and crime. In fact, the findings speak against that, if anything, as the sites appear to be linked to slightly lower drug-related crime.
But the review indicates that the sites are not as evidence-based as supporters often claim, and more research is needed to reach hard conclusions about supervised consumption sites one way or the other.
What the new review of the research found
The new review of the research, from Tom May, Trevor Bennett, and Katy Holloway at the University of South Wales in the UK, was a standard meta-analysis. The researchers first searched for previous studies on supervised drug consumption sites, pulling out 40, most of which looked at sites in Vancouver, Canada, and Sydney, Australia.
They then tried to weed out the weaker studies — meaning, in scientific terms, those that didn’t provide fully replicable data and those that didn’t have a comparison group. That left them with eight studies total.
The researchers then looked through the eight studies to measure the possible effects of the sites on several outcomes, including ambulance attendances relating to opioid-related events, overdose mortality, drug-related crime, borrowing or sharing syringes and injecting equipment, and problematic heroin use or injection.
Ultimately, the researchers concluded that supervised consumption sites had no significant effect on most outcomes. The sites only had a small favorable relation with drug-related crimes, and a small unfavorable association to problematic heroin use or injection.
The unfavorable result, however, does not necessarily mean that supervised consumption sites lead to more problematic heroin use or injection. By their very nature, these sites are built for people who are using heroin in a problematic way — that’s why these people may need such an intervention and supervision in the first place. In other words, the finding may only speak to the existing population that supervised consumption sites attract.
The researchers noted as much: Supervised consumption sites “have been found to attract the most problematic heroin users.” They went on: “This might influence outcomes as a result of comparing pre-existing risk behaviours and related health harms with less serious behaviours among the non-[site] group.”
Rebecca Goldin, director of STATS.org, said this reflects a common problem in this kind of research: “No meta-analysis can overcome systematic bias or problems that occur with the body of literature it incorporates. To mind, a risk in this particular literature is that a higher risk population is taking part in [supervised consumption sites], resulting in a diminished effect in the assessment.”
Still, as the first meta-analysis of supervised consumption sites to look at a more thorough list of outcomes, it presents disappointing findings — suggesting that these sites may have little to no impact overall.
There are limitations in the review. The researchers might have missed some potentially strong studies, particularly those that weren’t in English and didn’t provide fully replicable data. The researchers also acknowledged that “there were relatively few studies suitable for meta-analysis,” and once the body of research grows, it could lead to different conclusions.
The review contradicts past research
The new review’s conclusions also sharply contradict previous reviews of the research.
For example: Drawing on more than a decade of studies, the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) in 2018 concluded that safe injection sites led to “safer use for clients” and “wider health and public order benefits.” Among those benefits: reductions in risky behavior that can lead to HIV or hepatitis C transmission, drops in drug-related deaths and emergency service call-outs related to overdoses, and greater uptake in drug addiction treatment, including highly effective medications for opioid addiction.
But EMCDDA’s review wasn’t a traditional meta-analysis, so it wasn’t as rigorous or selective in what studies — and what quality of studies — were included in the review. That allowed EMCDDA to include more studies, but many of those studies may have been of poor quality.
For Humphreys, the new review is more reliable than EMCDDA’s look at the research. As he put it, “If you impose even a modest methodological bar, and then those [studies’] effects go away, to me that’s worrisome.”
Beletsky pushed back — pointing out that the review of the evidence only looked at eight studies, none of which were randomized controlled trials. “That signals in and of itself that they’re not literally looking at the full picture,” Beletsky said. That’s why he favors the systematic reviews that have been done in the past and included far more studies, such as EMCDDA’s.
The eight studies, though, were meant to be the best that the researchers could find. The studies that were excluded were those for which the researchers couldn’t get full data sets and which didn’t have comparison groups — fairly big methodological gaps.
This is typical in meta-analyses: The ideal is randomized controlled trials. But if none exist, researchers start looking at other kinds of studies, while maintaining some level of rigor, to tease out the evidence that is available.
David Wilson, a criminologist at George Mason University, said the review “is a solid meta-analysis and adheres to basic practice standards for this type of study.” But he took issue with one of the models the researchers used, and felt they could have paid more attention to publication bias.
Others were more critical. Michael Lavine, a statistician at the University of Massachusetts Amherst, acknowledged that the methodology in the meta-analysis is “very common in medical and social science research.” But he called that methodology “bad,” and warned that it can dramatically understate or overstate the benefits of an intervention depending on how an outcome measure — known as “odds ratio” — is calculated.
Regina Nuzzo, a statistician at Gallaudet University, echoed Lavine’s concerns. She also emphasized that not only did the review analyze just eight studies, but that those eight studies only looked at four supervised consumption sites total — which she said is “a bit like double-dipping.”
Another way to look at this, though, is not that this review is flawed, but that the underlying research is flawed — if these truly are the eight most rigorous studies in the field — and, as a result, the research can’t give us much information about the effectiveness of supervised consumption sites.
“If you are an advocate, you could say correctly that if we assume these are effective, we do not have sufficient information to confidently overturn that presumption,” Humphreys said. “But it’s equally true if you took another view — just look at it as a cold, scientific question — you could say we also don’t have the evidence to overturn the presumption that these don’t make any difference.”
The potential problem: supervised consumption sites may not scale well
One thing supervised consumption sites do is reverse overdoses — thousands over the years, by some advocates’ estimates. That’s why the sites’ staff have naloxone, the opioid overdose antidote, and oxygen tanks on-site. So how could it possibly be that the sites don’t reduce overdose mortality, perhaps the most important metric in an increasingly deadly opioid crisis, when they’re reversing all these overdoses?
Part of it, Humphreys suggested, is most overdoses are not fatal. It’s also possible that the sites may enable more drug use, leading to more overdose deaths even as others are stopped — although the there’s no good evidence to support this possibility.
The bigger problem, though, seems to be that supervised consumption sites may not have enough reach to have a significant impact.
The review of the research speaks to this point, noting that “facilities are limited in the number of users they can accommodate.” Consider that Vancouver, for example, was previously estimated to have about 5,000 injection drug users. A site that can hold at most a dozen or so users at a time and is closed for some parts of the day is simply not going to have much of a reach in such a large population — servicing, the review suggested, “a small fraction of users each day.”
That’s made worse by further restrictions on who supervised consumption sites will accept. They often won’t, for example, allow people to share drugs or assist each other in injecting. So users who share drugs or need assistance from others will simply use elsewhere — in the streets, at home, in a motel, wherever. That further limits these sites’ reach.
Beletsky agreed: “It’s not surprising to me that the population-level impact is limited because the capacity of these facilities is limited in terms of hours, throughput of people, and so forth.”
Humphreys guessed that the likely truth is supervised consumption sites work “really little.” It’s not that they don’t have any effect, but that the effect is likely so small that it’s not going to be picked up at a population level by the research.
To this end, some advocates are trying to expand the reach of supervised consumption sites. In Canada, for instance, activists have deployed more mobile pop-up sites that can reach communities where a fully staffed building may not always be needed or available.
Another point, made by Beletsky, is perhaps single supervised consumption sites aren’t supposed to have big effects on a population scale. Maybe it’s fine if the sites just help a limited group of people who need them.
But by scaling them up through other means — like pop-up sites — you may start seeing a broader community effect, Beletsky argued. “Thus far, these interventions have been limited,” he said. “They’ve been mired in legal and political battles. They’ve been artificially suppressed. They could be doing a lot more.”
There are plenty of evidence-backed solutions to the opioid epidemic
Despite the disappointing results for supervised consumption sites, Humphreys said that he’s not discouraged about the country’s ability to fight the opioid epidemic. “We have plenty of other things that we know, with much more confidence, that work,” he explained.
At the top of those other things is treatment — specifically, medications like methadone and buprenorphine. There is decades of evidence behind these medications, showing that they reduce the mortality rate among opioid addiction patients by half or more and keep people in treatment better than other approaches. When France relaxed restrictions on doctors prescribing buprenorphine in response to its own opioid crisis in 1995, the number of people in treatment rose and overdose deaths fell by 79 percent over the following four years.
But these medications, and addiction treatment in general, remain largely inaccessible in the US. A 2016 surgeon general report concluded that only 10 percent of people with a substance use disorder get specialty treatment, in large part due to a lack of affordable and accessible treatment options. And even when treatment is available, other federal data suggests that fewer than half of treatment facilities offer opioid addiction medications.
Sticking exclusively to the realm of harm reduction, the US could do a lot more there too. Consider needle exchanges, where users can pick up sterile syringes and trade in used needles. The decades of research show needle exchanges combat the spread of bloodborne diseases like hepatitis C and HIV, cut down on the number of needles thrown out in public spaces, and link more people to treatment — all without enabling more drug use.
Yet needle exchanges remain scarce in the US, as Josh Katz reported for the New York Times: “According to the North American Syringe Exchange Network, 333 such programs operate across the country, up from 204 in 2013. In Australia, a country with less than a tenth as many people, there are more than 3,000.”
Even some more innovative, controversial solutions appear to have more evidence than supervised consumption facilities. Humphreys said that the evidence behind prescription heroin sites, as one example, is “much stronger.”
The idea behind prescription heroin sites: A certain segment of opioid users are going to use heroin no matter what. For whatever reason, traditional therapies just aren’t going to work for them — just like some treatments for, say, heart disease or cancer don’t work for some patients. So if that happens, it’s better to give them a safe source of the drug they’re seeking and a safe place to inject it, rather than letting them pick it up on the street — laced with who knows what — and possibly overdose without medical supervision.
Researchers credit the European prescription heroin programs with better health outcomes, reductions in drug-related crimes, and improvements in social functioning, such as stabilized housing and employment. Canadian studies also deemed prescription heroin effective for treating heavy heroin users. A review of the research — which included randomized controlled trials from Switzerland, the Netherlands, Spain, Germany, Canada, and the UK — reached similar conclusions, noting sharp drops in street heroin use among people in the treatment.
There is no prescription heroin program in the US.
Humphreys argued that, in a world with limited financial resources and finite political and cultural capital, governments should try to first support the more evidence-based approaches than those with less.
“Should you have a culture war over something that barely engages the population and at most has a teeny effect when we still have people who can’t get methadone and buprenorphine, which have a whopping effect and can engage a huge number of people?” Humphreys said. “For me, that would be an obvious decision.”
Beletsky rejected the idea that we have to choose between different approaches, arguing that safe consumption sites can complement other interventions in the opioid epidemic.
“We should be doing all those things that you mentioned,” Beletsky said. “But there are challenges in reaching some of the most at-need populations who can benefit from those interventions. And I think that safe consumption facilities provide a platform for reaching those folks.” He added, “Safe consumption facilities really operate as a low-threshold doorway for people who typically will not seek care in other settings.”
For example, someone who uses heroin may have had bad experiences with the criminal justice system or health care system in the past. That may make him skeptical of going to these institutions — or any other official institutions — for help. A supervised consumption site, though, can be different, since it’s an environment in which people are less judgmental about drug use. If the people running supervised consumption sites take advantage of this, they could use their better stature with people who use heroin to guide them to treatment and recovery.
But there’s no strong evidence to support the sites as an effective intervention for getting people into treatment and recovery — given that the new review of the research found no good studies that adequately evaluated for this.
That goes back to the core problem: There is a lot out there about supervised consumption sites that certainly seems promising, even intuitive. But until the empirical research backs it up, pouring time and money into this kind of intervention may not be as evidence-based as people think.
Sourse: vox.com
