Anti-depressants do work: reply to Stegenga

In a piece at Aeon linked to at Daily Nous, philosopher of science Jacob Stegenga (University of Cambridge) contends that "we simply have no good evidence that antidepressants help sufferers to improve." I don't think the evidence available supports Stegenga's argument, and I made a comment to that effect in the comments section over at Aeon (which Stegenga replied to). Because I think this is a very important issue--one that could potentially affect people's choices and public attitudes regarding anti-depressants--I want to address Stegenga's argument here at the Cocoon.

I want to begin by noting two things. First, this issue is very personal to me. Mental illness not only runs in my family, affecting multiple people I am close to - it has also affected friends of mine. Second, I have experience working in the mental health field. As an undergraduate, I interned for a year in an out-patient day program utilized by dozens of individuals with serious mental illnesses: schizophrenia, bipolar disorder, depression, borderline personality disorder, and so on. Then, after graduation, I was Assistant Director of a group home that housed around a dozen residents with serious mental disorders. Among many other things, I was responsible on a day to day basis for dispensing medications, including importantly changes to patients' medications.  

Bearing this in mind, let us turn to Stegenga's basic argument that "we simply have no good evidence that antidepressants help sufferers to improve." He contends, first, that "the best evidence about the effectiveness of antidepressants comes from randomised trials and meta-analyses of these trials." I will explain below why--on methodological grounds--I believe this to be false. Stegenga's argument then is that in randomized trials and meta-analyses, observed mean effect sizes are tiny:

In meta-analyses that include as much of the evidence as possible, the severity of depression among subjects who receive antidepressants goes down by approximately two points compared with subjects who receive a placebo. Two points...We saw above how clinical guidelines have held that drugs must lower severity-depression scores by three points to be deemed effective. On this standard, antidepressants do not pass.

So that's the argument:

1. Randomized trials and meta-analyses are the best evidence for efficacy of anti-depressants.
2. Randomized trials and meta-analysis indicate tiny mean effect-sizes for anti-depressants.
3. If (1) and (2) are true, then we have no good evidence that anti-depressants help sufferers to improve.
4. Thus (from 1-3), we have no good evidence that anti-depressants help sufferers to improve.

Here's the problem: (1) in this argument is arguably false, as is premise (3). Let me begin with (1).

It is widely thought that randomized clinical trials are the "gold standard" for determining efficacy of medical interventions. I believe this widespread assumption is false - and for a reason I gave in my comment over at Aeon. Randomized trials and meta-analyses at best record group-level effect-sizes, abstracting away from individual-level effects. Allow me to explain.

As I mentioned above, I have a lot of first-hand experience with mental illness and its treatment. Here, in my experience--with again a wide variety of patients--is how treatment often goes. A person has severe depression, or bi-polar disorder, or schizophrenia. They try a variety of interventions: talk therapy, exercise, cognitive-behavioral therapy, and so on. These interventions may have some effect on their symptoms, or none at all. Then the person is placed on a particular drug: say, an anti-depressant like Prozac. Because that drug does not alleviate their symptoms, their practitioner tries other drugs in the same class (Lexapro, Zoloft, etc.), as well as other classes of drugs (Buspar, etc.). Maybe a half-dozen of these drugs will worsen the person's symptoms. Another half-dozen have little effect. But then, after experimenting with a bunch of different possibilities, the doctor and patient finally happen on one anti-depressant that has profound effects. Whereas nothing else worked--the person was unable to function in everyday life, overcome with constant self-hatred ("I want to die")--that single drug appears to induce profound improvements in the person's symptoms. I am confident that any practitioner who actually works in mental health will tell you that this is what they have seen over and over again.

Here then is the problem. If you were to run statistics on this person, you would find that "anti-depressants have negligible effect." Why? Because most of the anti-depressants they tried either didn't work for them or had negative effects on their symptoms, and only one had strong positive effects. Now imagine, next, that you randomly assigned participants to receive particular anti-depressants. Across that random sample, you will find (statistically speaking) that "most antidepressants don't help most people" (and a small effect-size across the group sampled) even if, at an individual level, particular anti-depressants have immensely positively effects on particular individuals (and indeed, because people's brain chemistry differs, the one drug that works for one person might be a very drug that does not work "for most other people"). 

This is the basic problem with taking randomized trials and meta-analyses to be the "gold standard" for evaluating the causal effectiveness of medical interventions. Group-level statistical differences can be a good mark of causation. However, they can also miss genuine causation at the level of individuals--because they only infer causation from group differences. This is why, in actual medical research and practice, there are other methods for inferring causation, including clinical observational studies.

Stegenga can of course argue that this method of investigating individual-level causal efficacy is mistaken - as indeed he does later in his Aeon piece. However, Stegenga confidently pronounces that he is right about this, claiming that "if you hear of someone benefiting from antidepressants, this was likely due to the natural course of the disease fluctuating or improving over time, confounded by the placebo effect, and exaggerated by confirmation bias." As someone who has seen anti-depressants literally save lives--indeed, as someone who has seen people go from suicidal despite many other interventions to suddenly becoming stable after finding  the right drug, and as someone who has seen people very close to me maintain their mental well-being and ability to function in society for years or decades until the moment they go off their meds (or their doctor changes their meds), suddenly lapsing back into depression until they take the same med again, and whose symptoms worsen when they try anything else--I am, to put it mildly, unpersuaded that it is chance fluctuations, confirmation bias, or placebo effect. The best explanation of ample clinical experience, it seems to me, is that anti-depressants can have genuine, and very powerful, individual-level effects. And I think it is unfortunate that Stegenga so confidently dismisses this kind of evidence in a piece for the broader public.

Comments

[Brad]

Marcus,
there is a (critical) review of Stegenga's new book in Metascience. It is On-line first. It was written by Don Gillies. You may want to read it.

[Marcus Arvan]

Hi Brad: Thanks for bringing it to my attention. I would love to read it. However, it is unfortunately behind a paywall and I cannot access it.

[Nathan]

Hi Marcus, I have access and am happy to send it to you.

[Marcus Arvan]

Hi Nathan: that would be great - thanks! My email is [email protected]

[Tom]

scihub, yo!

[Amanda]

Thanks Marcus! I completely agree with you. And seeing articles like Stegena's infuriate and sadden me. Thanks for writing the reply. Anyway, I think two things are true about current anti-depressants:

1. As a class of drugs, on the whole, the results are pretty bad. What I mean is that, (1) they don't work for a lot of people, and (2) the side effects are costly. - caveat if everyone who unsuccessfully tried one anti depressant kept trying new ones, this might be different. But most people give up after one or two tries.

that said,

2. They do work very well for *some*. They change, save, and make lives worth living for those who are lucky enough to respond to them, or perseverant enough to keep trying and find one that works.

Also, there are various anti-depressants that are not yet FDA approved that I have high hopes for. That is another story, but I do think the next 20 years will see major changes in so far as we will see better depression meds that help more people with less side effects.

[Megan]

Hi Marcus! I'm wondering if you could answer a question of mine about the aims of this piece? At most, it seems like the argument in this post shows that premise one of Stegenga's argument is false. However, it seems to replace it with a worse premise: "There is no test that can accurately determine the efficacy of antidepressants." If this is not the author's intent, and if the author thinks that testimony is enough evidence of the efficacy of antidepressants, then I'm concerned the result of this would be that homeopathy also enjoys strong testimonial evidence for its efficacy--potentially even stronger evidence than antidepressants.

[anon]

What you wrote here, is what is categorized as 'anecdotal evidence', I think?

Most scientists don't prefer that to group data, but philosophy is a different kind of science I guess...

[Marcus Arvan]

Anon & Megan: No, it isn't just anecdotal evidence, nor is it untestable (contrary to what Stegenga also noted in his response to me over at Aeon).

There are many different types of empirical studies and data. Randomized controlled studies are one type of study. Another kind is an observational field study. And there are other types of studies as well. While randomized controlled studies have certain advantages (randomized assignment and control groups), they also have disadvantages (measuring group-level effects but not individual-level effects). This is why many fields in science (physics, biology, psychology, sociology, etc.) utilize *multiple* kinds of studies, including observational studies, interventionist field studies, and so on.

And in fact Stegenga more or less recognizes this in a 2011 paper of his own, where he argues: "Meta-analysis fails to provide objective grounds for intersubjective assessments of hypotheses because numerous decisions must be made when performing a meta-analysis which allow wide latitude for subjective idiosyncrasies to influence its outcome. I end by suggesting that an older tradition of evidence in medicine—the plurality of reasoning strategies appealed to by the epidemiologist Sir Bradford Hill—is a superior strategy for assessing a large volume and diversity of evidence." (https://philpapers.org/rec/STEIMT ) That's right, Stegenga admits that there are a plurality of reasoning strategies and forms of epidemiological evidence.

To see how multiple kinds of studies can provide good empirical evidence, consider one classic test of causation from the philosophy of science and scientific practice: tests of co-variation. You manipulate an independent variable (in this case, the drug a patient is taking) multiple times, and measure whether there are replicable differences in a dependent variable (in this case, symptoms).

You can run a simple set of co-variation tests, for instance, to determine whether prayer causes good things to happen. Try praying. Some good things may happen. Stop praying. Good things may still happen. Try praying. Nothing may happen. Try praying harder. Something bad may happen. We see in this case--even in the case of individuals--that there is no replicable co-variation between the manipulated independent variable (praying) and good things happening. *This* test of co-variance is all you need to know that a given person praying doesn't cause good things to happen to them. You don't need a randomized control study to show this.

Now consider clinical tests of co-variation in psychopharmacology. Suppose you have a person with consistent, unresolved symptoms of depression. Chances are, with a patient with severe depression, a placebo will do nothing (trust me, I have ample experience working with individuals with severe depression. Giving them a sugar cube isn't going to lead them to stop thinking, "I really want to die"). Then try talk therapy and exercise (no, these things won't suddenly cause a seriously depressed person to improve). Then try an anti-depressant (Prozac). If it has no effect, you try another (Lexapro). If that manipulation of the independent variable has large observed effects (large improvements in mood), then you can then manipulate the independent variable *further* to test causal co-variance. You can increase dosage and decrease dosage. Since the drug is hypothesized to have dose-dependent effects, you can observe as a clinician whether those effects are manifest. Then, if your patient stops taking their meds, you can see whether their symptoms reappear (and trust me, when working with patients with depression this is *exactly* what happens when they stop taking a medication that works). Then if you put them back on the same med, you can see whether they improve again (they will).

These types of interventions are good tests of causal co-variation. Having worked in mental health, I can tell you with absolute certainty that anti-depressants that work for a patient aren't like prayer. There are *clear* manipulations of independent variables (giving a particular patient a particular drug) with systematically co-varying effects in particular patients. That is not a randomized control trial - but it is *substantial* evidence of a causal relation in an individual patient.

Anyway, Stegenga could of course provide a counterargument that this is bad, merely anecdotal evidence. I think that is wrong, but there are legitimate debates to be had about scientific methodology, and I am happy to have them. What I am concerned about is that Stegenga's Aeon article makes these complex and debated methodological issues seem simple and settled, confidently asserting that "we simply have no good evidence that antidepressants help sufferers to improve" when, given the methodological issues here, this is very much open to debate.

[Liz Wood]

I have done published research on psych drugs, never done an RCT or meta analysis, never anticipate that I will. They are specific tools for specific applications but don’t even represent half of how we know what we know about antidepressants. I can’t xomment as to how this is as philosophy, but he is ignoring huge swaths of science.

[Marcus Arvan]

Liz Wood: exactly right.

And causal co-variation in clinical practice (which I discussed in my previous comment) is only a small part of it. There is also neuroscience, where the causal roles of certain types of receptors and neural pathways and their relation to mood and cognition are pretty well-understood--pathways that particular anti-depressants are known to have demonstrable effects in.

Combine the neurophysiological evidence with clinical intervention and co-variance, and many other lines of evidence, and there is plenty of evidence for causal effects of anti-depressants.

[Postdoc]

The philosophy job market caused me to become very depressed and suicidal.

I started walking across roads without looking hoping a car would hit me.

Anti-depressants saved my life. If that was just a placebo effect, I'm impressed!

I guess I'm open minded enough to believe that perhaps the pills effect was 80% placebo and only 20% real.

But I know if I hadn't taken them there is a good chance I would be dead today or at the very least I would have seriously injured myself.

[anon]

This is anecdotal evidence:
"Chances are, with a patient with severe depression, a placebo will do nothing (trust me, I have ample experience working with individuals with severe depression. Giving them a sugar cube isn't going to lead them to stop thinking, "I really want to die")."

But there are studies that prove there is a strong placebo effect with anti-depressant (not for everyone, but the point is, anti-depressants don't help everyone either, but you are saying that because they help some people therefore they work). I think that summarizes what you are saying?

There are lots of things to take issue with in terms of what you are saying:

"Then, if your patient stops taking their meds, you can see whether their symptoms reappear (and trust me, when working with patients with depression this is *exactly* what happens when they stop taking a medication that works)"
>> describes what is scientifically noted but marginalized knowledge about _withdrawal_ effect of psychiatric drugs mimicking the problems the drugs supposedly cures. There are practices of how to safely withdraw from psych drugs that aren't cold turkey.

"That is not a randomized control trial - but it is *substantial* evidence of a causal relation in an individual patient"
>> could be applied to vaccines and SIDS, or vaccines and autism: Most people seem to prefer the overall studies, but there are individual cases that are *substantial evidence* of a causal relation

When you write about "an older tradition of evidence in medicine—the plurality of reasoning strategies appealed to by the epidemiologist Sir Bradford Hill—is a superior strategy for assessing a large volume and diversity of evidence"
>> That seems good, but I guess it points to being clear about what points you are trying to prove. Are you saying anti-depressants 'work' to say that they are better than placebo? To say that overall they are worthwhile? Is he saying there's no evidence that they've ever worked for anyone, or that there's no evidence that they are more effective than random/placebo?

My response to a meta-analysis type of study published last year that said anti-depressants were effective, was to raise these issues:
*Need full acknowledgement of placebo influence re medical depression treatment
*The lack of science behind 'chemical imbalance' myth
*The unacknowledged anti-depressant med side-effect harms (they are chemicals, and do have effects, including increased rates of suicide and homicide)
*The opportunity-cost of treating w/ meds (vs what else could use the money / R&D / professionalism for)
*What are the actual causes of depression? (see above, with documentation that 'chemical imbalance' theory isn't based on evidence, but external factors are often not acknowledged in medicalization of the problem)
... (etc) ...

[Jonathan Ichikawa]

Your case seems pretty compelling.

It'd be like arguing that medicine doesn't work on the grounds that if you give a sick person some random medicine off the shelf, the average positive effect would be small.

[Megan]

I'm a bit confused by your response, because it doesn't seem to actually address the placebo worries, about from saying that in your professional opinion your patients' responses to their medication didn't seem placebo. But I'm not sure why I should think that there would be a way to tell, on a case-to-case basis, whether an improvement in psychological health was due to medication or placebo effect.

"To see how multiple kinds of studies can provide good empirical evidence, consider one classic test of causation from the philosophy of science and scientific practice: tests of co-variation. You manipulate an independent variable (in this case, the drug a patient is taking) multiple times, and measure whether there are replicable differences in a dependent variable (in this case, symptoms)."

This sort of test seems like it would be unhelpful when trying to test for placebo, as presumably the patient is aware of whether she is or is not taking medication. If you are suggesting secretly replacing medication with sugar pills, this seems more promising although, as Anon noted above, I believe this has been done and *has* shown evidence of placebo effect.

Your point about the efficacy of prayer is also argumentatively confusing, as billions of people around the world find prayer extraordinarily effective, noting that their lives get markedly worse in the absence of it. So I'm not exactly sure of the point you're getting at, there.

[Amanda]

I'm curious if anyone really doubts that anti depressants work, and work well, for *some* people. That of course is consistent with them not having very good results when tested in large groups. As far as I'm concerned, if anti-depressants only work for 10% of the depressed population (and I think it's much higher than this, but just assume) that is plenty of enough reason to support their use. That 10% matters. And those who don't notice an improvement or think the side effects are too bad..they can stop using them. A low efficacy rate, of course, is also reason to keep trying more things so that better drugs are developed that help more people.

Anyway, as people like LIz Wood know, there are lots of other ways to do studies than the one Stegena mentions. Regardless of what you think of Marcus's response, that seems hard to refute.

[pendaran]

"Hi Marcus! I'm wondering if you could answer a question of mine about the aims of this piece? At most, it seems like the argument in this post shows that premise one of Stegenga's argument is false. However, it seems to replace it with a worse premise: "There is no test that can accurately determine the efficacy of antidepressants." If this is not the author's intent, and if the author thinks that testimony is enough evidence of the efficacy of antidepressants, then I'm concerned the result of this would be that homeopathy also enjoys strong testimonial evidence for its efficacy--potentially even stronger evidence than antidepressants."

I was going to write this comment but then found someone else already had. So, I guess I'll just say that I agree with Megan here.

I am willing to admit that I've been on anti-depressants before for depression and they seem to have helped, but I also must admit that we have little reason to believe they actually do. It was probably just a placebo effect.

[Amanda]

But we have positive evidence homeopathy doesn't work. And we have medical reasons to think that anti-depressants *can* have effects on the brain that we have reason to believe *can* mitigate depression. It is not as though scientists are just taking a random pill of a shelf and labeling them for use in depression. There is scientific study behind what sort of stuff goes into them. That is why testimony is different for anti depressants than for homeopathy. It is testimony plus knowledge of the type of chemical play at use. Whether or not that is good enough evidence depends on one's conception of evidence. Seems good enough for me, because there is no way to develop effective depression drugs, or other types of mental health drugs, without heavily relying on testimony.

[pendaran]

"But we have positive evidence homeopathy doesn't work."

We have very good gold standard evidence that anti-depressants have minimal or no effect and so do not work. This is Stegenga's point.

"And we have medical reasons to think that anti-depressants *can* have effects on the brain that we have reason to believe *can* mitigate depression."

I don't know but it's my understanding that Stegenga et al. have called the theoretical foundation behind these drugs into question.

"Whether or not that is good enough evidence depends on one's conception of evidence. Seems good enough for me, because there is no way to develop effective depression drugs, or other types of mental health drugs, without heavily relying on testimony."

The problem is that a few billion people on this earth believe very strongly based on their personal experience and testimony in homeopathy, Chinese medicine, crystal cures, or any combination of these things and/or many other such outlandish treatments etc. Entire cultures have revolved around fantasy and fantasy treatments and still do. And these people will say they have *seen* the cures work. They will swear up and down that they work and even choose these treatments when their lives are at stake. But of course we know that none of these treatments work.

[Amanda]

Homeopathic evidence is scientifically indisputable. That is not true wit anti depressants. There is nothing close to a consensus. It is not fair to compare the two.

You don't seem to get my point about testimony. Testimony is different when it is supplemented by scientific evidence about the type of chemicals that make up the substance. Not all testimony is the same. Whether or not we can trust testimony depends on other information we have about the subject being testified about. This is not controversial. In the case of anti depressants, unlike the case of homeopathy, we have additional supporting evidence which gives us extra reason to trust the testimony.

Are you saying that because, for any subject whatsoever, we can find examples of unreliable testimony, that therefore we have no reason to think testimony is ever reliable? We would know very little if that is true. The testimony we gather from homeopathic users is completely different from the testimony we gather about anti depressants.

Besides, what is the upshot of your point. Testimony about anti depressants can never be reliable, therefore we can never know anything about whether they work or not? And therefore, what, we shouldn't try to make any mental health drugs? Or we should make them and just be completely agnostic about whether they work? But if that is true, then there is no purpose in scientifically coming up with a chemical substance that you believe has reason to help depression. After all, there will never be any evidence to support its efficacy. I don't know where this is going.

[Derek Bowman]

Here's the thing about the placebo effect: it's an effect.

If I take a homeopathic remedy for cancer and I feel better because of the placebo effect, I'm still in trouble because my underlying condition has not changed.

If I'm depressed and anti-depressants make me feel better because of the placebo effect, then I'm no longer suffering from depression.

So if the best rebuttal to Marcus's points is "we don't know it's not the placebo effect," that's an interesting scientific/philosophical question (one that Marcus and Amanda have offered rebuttals to). But from a clinical standpoint it makes no difference - whether the effect is pharmacological or psychosomatic it's effective.

[Marcus Arvan]

What Amanda, Derek, and Liz Wood said.

There are many lines of converging empirical evidence that anti-depressants work wonders in some people: observational field studies, the science of neuropharmacology and biology, clinical practice and so on. When you have converging lines of evidence, it no longer merely anecdotal.

An analogy: I've never seen anyone do a randomly-assigned controlled trial on gravity, but I'm pretty sure my evidence for it isn't merely anecdotal. Like anyone else, I can test it systematically in daily experience (I just dropped my pen, there it goes!)--utilizing the method of covariation--and there are a lot of other reasons to believe that gravity exists.

Something like this is true for psychiatric meds. Amanda has it right: the mean effectiveness of anti-depressants may be small (maybe they have strong effects in only 10% of patients). But something like this is also true of cancer treatments (such as chemotherapy and radiation, which often fail). We use these treatments because we have *converging* evidence that they are the best treatments we have.

In contrast, there is ample converging sources of evidence that homeopathy and prayer don't work.

If these differences are seriously denied, then I quite frankly think this conversation is either a matter of some form of strange science (or at least psychiatry) denialism, or else an unjustified insistence that the only valid empirical test of causation is randomly-controlled trials--something that is not endorsed in any science, including but not limited to neuroscience and pharmacology (for basically the grounds I give in the OP).

[Russell]

Megan, you write that co-variation tests seem unhelpful in ruling out placebo effect, because the patient still knows they are on medication. I think you're missing the fact that, as Marcus emphasizes, antidepressant treatment usually involves hopping from one drug to the next, hoping one of them will work. In a particular case, drugs #1-3 might all fail to have any effect, but drug #4 will finally do the trick. If this is purely due to placebo, then why wouldn't it happen with drug #1? Why was only one specific drug, out of several, able to have the effect it did?

This is, as far as I am aware, a totally unique feature of antidepressant treatment. In the case of homeopathy, I suspect that the placebo effect kicks in on any drug tried by the patient, not just specific ones (though this is only a suspicion as I am unaware of whether this has been directly tested). As for prayer, it is most plausible to think that prayer works by improving someone's ability to respond to bad things happening to them, not by actually causing bad things to happen less frequently. (Again, not sure if it has been directly tested, but the kind of co-variation test Marcus has in mind could easily establish this.) So the individual experience of patients survives tests of co-variation in the antidepressant case, which it does not for pretty much anything else which people "swear up and down" is effective without good evidence.

[Dave Baker]

It seems like it would be possible in principle to put together a modified RCT that tested the effectiveness of the typical treatment strategy, by pursuing the usual process with the experimental group (switching between SSRIs until you find one that works), and then with the placebo group, switching between placebos until you find one that "works." I suspect such a trial would find in favor of SSRIs, especially as a long-term anxiety treatment.

Also relevant: https://slatestarcodex.com/2018/11/07/ssris-an-update/

[Amanda]

"or else an unjustified insistence that the only valid empirical test of causation is randomly-controlled trials--something that is not endorsed in any science, including but not limited to neuroscience and pharmacology..."

Oddly, I am running into more and more academics, typically philosophers, who seem to be claiming something like this. You can present all sorts of evidence to them, and they reply will be something like, "well that is not controlled scientific trial, you really have no real evidence..." Huh? It is very strange, and smells like the age old philosophical problem of philosophers being insecure because they are not recognized as scientists. But of course, scientists themselves wold never make such a claim about evidence.

Saying there is "no" evidence for the effectiveness of anti depressants is just such an over the top claim it is hard to know where to begin to refute it. Does the fact that some of the greatest minds in medicine spent decades developing chemical compounds for just this purpose mean nothing? We might as well pick up some homeopathic medicine instead of something that has been painstakingly worked on and studied for decades by some of the smartest minds in medicine, after all," there is no evidence" either of them are effective...

What really bothers me about all this is anti depressants help a lot of people. Those people are already in rough situations, people question the validity of mental health claims 100x more often than physical health claims. They also have to deal with people telling them nonsense like, "happiness is just a matter of attitude." And then in spite of this they find treatment that works, and then they have to deal with academics running around and claiming they are fooling themselves and might as well be taking a placebo. That has real power to hurt people. (This is not to say that if there was strong evidence that anti depressants didn't work, no one should write about it. It is just to say one should be careful when making these types of claims, and be damn sure there is good reason to support what they are claiming.)

[Marcus Arvan]

Amanda: exactly. I could not agree more!

[Adam P]

A few things worth noting:

I think Stegenga is correct that if RCTs were giving these same effect sizes on any other purported remedy e.g. homeopathy, we would very much conclude that that remedy does not work. Incidentally it's worth noting that some old meta analyses of RCTs on homeopathy found positive effect sizes in favour of efficacy. (This doesn't show homeopathy works, it shows experimenter expectancy effects and bias are very real).
https://www.ncbi.nlm.nih.gov/pubmed/9310601

The next question is what to do with this data. Stegenga thinks we should be consistent with how we would treat other remedies, several people here are trying to point to other kinds of evidence to explain the results to say we shouldn't treat them alike e.g. pointing to how many people get better after trying different drugs. Again, I think it is true that if we a observed similar thing in homeopathy, we wouldn't say therefore homeopathy has high individual variation in efficacy, we would still say it doesn't work, and we should say Stegenga's argument here is reasonable. It's also worth noting that because of their very noticeable side-effects, patients can often tell when they're having the real thing vs a placebo, and so observations of people getting worse once they stop and better when they start again can still plausibly be attributed to placebo effects.

However, I think there is some higher-order evidence which counts against Steganga. Marcus' experience seems quite widespread among people who actually work 'on the ground' so to speak. I have friends who are doctors, psychiatrists, or doing PhD's in psychology, and all say the received wisdom from their even smarter and more qualified co-workers is that most antidepressants have no effect for most people, but some have large effects that are not attributable to a placebo effect.

Additionally, Ioannidis, who originally authored the famous meta-analysis causing doubt on the whole field, authored a later, more thorough meta analysis which concluded they do work (d = 0.3). It thus seems like Steganga is taking a kind of bet. *Either Ioannidis, Marcus, and the current psychiatry establishment are all wrong, or he is*. Sure, philosophers sometimes benefit from being outsiders and can see bias no-one else can. But these people are all aware of the issues Steganga raises. They are equally qualified, if not more so, in the pitfalls of RCTs and biases in peer review, and they seem to be reaching the conclusion that antidepressants still work. Steganga is thus betting that he has worked out something these epistemic peers haven't, and I think when you're in that kind of position, philosophers would do well to use more caution. At the very least identify you are going against an establishment of actual authorities with more expertise and training, who are aware of all the arguments against their view, and yet don't agree with you. Steganga's argument is reasonable, and he's certainly not uninformed, but I'm betting on the establishment.

[pendaran]

I'm not an expert on any of this. But it seems reasonable to me that if randomized controlled trails find at best a modest effect for a drug or class of drugs over placebo (.3 about seems to be the effect size found for anti-depressants) that we should be cautious about whether that drug or class of drugs does much at all. This is especially true when we know most trails are done by Big Pharma, which have an obvious conflict of interest. And with anti-depressants, there are obvious side-effects, which harm the blinding. So, there is an additional concern relevant here for anti-depressants. Stegenga mentions other concerns that are interesting and worth thinking about, but I won't repeat them all now.

Maybe anti-depressants are effective. Maybe even though they aren't very effective overall they are effective for certain people, or maybe, for whatever reason, certain anti-depressants are only effective for certain people. Maybe the experiences that clinicians report cannot be explained by confounding factors. No one here is going to be able to prove that any of these conjectures are definitely false. Of course, I am sure that those who have experienced the drugs 'working' will adamantly insist on their efficacy.

However, it's not obvious that the experiences of clinicians cannot be explained by confounding factors: normal fluctuations in depression, confirmation bias, and the placebo effect. I doubt most of the people defending anti-depressants can prove they can account for these factors, or at least I have seen nothing convincing in these comments so far. It's possible for example that a clinician changes a patient's drugs until by chance the patient begins to improve on their own. They both then associate this improvement with the drug. As the patient believes the drug is making them better and fears that coming off will make them worse, a nocebo effect causes them to get worse when they stop, and a placebo effect cases them to feel better when they start again. As it's known that anti-depressants have notably strong placebo effects, the present commenters cannot be so sure placebo effects do not explain what they see.

In conclusion, I admit that I really don't know the truth about the matter. I've been on anti-depressants before, and they did seem to help. However, a sugar pill would have also helped. So, it's hard to say. I think we all need to stand back from our personal experiences and consider the data and arguments as objectively as possible, but it seems to me that some on this forum take this issue personally. Stegenga may not ultimately be right, but he raises reasonable concerns about the efficacy of these drugs.

[Amanda]

"However, it's not obvious that the experiences of clinicians cannot be explained by confounding factors: normal fluctuations in depression, confirmation bias, and the placebo effect. I doubt most of the people defending anti-depressants can prove they can account for these factors, or at least I have seen nothing convincing in these comments so far. "

First, interesting that we don't trust the clinicians to think about these possibilities. It takes a philosopher to point out to a clinician that these possibilities, in fact, can explain their experience.... (or it could be the clinician knows this, and is convinced the odds that the other explain things are low..)

Of course its *possible* those things could explain the results. The point is that would just be incredibly unlikely - it would involve a whole lot of confounding coincidence. So yes, if someone tries 3 anti depressants and they do not work, and the 4th one works - there are more than one way to read this. One way is that the 4th anti depressant (a drug designed to relieve depression) did in fact relieve the depression. Another way to read things is that for who knows what reason the placebo effect took place on time number 4, and not the other 3. The other possibility is that depression just went away on its own. (but had not gone away for years before, and during the time the 3 other drugs were tried.) The thing is, when this type of scenario happens again and again, it is just unlikely that the explanation is one of the second two.

Asking someone to "prove" that there is not other explanations for depression drugs efficacy is like asking someone to "prove" there is not other explanations for headaches going away after taking Tylenol. Yes, a headache could have just gone away on its own, yes, the person could be fooling themselves...but this is really an over the top standard of evidence to ask. In fact, you could make the claim with hundreds of different types of medication that it isn't really the medicine, it is just the disease coincidentally getting better on its own. So if that is the standard required to prove medicine works (i.e. proving that the disease didn't just happen to disappear) well, apparently hundreds of medications we assumed to work, well, we no longer have good reason to think they are effective.

[pendaran]

"First, interesting that we don't trust the clinicians to think about these possibilities. It takes a philosopher to point out to a clinician that these possibilities, in fact, can explain their experience.... (or it could be the clinician knows this, and is convinced the odds that the other explain things are low..)"

I don't find it interesting really. It's just kind of obvious that a clinician isn't going to have the objectivity of someone not working with mental illness directly.

"The point is that would just be incredibly unlikely - it would involve a whole lot of confounding coincidence. So yes, if someone tries 3 anti depressants and they do not work, and the 4th one works - there are more than one way to read this. One way is that the 4th anti depressant (a drug designed to relieve depression) did in fact relieve the depression. Another way to read things is that for who knows what reason the placebo effect took place on time number 4, and not the other 3. The other possibility is that depression just went away on its own. (but had not gone away for years before, and during the time the 3 other drugs were tried.) The thing is, when this type of scenario happens again and again, it is just unlikely that the explanation is one of the second two."

The idea is that depression has natural fluctuations. So, the depression was naturally getting better around the time that pill 4 was started. The clinician and the patient now associate pill 4 with relieving the depression. This causes a placebo effect for this pill, making it seem the pill has even more of an effect, and a nocebo if the pill is stopped. In fact, unless there are some factors by which we can determine which pills will work for which people at which times (I admit I don't know if there are), there is decent reason to think that this kind of deflationist explanation is what's really going on, because no reasons can be given for why pill 4 and not 1, 2, and 3 work.

"Asking someone to "prove" that there is not other explanations for depression drugs efficacy is like asking someone to "prove" there is not other explanations for headaches going away after taking Tylenol. Yes, a headache could have just gone away on its own, yes, the person could be fooling themselves...but this is really an over the top standard of evidence to ask."

Tylenol is a funny example. It's not hard to find articles about how it's largely useless.

"In fact, you could make the claim with hundreds of different types of medication that it isn't really the medicine, it is just the disease coincidentally getting better on its own. So if that is the standard required to prove medicine works (i.e. proving that the disease didn't just happen to disappear) well, apparently hundreds of medications we assumed to work, well, we no longer have good reason to think they are effective."

Hate to tell you but something like this is what Stegenga is doing. He's a medical Nihilist and believes that the vast majority of modern medicine is ineffective. You might disagree, but it's not an argument to simply state that you disagree. I am currently in the process of reading his book and find his arguments reasonable. He makes points I have thought of on my own over the years. The book has nothing like a proof, but I think it makes a reasonable case for skepticism.

Maybe the position that modern medicine is largely ineffective sounds crazy to you. Fine, you can choose to ignore the position and the arguments. But stating your bewilderment isn't an argument, and it isn't going to convince Stegenga that he doesn't have reasonable concerns.

Note: I suspect way less modern medicine is effective than is sold to us, but still think that by at least a slim majority modern medicine is effective. Stegenga has a more radical position. We agree in that we both think that we should be skeptical of modern medical claims.

[Marcus Arvan]

Pendaran: I have to confess that I find it very troubling when philosophers question entire scientific fields they don’t specialize in on the basis of armchair reflection.

There are legions of scientists who specialize in this stuff—who carry out countless studies, debate complex methodologies, and so on. And, as I understand it (following Liz Wood, Adam P., and others), there is little doubt among those who actually work in the science or clinical practice that anti-depressants do work (at least for a good number of patients).

On that note, I would encourage you to read the slatestarcodex Dave Baker links to above, which gives a thorough discussion of the issues—and which notes that one of the famous meta-analyses calling into question anti-depressants was redone by its own author and found something different, and another analysis indicating *very* high (0.7) mean effect of anti-depressants on anxiety.

When it comes to some of these discussions, my feeling is that some of the arguments being made are obtuse to scientific methods and standards of evidence.

For example, you keep taking about what can or cannot be "proven". You write:

"No one here is going to be able to prove that any of these conjectures are definitely false. Of course, I am sure that those who have experienced the drugs 'working' will adamantly insist on their efficacy.

However, it's not obvious that the experiences of clinicians cannot be explained by confounding factors: normal fluctuations in depression, confirmation bias, and the placebo effect. I doubt most of the people defending anti-depressants can prove they can account for these factors, or at least I have seen nothing convincing in these comments so far."

I will be frank. I think the epistemic standards you are working with here are absurd--and roughly the kind of standard that led "anti-vaxxers" to argue there is no scientific proof that vaccines don't cause autism.

Very few things are ever "proven" in science. There are *always* possible confounds. No scientist would dispute this. This is true of everything, including randomly controlled studies (which are simply *statistical* results). It is folly to hold science to a Cartesian standard of proof beyond all possible doubt--which seems to me what is going on when you and others say it "could" all be placebo effects and whatnot. Yes, it *could* be...but scientists and clinicians working in these areas will tell you that that is not what the balance of evidence shows. And, if you don't know that scientific literature and clinical practice in and out, one should be more humble--I think--about the philosophical arguments one makes, especially in posts to the general public.

[Amanda]

Well, okay I didn't realize I was dealing with total doubt of modern medicine and medical professionals. Yes, it seems in that case our starting points are so far off conversation is not frutful. But why focus on mental health rather than anything else. Most of the time it could, theoretically, have been a placebo effect.

[Amanda]

I also find it kinda ironic you admit Stegenas arguments are nothing like proof, yet you find them compelling, and yet you keep criticizing the other side for not providing proof.

[pendaran]

"There are legions of scientists who specialize in this stuff—who carry out countless studies, debate complex methodologies, and so on. And, as I understand it (following Liz Wood, Adam P., and others), there is little doubt among those who actually work in the science or clinical practice that anti-depressants do work (at least for a good number of patients)."

I'm not sure if I know what the argument is, but under my standard attempts to disambiguate it I think it's clear that Stegenga has thought of this argument and would find it very unconvincing.

"On that note, I would encourage you to read the slatestarcodex Dave Baker links to above, which gives a thorough discussion of the issues—and which notes that one of the famous meta-analyses calling into question anti-depressants was redone by its own author and found something different, and another analysis indicating *very* high (0.7) mean effect of anti-depressants on anxiety."

I'm not an expert but from what I've read meta-analyses generally find modest effect sizes for anti-depressants, and I don't think this has changed as of late. However, if the meta-analyses show very high effect sizes, then of course I'd be willing to say that we have much better reason to believe that anti-depressants work. Stegenga and I are here working under the assumption that the best meta-analyses show at best a modest effect size of around .3, with .2 being considered low and .5 being considered medium.

"I will be frank. I think the epistemic standards you are working with here are absurd--and roughly the kind of standard that led "anti-vaxxers" to argue there is no scientific proof that vaccines don't cause autism."

I don't think they are absurd. I don't know anything about the arguments of anti-vaxxers. I think bringing up anti-vaxxers is a straw man or red herring, unless you can show how Stegenga's position implies that the anti-vaxxers are right. In his book he clearly states he approves of vaccines.

"It is folly to hold science to a Cartesian standard of proof beyond all possible doubt--which seems to me what is going on when you and others say it "could" all be placebo effects and whatnot."

I think this is a straw man. The claim isn't that we shouldn't believe that anti-depressants work, because we don't have Cartesian level proof they work. The claim is that all things considered we have no good evidence they work. Here I'm just stating what I think is Stegenga's position. It is the latter not the former. (I admit I'm not 100% clear on his position and its best formulation.)

Sorry that I can't respond more thoroughly. I know I'm being quick but I don't have the time for longer responses right now.

[David Mathers]

I think it would help a *great deal* if, rather than shouting that they are right, people had a look at any info that is out there (I don't know how much is) on the general rate of regression to the mean with depression (i.e. people going back to non-depressed having been depressed absent treatment) and the *typical speed* of such regression. If we knew this, it would be much easier to work out how likely it was that people would suddenly dramatically improve right when they were switched to their 5th anti-depressant, even if the drug wasn't actually having an effect. Without some kind of serious statistical work using info on this stuff, all you can really sensibly say is that Marcus *might* be right that non RTC evidence supports occasional large effects in some individuals, but also *might* just be being fooled by regression to the mean.

[Pete]

Marcus,

Great post, I actually read it shortly after making my way through the Aeon article. Shortly after that there was an interesting article from phys.org (I believe the health related articles link to "medicalxpress.com"):

https://medicalxpress.com/news/2019-03-neurons-antidepressants.html

I wanted to post this sooner and other things got in the way, but I thought this might offer up some interesting evidence of physical differences that might get us to really start understanding why these drugs don't work on certain subsets of patients. Would love to get your thoughts. Interesting excerpts below:

"The cause of depression is still unknown, but scientists believe the disease is partly linked to the serotonergic circuit in the brain. This is largely because SSRIs, which increase levels of the neurotransmitter serotonin at neuron connections, help alleviate the symptoms of many people diagnosed with depression. Yet, the mechanism of why some people respond to SSRIs, while others do not, remains a mystery. Unraveling the puzzle of SSRI resistance has been challenging because it requires studying the 300,000 neurons that use the neurotransmitter serotonin for communication within a brain of 100 billion total neurons. One way scientists have recently overcome this obstacle is to generate these serotonergic neurons in the lab...Neurons from SSRI non-responders had longer neuron projections than responders. Gene analysis revealed that the SSRI non-responders also had low levels of key genes (protocadherins PCDHA6 and PCDHA8) involved in forming neuronal circuits. When these genes were made non-functional in serotonergic neurons (mimicking the low gene levels previously observed), the neurons developed the same unusually long projections in the SSRI non-responders. These abnormal features could lead to too much neuronal communication in some areas of the brain and not enough in other parts, altering communication within the serotonergic circuitry and explaining why SSRIs do not always work to treat MDD."

[Andy]

"As someone who has seen anti-depressants literally save lives--indeed, as someone who has seen people go from suicidal despite many other interventions to suddenly becoming stable after finding the right drug, and as someone who has seen people very close to me maintain their mental well-being and ability to function in society for years or decades until the moment they go off their meds (or their doctor changes their meds), suddenly lapsing back into depression until they take the same med again, and whose symptoms worsen when they try anything else--I am, to put it mildly, unpersuaded that it is chance fluctuations, confirmation bias, or placebo effect. The best explanation of ample clinical experience, it seems to me, is that anti-depressants can have genuine, and very powerful, individual-level effects."

Marcus, I want to offer an alternative explanation for what you have seen. After being on antidepressants for a short time people become dependent on them. If they are withdrawn from them they can often get severe abstinence symptoms. Often these symptoms can be confused with a relapse of the original problem. When the patient then reinstates the drug the recovery can be remarkable. However, this does not necessarily mean that the drug is effective at treating depression. It simply means that it is effective at treating a dependence that it created in the first place. Of course it can be very difficult to separate out depression from the withdrawal symptoms since they can be so similar. Anyone witnessing this improvement would most likely come to the conclusion that the drugs are very effective, however that is not necessarily the case.