It’s not quite so cut and dried.

Recently, several well designed psychotherapy studies have been published, and they illustrate some of the complexities involved in evaluating the evidence on psychotherapy efficacy. Before we get into the studies, let’s look at some of these methodological issues.

Hundreds of psychotherapy efficacy studies have been published over the last few decades, and most of them have reported a positive effect of the technique under study (specific references are cited below). But there is controversy about the adequacy of the control groups used in these studies.

Recall that clinical studies of medications randomize some patients to a pill placebo group in order to verify that the active treatment works through a specific biological mechanism, rather than through faith and a desire to please the doctor. For similar reasons, psychotherapy studies randomize patients to a control group in order to demonstrate a given therapy technique's efficacy. Most commonly, the control group is a waiting list.But the wait list control is suboptimal, because unlike the case for pill placebo, wait list patients don’t actually believe that they are getting treatment. Nonetheless, studies have shown that simply being put on a wait list results in substantial improvement, due both to “spontaneous remission” (i.e. the passage of time) (Posternak M et al., J Aff Disorders 2000;66:139-146) and to the therapeutic effects of receiving a baseline evaluation and follow-up symptom measurements throughout the study. These two effects combined aren’t as robust as pill placebo, but they come pretty close (Posternak M et al., Br J Psychiatry 2007;190:287-292).

A more rigorous test of therapy’s effectiveness would be to compare psychotherapy with a gold standard treatment, and in psychiatry, that gold standard is medication treatment. If therapy works as well as medication, especially if they both beat placebo, we’re tempted to conclude that therapy is, indeed, effective.

Here are some therapy findings that have recently received attention.

CBT has an established track record for treating a variety of disorders. The latest huge review of CBT, which was a “mega”-analysis reviewing 16 different meta-analyses, listed the following disorders for which CBT shows a large effect size: major depression, panic disorder, generalized anxiety disorder, social phobia, PTSD, and depressive and anxiety disorders in children (Butler AC et al., Clin Psychology Rev 2006;26:17-31). This sounds impressive, but keep in mind our caveats about methodology. In many of the reviewed studies, the high effect sizes were in relation to waiting list groups, and few studies included placebo.

In bread and butter psychiatric practice, CBT has developed a reputation for being good for moderate depression but not for severe depression. This was based on a single large study published in 1989 in which cognitive therapy, interpersonal therapy, and imipramine were all compared in major depression. While the two therapies performed as well as imipramine overall, for the subgroup of more severely depressed patients, imipramine was more effective than therapy (Elkin I et al., Arch Gen Psychiatry 1989;46:971-982).

Because of this finding, APA treatment guidelines recommend medication over therapy for severe depression. However, two years ago, another paper was published in Archives that appears to refute this wisdom. In this study, 240 patients with moderate to severe depression were randomized to paroxetine (n = 120), CBT (n = 60), or placebo (n = 60). Notice the good methodology: therapy is compared with both a medication and a placebo group. After eight weeks, there were no significant differences between the responses to paroxetine (50%) and CBT (43%); both active treatments were superior to placebo (25% response rate) (DeRubeis RJ et al., Arch Gen Psychiatry 2005;62:409-416).

Relatively little research has been conducted on psychodynamic therapy, even though it appears that this technique is still more popular among psychiatrists than CBT.

The best meta-analysis of short term psychodynamic therapy (STPP) was published in 2004 (Leichsenring F et al., Arch Gen Psychiatry 2004;61:1208-1216). Researchers identified 17 randomized controlled studies of STPP, all of which had clear, manual-based definitions of the treatment used. Disorders studied included depression, PTSD, bulimia nervosa, GAD, personality disorders and several others. Because there were so few good studies, the authors could not analyze separately the effects of the technique on specific disorders, but they reported that there was a large overall effect size for a variety of symptoms as measured in scales such as the Hamilton depression scale. However, the control groups were either waiting list or “treatment as usual.”

Such is the paucity of good data and so great is the eagerness for proof of efficacy that when a halfway decent study shows that psychodynamic therapy works, it is released with an embarrassing degree of fanfare. This was the case for the June issue of the American Journal of Psychiatry, which contained a study comparing three different therapies for borderline personality disorder (Clarkin JF et al., Am J Psychiatry 2007;164:922-928). In this study, 90 patients with BPD were randomly assigned to transference-focused psychotherapy, dialectical behavior therapy, or supportive therapy. After one year, patients improved in all treatments, with no significant differences between them. But the lack of a placebo control group makes it impossible to know if the improvements were actually due to the treatments or due to nonspecific effects from any encounter between patients and therapists.

Nonetheless, an accompanying editorial gushed that the study “establishes Kernberg’s brand of psychoanalytic object relations therapy as a treatment that produces substantial change in a 12-month period….” (Gabbard G, Am J Psychiatry 2007;164:853-855). This enthusiasm seems to far overreach the results of the study. Left unanswered is this core uncertainty: Does psychodynamic therapy contain a special effective ingredient not contained in other techniques? Unknown.