Roth and Fonagy have tackled an
enormous and difficult question with all the nuance and intellectual muscle required
in their second edition of What Works for Whom? They have not shied
away from deeply examining the pros and pitfalls of quantitative psychotherapy
research, and they have scoured the English language evidence thoroughly in
their search for randomized controlled trials and meta-analyses. They have
also organized their book by disorder, primarily relying on categorical
definitions of disorder, as found in DSM-IV-TR¹, though disorders can be easily matched to their ICD-10 counterparts². The book has a tremendous potential for good use and misuse, and the authors are aware of this, taking pains in many places to point out the limits of what their findings do and do not support, especially in terms of managed care. This book is not for a lay audience, and general readers might quickly find themselves lost in a whir of scientific terminology, nuanced philosophical debate, and truckloads of statistics. For anyone with an interest in psychotherapy research of a quantitative nature, or for anyone with an interest in the philosophy of science, this is an
excellent discussion of theoretical issues and a timely, exhaustive reference.

The book starts with an effort at "Defining
the Psychotherapies," accepting Strupp’s definition of "an
interpersonal process designed to bring about modifications of feelings,
cognitions, attitudes and behavior which have proved troublesome to the person
seeking help from a trained professional." They next point out that Kazdin
identified over 400 therapies in 1986 and state they will focus on "major
orientations," defined as psychodynamic psychotherapy, behavioral and
cognitive-behavioral psychotherapy, interpersonal psychotherapy, strategic or
systemic psychotherapies, supportive and experiential psychotherapies, group
therapies, and counseling (p. 6). The remainder of the chapter goes on to
define these schools, and steers the reader toward seminal works by founders or
modifiers of the various schools.

Chapter 2, "Research and
Practice: Methodological Considerations and Their Influence on This Review"
is, in my view, the foundation of the book and possibly its most important
contribution to the theoretical literature on psychotherapy research. I would
insult the chapter by attempting to summarize it, but it does tackle a number of
important issues which can be made plain. Psychotherapy is a very complex
interaction between two or more people. It involves a sufferer and an
individual attempting to understand and bring relief. These individuals are
informed by their life experiences, temperaments, expectations, and, in the
case of the therapist, theoretical orientations within psychotherapy. The
number of potential variables which need to be controlled in order for a
quantitative approach to produce meaningful results is fiendishly large.

For instance, what and who defines
suffering? The suffering reported by the individual is frequently at odds with
what is observed, but quantitative methods rely on observation. How do we know
when therapy proper is taking place, as opposed to say, a sympathetic hearing?
Psychotherapy can be defined in a very broad way, as in the Strupp definition,
but operationalizing it for research purposes is difficult. Then the issue
arises, even once a technique can be recorded in a manual, and even if an
inexperienced therapist is fully adherent to the letter of the manual,
something of the spirit may get lost. What are the differences between people
recruited into psychotherapy trials through newspapers compared with those who
present with difficulties at clinics? We know very little about what motivates
someone with psychological difficulties to seek help from a clinic versus
responding to an ad, therefore it is difficult to generalize from research
findings to clinical practice.

There are also questions about
research populations. On what basis can we make generalizations from a highly
selected research population to a completely unselected clinical population? Research
populations need to be as homogenous as possible, for example, depressives
only, with no comorbid anxiety, or substance abuse, so that the study minimizes
confounding variables and maintains internal validity. However, the clinic
takes all comers, with all degrees and combinations of symptomatic
difficulties. Studies of efficacy, based on homogenous clinical samples
studied under controlled research conditions do not establish effectiveness,
which is the ability of a modality to handle the problems typically seen in a
real-world clinic, where comorbidity of 50% to 90% between DSM disorders is
common (p. 487).

Questions about placebo controls or
other control interventions are important as well. Sitting on a waiting list
or, "doing nothing," is currently one of the controls used, with
assessment instruments applied to wait-list controls and individuals in
therapy. Just about every psychotherapy looks good against doing nothing, and
effect sizes diminish remarkably when two therapies are compared head-to-head.
Allegiance effects are also known, whereby the preferred therapy, often CBT, is
administered by experienced therapists, versus the "control therapy,"
which is often done by inexperienced therapists (p. 95). Regression analysis
virtually eliminates the superiority of the preferred modality.

The questions go on and on, with
very deft handling by the authors, who frequently conclude that there are
neither easy nor self-evident replies. Quantitative approaches are limited in
what they can tell us. Roth and Fonagy further note that there are
psychotherapeutic schools, such as psychoanalytic or psychodynamic, which
thoroughly repudiate quantitative approaches as wrong-headed, preferring
qualitative, in-depth case studies. As a result of this bias, the voice of
psychodynamic therapy is notably absent in the book. The authors appeal to
psychodynamic practitioners to consider the potential usefulness of controlled
trials in proving the usefulness of their approaches. Peter Fonagy, the Freud
Memorial Professor of Psychoanalysis at University College of London, can
hardly be said to be a detractor of psychodynamic approaches, but he clearly
swims against the psychoanalytic stream with his interest in quantitative
research. The second chapter will be of greatest interest to philosophers and
students of quantitative methodology, and this is why it garners the lion’s
share of this review.

The third chapter is largely a
warning to those who fund mental health services not to simplistically handle
evidence. For instance, good evidence shows that depression responds to 16
sessions of CBT. A mental health planner might conclude that only CBT will be
funded for depression, for up to 16 sessions. But good evidence also shows
that people who have only had 16 sessions have only a 25% chance of being well
one year later, while those getting maintenance therapy are more likely to
avoid a relapse. But maintenance therapy is expensive. Economically, it may
make more sense for an insurance company to limit treatment to 16 sessions and
then not renew the individual’s benefits because of their high risk of
relapse.

Further, absence of evidence for a
particular brand of therapy is not the same as proof of inefficacy, and what
counts as evidence is very much in the eye of the beholder, whether that is the
therapist, researcher, patient, or payer. Roth and Fonagy point out that rigid
funding policies would eliminate second line treatments, necessary for those
not responding to initial approaches. These policies would also dampen
clinical innovation. There would be little point innovating if it wouldn’t end
up being paid for due to its non-evidence-based status.

Chapters 4 through 15 are the meat
of the book, where Roth and Fonagy explore various specific disorders, looking
at published controlled trials comparing various therapies to controls or else
to each other head-to-head. The main message that jumps out from these
chapters is that the proponents of CBT have clearly done their homework.
Researchers have tried CBT in a randomized controlled trial for virtually every
psychiatric disorder, and it often produces positive results, particularly for
depression and anxiety, the two sorts of disorders which have always been most
responsive to psychotherapy. Other modalities have a long way to go,
particularly psychodynamic, that is, if their proponents ultimately decide that
there is anything to be gained from a quantitative effort.

Chapter 16, "The Contributions
of Therapists and Patients to Outcome," is another interesting exploration
of both published literature and theoretical discussion. It turns out to be
rather hard to account for why different therapists produce detectably
different outcomes (p. 475). Studies looking at associations typically have
too few patients per therapist to detect meaningful differences. Studies
looking at therapist experience tend to confound the fact that experienced
therapists usually end up with the most difficult patients, so on paper may
look much less effective than novice therapists. To date, all studies of
therapist characteristics have been in the secondary analysis, not built into
the initial design features of the study, which severely limits their value.
The study which randomizes patients to therapists, rather than to therapeutic
modality, has not yet been undertaken.

The book’s "Conclusions and
Implications" (Chap 17) are again long, nuanced, and replete with
cautions, all well justified by the discussion. Furthermore, even when only
one disorder is cornered, it is clear that the symptoms themselves could be the
result of numerous causal pathways, even if they look broadly similar across
patient or study populations. Thus far, psychotherapy research has looked only
at diagnoses, not at whether particular psychotherapies may be uniquely suited
to particular causal factors such as childhood abuse. Crucially, they also
note that researchers only tend to publish results showing a positive
difference due to their espoused intervention. Negative or neutral studies get
filed away, so the scientific literature is replete with publication bias.

There is much more I’d like to have
discussed in this book, but it’s just too jam-packed for a short review to do
it adequate justice. The issues are of critical relevance to those who seek
psychotherapy, those who provide it, and those who pay for it. There are also
warnings and guidance to those who wish to conduct adequate research using an
RCT paradigm. Payers like insurance companies look to shibboleths like the
American Psychological Association’s "Empirically Supported Therapies"
for guidance and possibly as a defensive shield. To gain the APA’s imprimatur,
a therapy has to have been proven by RCT. Unsurprisingly, CBT has far and away
been the winner, bolstered by legions of researchers who ascribe to the cognitivist
presuppositions which currently undergird modern psychology yet are seldom
scrutinized critically. As a result, access to psychotherapy has massively
eroded across the United States, and patients are the losers. It is
interesting that the U.K. Department of Health chose to reject the EST approach
in its mental health policy (p. 51). A close reading of Roth and Fonagy is an
important tonic, cutting through a lot of dogma and forcing the reader to confront,
again and again, that the answer to the question "What works for whom?"
if phrased simply would be, "Some things, some of the time, for reasons
which aren’t yet very clear."

 

Reference
List

 

1. American Psychiatric Association. Diagnostic
and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision. Washington, DC: American
Psychiatric Association, 2000.

2. World Health Organization. International
Statistical Classification of Diseases and Related Health Problems 10th
Revision [Web Page]. 2003; Available at
http://www3.who.int/icd/vol1htm2003/fr-icd.htm.

Robert Tarzwell is in his 5^th
and final year of psychiatry residency at Dalhousie University in Halifax, Canada.
He is interested in the philosophy of psychiatry. He is also interested in
critical approaches to psychiatric literature, particularly randomized
controlled trials of drugs and psychotherapies. His main clinical interest is
intensive dynamic psychotherapy.

Categories: Psychotherapy