Editorial, J Trauma Stress Disor Treat Vol: 1 Issue: 1
Methodological Limitations of the RCT in Determining the Efficacy of Psychological Therapy for Trauma
| Nigel Hunt* |
| University of Nottingham, UK |
| Corresponding author : Nigel Hunt University of Nottingham, UK E-mail: nigel.hunt@nottingham.ac.uk |
| Received: June 11, 2012 Accepted: June 19, 2012 Published: June 21, 2012 |
| Citation: Hunt N (2012) Methodological Limitations of the RCT in Determining the Efficacy of Psychological Therapy for Trauma. J Trauma Stress Disor Treat 1:1. doi:10.4172/2324-8947.1000e101 |
Abstract
Methodological Limitations of the RCT in Determining the Efficacy of Psychological Therapy for Trauma
Which therapy to use with traumatized individuals is always a difficult choice, particularly as the evidence regarding the therapies available is often contradictory.
Currently, the National Institute for Health and Clinical Excellence (NICE) Guidelines in the UK recommend, on the basis of a number of trials and reviews, that practitioners in the National Health Service (NHS) only use trauma-focused cognitive behavior therapy (TFCBT) and eye movement desensitization and reprocessing (EMDR).
| Which therapy to use with traumatized individuals is always a difficult choice, particularly as the evidence regarding the therapies available is often contradictory. | |
| Currently, the National Institute for Health and Clinical Excellence (NICE) Guidelines in the UK recommend, on the basis of a number of trials and reviews, that practitioners in the National Health Service (NHS) only use trauma-focused cognitive behavior therapy (TFCBT) and eye movement desensitization and reprocessing (EMDR) [1]. There is little attempt to consider the methodological limitations of the research which identifies these two therapies as the most effective. While it is true that TF-CBT and EMDR work well for many people, they are not effective for everyone. While the NICE guidelines do recognize drug treatments for individuals who express a preference not to engage in trauma-focused therapy there is no recognition of other forms of therapy. | |
| Theoreticians and clinicians disagree about the efficacy of different kinds of therapy for traumatic stress disorders. The evidence for and against each type of therapy is contentious and, while we are rightly searching for the ‘best’ therapies, we are at the same time - in my view - leading ourselves up several garden paths. Here I want to focus on two of these. The first is methodological. The randomized controlled trial (RCT), while it may be an ideal approach for much of medical science in terms of treatments and therapies, is not always the most appropriate for assessing psychological therapies for traumatic stress disorders. The second point is that there is no any single ideal therapy for traumatized individuals. Cognitive Behavior Therapy (CBT) may be effective for some, but it is unbearable to others. There may be limited scientific support for psychoanalytic psychotherapy, but that does not mean it is not the most appropriate for some traumatized people. | |
| These two points are related. If we are going to find out how to effectively treat traumatized people we need to know both which therapies are effective for at least some people (currently we rely largely on RCTs for this), and then match the therapy to the person. Both are difficult to achieve. It is contentious whether it is appropriate to use the medical model (which assumes that the RCT is the ideal approach), from which it follows that the best form of literature review is the systematic review. Systematic reviews draw heavily on RCTs and downplay the importance of other kinds of design. Added to this is the problem of matching the therapy to the person, and very little research has explored this question. | |
| The limitations of the RCT are well documented elsewhere, including within medical science [2]. There are a number of key weaknesses of the RCT, such as the costs (resources, money) of running a full trial, problems with double-blinding, the ethical implications of limiting what the researcher believes to be an effective treatment to only one group, informed consent, the lack of applicability to the wider population because of the stringent selection techniques, recruitment bias due to some people refusing to take part, or because they are too ill (or not ill enough), and the results may not reflect real life. Some of these problems can be overcome, but others which are difficult to deal with in the medical context, become impossible to deal with in the context of psychological therapy. For instance it is almost impossible to fully blind a trial of a psychological therapy as the patient is likely to know whether or not they are receiving therapy. Recruitment bias is always going to be difficult with traumatic people, many do not want to go through a ‘talking therapy’ because they do not want to confront their memories (this links to the issue of getting the right therapy for the right person). It is also very difficult to match the seriousness of the traumatic disorder, particularly in cases of complex trauma. Furthermore, therapies for traumatic stress are often used in dangerous environments such as war zones or places where there has been a natural disaster, and it is very difficult or impossible to conduct a well-controlled study. Ethically, it is more important to do something that is likely to help than exclude people from the treatment arm. | |
| There are also problems when reviewing the literature on RCTs to determine the effectiveness of a therapy. The first relates to publication strategy. The salami-slicing of results may lead to problems with systematic reviews and meta-analyses. Rosner (2003) found that there were 44 publications for 31 clinical trials, resulting in an 18% oversampling in a meta-analysis [3]. By using the same data in several publications, support for a particular therapy may be exaggerated. Systematic reviews themselves are problematic in this area. As they rely largely on RCTs, and as these have severe limitations in psychological therapy, those reviews that are published are biased towards therapies that can be conducted in a fairly controlled manner in ‘safe’ environments, and as we have seen, therapy for trauma is often used in difficult and dangerous situations where it is not possible to use an RCT. Furthermore, another major problem with reviewing the literature is that a plethora of means of assessing trauma symptoms are used, which limits the opportunities for meta-analyses. | |
| Currently, it is getting more difficult to have review papers accepted in many journals unless they are using systematic review methods; this means that people are encouraged to use RCTs to evaluate therapies, a method which, as we have seen, has a number of significant problems; which are exacerbated if it is the case that people require different therapies for successful treatment. If research into therapy continues in this way it will go in ever decreasing circles and it will be difficult to progress to a position where we acknowledge the value of a wide range of approaches. | |
| By failing to take into account individual differences regarding the need for different therapeutic approaches, current research has a critical limitation. This limitation becomes more significant when we recognize that people have different types of traumatic stress disorder, along with varying comorbid disorders such as anxiety and depression. Partly, these limitations derive from a statistically-driven approach to research, which requires providing numerical data which can be analyzed to determine difference, with the effect that genuine individual differences are treated as error variance - a perennial problem in psychological statistics. There is also a problem with dropout rates. While numbers of dropouts are usually recorded, it is rarely known why they have dropped out because they usually just fail to turn up for a session and are then not followed up to explore why. Finally, current research is usually based on the erroneous assumption that there is a single best therapeutic approach. Inevitably, this relates to the fashions of the day. After the Second World War behavioral approaches were more popular; from the 1960s a cognitive behavioral approach became popular and has continued to dominate until the present day. The use of EMDR was ridiculed when it first appeared in the 1990s; now it is seen as an effective treatment. Then there are those who advocate psychoanalytic, existential/humanist or other approaches. The NICE guidelines are not only driven by an RCT approach, they are also driven by cost-effectiveness, which means that short relatively cheap therapies are favoured over longer term more expensive approaches. | |
| Most therapies for traumatic stress do have some overlap, but there are clear differences. Some therapies require explicit focus on the traumatic memory, others involve more implicit approaches; some explore behavioral consequences, others have less focus on this. While research has not elucidated which therapies work best for which people – and that should be a key focus of research – it has shown that no one therapy will work for everyone [4]. The NHS recommends TFCBT and EMDR. These work for a proportion of traumatized people, though we don’t know what proportion. Future research should focus on the key individual differences that determine which kind of therapy is going to be most effective for a particular individual. This does mean that the standard RCT will not always be the best method, and that in turn means that we need a more sophisticated approach to methods and reviews than that provided by systematic reviews. | |
| There are several routes to conduct research to determine the key individual differences in therapy efficacy. In the first place we have detailed clinical assessments and observations about people who have received individualized treatments from eclectic therapists. Secondly we do know something about individual differences in relation to trauma, and which variables are important. We can use these to explore the effectiveness of different kinds of therapy. Finally, qualitative approaches explain people’s deeper views and feelings regarding trauma and therapy - such approaches may enhance our understanding of these differences. | |
| By limiting ourselves to using RCTs and systematic review methods we are limiting our understanding of the best ways to treat traumatic stress. We need to develop new strategies to account for critical individual differences and find ways of mapping the most appropriate therapy to patient. While this is done implicitly by eclectic therapists there is still a view that certain types of therapy are effective and others are not. At this point we do not really know the mechanisms of this; we do not even know whether it is a problem with the therapy or a problem with appropriate allocation of patient to therapy. While we know TFCBT and EMDR are effective, we do not really know how effective they are both in terms of the proportion of traumatized people helped by them and the longer term outcomes. We need to more fully research other therapies, but in order to do so effectively we must be able to match the patient to the therapy. RCTs may be very effective in the medical arena, but they have limitations, particularly in psychology. Funding bodies should recognize this, and recognize the need to use different kinds of methodological approaches, ones which are more suitable to psychology. | |
| If we are to use RCTs - and we should - it is critical to find the best means of allocating patients to the appropriate treatment condition. If a person is not suited to CBT they should not be in the study. While clinicians have a mass of data to determine the best course of therapy, and we should be drawing on these data, taking this approach does depend on the clinician having access to explicit assessment information and to the full range of therapies. Where this information is available we need to develop the means to validate these assessments and ascertain the individual factors that play a critical part in this. This becomes particularly important when therapy is conducted in the field, in dangerous or difficult environmental conditions where we need a rapid and accurate assessment of need. | |
| The implications of these ideas are: | |
| 1. The need to acknowledge that a wide variety of therapies may be appropriate. Different therapies suit different people, and we need to develop the tools to determine the best fit of the individual to the therapy. This is where research is urgently needed. | |
| 2. While RCTs are an excellent method for determining the effectiveness of a treatment or therapy, they have serious limitations, including practical limitations, and researchers, funding bodies and journal editors should recognize this | |
| 3. Following on from this, if we recognize the need for a variety of methods to assess the efficacy of therapeutic interventions, then we must recognize the limitations of the systematic review and metaanalytic methods, and use other forms of review such as the narrative review. | |
| 4. There is a need for policy changes. The NHS in the UK not only restricts the therapies available for traumatized people, but actively attempts to implement cheaper and faster therapies. While this may be economically and politically appropriate, it is not always clinically appropriate. | |
References |
|
|
