Research Article, J Spine Neurosurg Vol: 9 Issue: 0

Patient Outcome Questionnaires in the British Spine Registry: Why are Response Rates Low and Which Patients Groups are responding?

Gregory Cunningham¹, Dan Wright¹, Colin Nnadi C¹ and David C Kieser^2*

¹Division of Spinal Surgery, University of Oxford Hospital NHS Foundation Trust, Oxford, England

²Department of Orthopaedics and Musculoskeletal Medicine, University of Otago, Christchurch School of Medicine, Christchurch, New Zealand

*Corresponding Author: David C Kieser
Spinal Surgeon, Department of Orthopaedics and Musculoskeletal Medicine
University of Otago, Christchurch School of Medicine, Christchurch, New Zealand
Tel: +64 211499829
Email: kieserdavid@gmail.com

Received: July 27, 2019 Accepted: February 18, 2020 Published: February 27, 2020

Citation: Cunningham G, Wright D, Nnadi CC, Kieser DC (2020) Patient Outcome Questionnaires in the British Spine Registry: Why are Response Rates Low and Which Patients Groups are responding? J Spine Neurosurg 9:1.

Abstract

Keywords: Registry; Patient reported outcome measures; British spine registry; Public health; Registry data

Introduction

Registry; Patient reported outcome measures; British spine registry; Public health; Registry data

Introduction

A surgical registry is a collection of data on a condition or surgical procedure that is organised to allow analysis of the recorded data.

The first national registry for orthopaedic surgery was developed in 1975 in Sweden [1], and showed the benefit of national registries for population based healthcare improvement by enabling hypothesis generation and subsequent focussed research [2]. Driven by prior registry success and an appetite for large data analytics to identify concerns and solve common problems, the number of registries listed internationally has exponentially increased.

In the United Kingdom (UK), the first similar registry was developed in 2003 to capture data on hip and knee arthroplasty [1]. However, some centres in the UK have also been involved with the first international spine registry, called Spine Tango, that was launched from Berne, Switzerland in 2002 [3].

More recently, the British Association of Spinal Surgeons (BASS) instituted the British Spinal Registry (BSR) in order to collect spinal data specific for the UK population. This was launched in 2012 [4], with the aim to collect clinical and patient outcome data from all patients undergoing spinal surgery and to use that data to improve patient safety and improve clinical decision-making. Expectedly, the activity and outcome data from this registry is also carefully scrutinised by employers, insurance companies and patients. However, like all registries, the validity of the BSR relies heavily on patient engagement and completion of the outcome questionnaires at each time point before and after surgery. Thus, an incomplete registry dataset risks being non-representative of the population it is studying and misinterpreted by those who do not recognise its limitations.

In order to capture patient specific outcomes, the BSR sends out questionnaires via email at specific time points to all patients who have consented to be involved. If a response is not obtained patients are sent a further three reminder emails. Failure to complete any prior questionnaires does not preclude patients from receiving future questionnaire emails. The data obtained is then stored for subsequent analysis.

Anecdotally, it was felt that patient compliance with completing the patient rated outcome scores sent to them from the BSR might be poor. This was confirmed in a recent report by the BSR that identified a complete data set collection of only 30% of recruited patients [5]. However, a specific understanding of which patients were being represented by the registry remains unclear.

This study retrospectively analysed the outcomes recorded in a single surgeon’s BSR record over a two year period in order to evaluate feedback rates and to stratify patient groups according to response rate.

Patients and Methods

Patients enrolled into the BSR by a single surgeon (CN) undergoing surgery for either lumbar degenerative or deformity diagnoses were retrospectively analysed for compliance with completion of the required questionnaires. It is this surgeon’s practice to proactively recruit every consenting patient to the BSR in his preoperative assessment clinic. Once a patient is recruited they are allocated time within the consult to complete their baseline outcome measures on a tablet computer.

Like all patients within the BSR, the patients were contacted via email by the BSR at sequential time points post-operatively (6 weeks, 6 months, 1 year and annually thereafter) and requested to complete standardised questionnaires including the Oswestry Disability Index (ODI), EQ-5D and a visual analogue scale (VAS). Patients were emailed by the BSR on three occasions for each of the required questionnaire time points. They had up to day 70 and day 270 postoperatively to respond to the 6 week and 6 month questionnaires respectively before they were considered incomplete.

Reports were run via the surgeon BSR database portal from January 2016 to April 2018. Patient demographics and Oswestry Disability Index (ODI) score at baseline, six weeks, six months, one year and two years were analysed.

Response rates at each time point were recorded. Then a subset analysis was performed according to age, sex, diagnosis, prior compliance and pre-operative ODI score. Descriptive statistics were generated within Microsoft Excel for Mac (Microsoft Corporation, Redmond, Wash) and analysis performed within SPSS 23 for Mac (IBM, Armonk, New York). The comparative odds ratios were calculated and a p-value of less than 0.05 was considered statistically significant.

Results

There were 297 patients included in this analysis. Of these, 132 had a minimum of a 2 year follow-up, 122 had a minimum of a 1 year follow-up and 43 had at least a 6 months follow-up after surgery. The baseline questionnaire was completed by 58% of all patients. This reduced to 33% at 6 weeks, 25% at both 6 months and 1 year, and 28% at 2 years (Figure 1).

The lumbar degenerative cohort were more likely to respond at the 6 week follow-up and the deformity cohort more likely at the 2 year follow-up OR 1.67 [95%CI 1.02-2.72 p=0.04] and OR 6.52 [95%CI 2.83-15.00 p<0.0001] respectively. Otherwise there was no significant difference between the degenerative and deformity cohorts at the other aforementioned time points (Figure 1).

Those patients who responded at 6 weeks were more likely to respond to subsequent questionnaires [OR 29.9 (95% CI = 9.3-95.7 p< 0.0001)]. Females were more likely than males to respond [OR 2.0 (95% CI 1.2-3.4 p=0.012)]. Similarly, the millennial generation were more likely than older generations to respond [6 week follow-up OR 2.37 (95% CI 1.36-4.12 p=0.0023)] (Table 1).

Table 1: Predictors of patient response at 6 week follow-up.

At the time of this study, patients recruited in 2016 were more likely to respond than those recruited more recently, at their 6 week follow-up. OR: 2.21 [95% CI 1.00-4.87 p=0.0482] (Figure 2). A histogram of baseline Oswestry Disability Index (ODI) scores for all patients within the lumbar degenerative treatment group was generated (Figure 3). This reveals a normal bell-shaped curve distribution of baseline ODI scores (Figure 4A-D).

Discussion

The need for surgical registries has been widely discussed and is generally accepted as valuable resources. Within the UK the BSR is fast becoming an integral part of recording spinal surgical activity and its outcomes. The process has enabled the collection of a large volume of prospective surgical outcome data that is hoped to improve spinal surgery services nationally.

We have shown, however, that the patient reported outcomes scores are incomplete, which is confirmed by the 2018 BSR report [5]. Thus, there is concern that the data obtained by the BSR is not fully representative. The implications of this are far reaching, including inaccurate visualisation on surgeon specific outcomes, modification on funding strategies, insurance company misinterpretation and the appropriateness of national spinal directives. For that reason, we wanted to determine which factors contribute to patient compliance rates in order to more accurately understand who is being represented by the BSR.

We identified a poor initial compliance despite allocated time within the clinic environment to complete questionnaires. Subsequently, we identified individual response fatigue, with a reduction in response at the 6 week and 6 month time points. After this, we identified a plateau of around a quarter of patients responding to the questionnaires.

In a system based solely on email, some degree of familiarity with information technology is mandatory and therefore it comes as little surprise that the younger patients in our cohort were more likely to respond. We also found that female patients were more likely to respond. Although unclear, it may be that women feel more comfortable and inclined to record their outcomes, in a similar way to their being more likely to seek medical advice [6].

Experiences at the extremes are known to be more likely to trigger a response and questionnaires delivered in a non-face-to- face manner are likely to deliver more extreme response choices [7]. We have identified a similar pattern in our results, with those with extreme ODI scores being more likely to respond over the long term. With the majority of patients not responding, it remains impossible to understand the majority’s functional state. Unfortunately no satisfactory solution exists to compensate for this missing data in an analysis of the registry results [8].

The BSR therefore remains limited by its poor patient response rate and an inability to interpret a non-response to email followup. Thus, this large cohort of patients requires further investigation. Perhaps it could be interpreted in the same way as a ‘did not attend’ in the clinical setting, which is commonly construed as if the patient is better and feels they do not require a review. Alternatively, it could be that the patient has sought alternative treatment and feels no obligation to complete questionnaires for a treatment that failed to help their symptoms.

Van Hoof performed a systematic review of 25 spine registries, representing 14 countries. They recommended that to reduce bias, a 60-80% follow-up response is required at 12 months. To achieve this they felt that more reminders should be sent [9]. However, deleting a repercussion free reminder remains a relatively easy thing to do. In New Zealand, up to 75% response rate is achieved by the New Zealand Joint Registry (NZJR), which uses email, postage and direct telephone calls to attain their follow-up data [10]. While there is clearly an expense to such an endeavour, it ensures that the data obtained is representative and accurate. This approach also ensures that patients with limited email access or technical capability are not excluded from analysis.

Alternative options may be access to follow-up clinic appointments only being granted when the feedback is completed, or commercial NHS discounts afforded to employees being extended to patients who respond. [11] While these options work internationally, they may not be suitable for the NHS as a state funded health care system.

Alternatively, it should be entirely the responsibility of the surgeon to improve compliance with feedback. In our centre, despite allocated time and computer tablets provided for patients in the clinic our response rate remains below the proposed 60-80% response rate required to reduce bias [9]. Furthermore, an international survey of spine surgeons concluded that the main barriers to implementing PROMs were a lack of time to administer the questionnaires (57%) followed by a lack of staff to assist in data collection (55%) and the length of time to fill out the questionnaires (46%) [12]. Thus, it may be prudent to assess the questionnaires themselves as barriers to compliance.

Regularly completing the EQ5D, VAS and ODI scores may be far too long-winded and off-putting for some patients. Yet, some feedback may be better than none, particularly from our silent majority. Therefore, an abridged set of questions might be more realistic and encourage greater patient engagement [13]. Analysts can still gain something from an overall holistic patient rating of ‘I am better following my surgery’ than knowing exactly how long a patient can sit before they become uncomfortable.

The future may lie in a mixed model design of questionnaire. This could involve email and/or text message with a link to a website or phone call. One study, looking specifically at response rates to text message surveys, found that text, text with a link to a website or direct phone call to respondent had response rates of 12%, 11% and 41% respectively [14]. Furthermore, Saleh summarised that surveys were completed more fully by the recipient if they knew how long it would take to fill out beforehand, if they were assured that the answers would remain confidential, if the questions were short and concise, if questionnaires were received at the beginning of the day and from an organisation they immediately recognised [15]. Thus, the BSR should look to address these factors if compliance is to be improved.

Our study is clearly limited by the fact that it only assesses a single surgeon’s data. While, the compliance rates in this study are comparable to the national average and may represent a national picture of who responds. Further analysis should be carried out across a greater number of surgeons to validate our conclusions [5].

Conclusion

In conclusion, patient response rates to the BSR are low and declining. Those that respond are more likely to be females, millennials and those with either very high or very low ODI scores. These results should stimulate discussion around improving patient engagement and instil caution when using these registry outcomes to guide future practice and funding distribution.

Acknowledgments

Glynny Kieser for her editorial input.

References

1. Delaunay C (2015) Registries in orthopaedics. Orthop Traumatol Surg Res 101: S69–75.
2. Drolet BC, Johnson KB (2008) Categorizing the world of registries. J Biomed Inform 41: 1009–1020.
3. Melloh M, Staub L, Aghayev E, Zweig T, Barz T, et al. (2008) The international spine registry SPINE TANGO: status quo and first results. Eur Spine J 17:1201–1209.
4. Breakwell LM (2013) Understanding the need for spinal registries: Lee Breakwell reviews the importance of registries in spinal research and explains why the British Association of Spinal Surgeons (BASS) has decided to set up its own registry. Eur Spine J 1: S5–6.
5. The British Spine Registry End of Year Annual Report (2016 -2017) 1–19.
6. Wang Y, Hunt K, Nazareth I (2013) Do men consult less than women? An analysis of routinely collected UK general practice data. BMJ Open 3: e003320.
7. Bowling A (2005) Mode of questionnaire administration can have serious effects on data quality. J Public Health (Oxf) 27: 281–291.
8. Altman DG, Bland JM (2007) Missing data. BMJ 334:424–424.
9. van Hooff ML, Jacobs WCH, Willems PC, Wouters M, de Kleuver M, et al. (2015) Evidence and practice in spine registries. Acta Orthopaedica 86: 534–544.
10. Hobbs T, Frampton C, Devane P (2018) The New Zealand Joint Registry. Ninteen Year Report.
11. Becker R, Glauser D (2018) Are Prepaid Monetary Incentives Sufficient for Reducing Panel Attrition and Optimizing the Response Rate? An Experiment in the Context of a Multi-Wave Panel with a Sequential Mixed-Mode Design: Bulletin of Sociological Methodology/Bulletin de Méthodologie Sociologique 139: 74–95.
12. Falavigna A, Dozza DC, Teles AR, Wong CC, Barbagallo G, et al. (2017) Current Status of Worldwide Use of Patient-Reported Outcome Measures (PROMs) in Spine Care. World Neurosurg 108: 328–335.
13. McCormick JD, Werner BC, Shimer AL (2016) Patient-reported Outcome Measures in Spine Surgery. Spine (Phila Pa 1976) 41:429-37.
14. Marlar J (2017) Using Text Messaging to Reach Survey Respondents. Gallup Methodology.
15. Saleh A, Bista K (2017) Examining factors impacting online survey response rates in educational research: Perceptions of graduate students. JMDE 13: 63-74.