DO PAPER SURVEY RESPONSES AGREE WITH NON-PAPER ALTERNATIVE SURVEY RESPONSES FOR PATIENT-REPORTED OUTCOMES IN PATIENTS WITH CHRONIC LOW BACK PAIN?

A. Banerjee^1,2,3, P. Hendrick², H. Blake^2,4

Background: Mixed-mode survey design allows flexible survey completion using paper-and-pen, or non-paper alternatives (NPA- online or over the telephone) and has a higher response rate than a single-mode survey. With the rapid growth in internet accessibility, remote consultation, use of online patient-reported outcomes (PROs) within mixed-mode surveys are trending among patients, clinicians and researchers. Studies investigating equivalence in the mixed-mode survey design use product-moment correlation (Cronbach’s α, kappa or intraclass correlation coefficients-ICC) that examine the strength of association between two modes but Bland and Altman proposed Limits of Agreement (LoAs) that have not been adequately examined in mixed-mode survey equivalence studies in patients with chronic low back pain (CLBP).

Purpose: The purpose of this study was to determine the reliability and agreement between paper and non-paper alternatives (NPA) either online or over the telephone survey modes within a mixed-mode survey in patients with CLBP.

Methods: This prospective test-retest study was part of a longitudinal cohort study involving working age community ambulant adults (18-65 years), who had attended outpatient physiotherapy for their CLBP (pain duration ≥ three months). The study protocol was approved by a UK National Health Service Ethics Committee (14/ES/0167). Thirty-nine participants completed paper-based questionnaire measures of pain intensity (Numeric Pain Rating Scale), physical disability (Roland Morris Disability Questionnaire) and self-management (Health Education Impact Questionnaire). Participants completed the same measures within two weeks in NPA either online or over the telephone according to patient preference. Reliability (interclass correlation with random effect model) and Bland and Altman LoAs between the paper and NPA survey modes were analysed.

Results: Five patients who had completed the online survey after 14 days from baseline were excluded from the analysis. The remaining 34 (25 online and nine over the telephone) participants [mean age 41.8 (standard deviation 13.1) years and 62% female] were included in the analysis. The reliability of the paper and NPA survey modes was high (Cronbach’s alpha 0.89-0.95 and interclass correlation 0.89-0.96) for all variables. There was no statistically significant or clinically meaningful difference in mean scores between the paper and NPA survey modes. The LoAs [mean difference (upper to lower limit)] between the paper and NPA survey modes were -0.12 (2.96 to -3.20) for pain and -0.21 (4.56 to -4.97) for disability. The LoAs of the SM constructs ranged from -0.04 (0.75 to -0.82) [in Positive and Active Engagement in Life] to 0.03 (0.60 to -0.54) in [Social Integration and Support]. One of the self-management constructs (Health Directed Activity) scores showed a proportional bias between paper and NPA survey responses, which was attributed to outliers after sensitivity analysis. The telephone survey was not separately compared to the paper survey due to small sample size (n=9).

Conclusion(s): The results indicate good agreement and reliability between paper and NPA survey administration of PROs in patients with CLBP. Future research should examine any effect of the survey mode, survey order (and their interactions) on the LoAs.

Implications: Data quality in PROs is comparable between paper survey and non-paper alternative survey for data collection including online and telephone surveys.

Funding, acknowledgements: The University of Nottingham, UK funded this doctoral research study.

Keywords: chronic low back pain, mixed-mode survey, limits of agreement

Topic: Musculoskeletal: spine

Did this work require ethics approval? Yes

Institution: National Health Service/ Health Research Authority Ethics Committee

Committee: East of Scotland Research Ethics Service

Ethics number: 15/ES/1067