Abstract
BACKGROUND: Many researchers use observed questionnaire scores to evaluate score reliability and to make conclusions and inferences regarding quality-of-life outcomes. The amount of false alarms from medical diagnoses that would be avoided if observed scores were substituted with expected scores is interesting, and understanding these differences is important for the care of cancer patients. Using expected scores to estimate the reliability of 95% confidence intervals (CIs) is rarely reported in published papers. We investigated the reliability of patient responses to a quality-of-life questionnaire and made recommendations for future studies of the quality of life of patients. METHODS: A total of 115 patients completed the EORTC core questionnaire QLQ-C30 (version 3) after radiotherapy. The observed response scores, assumed to be one-dimensional, were summed and transformed into expected scores using the Rasch rating scale model with WINSTEPS software. A series of simulations was performed using a unified bootstrap procedure after manipulating scenarios with different questionnaire lengths and patient numbers to estimate the reliability at 95% confidence intervals. Skewness analyses of the 95% CIs were compared to detect different effects between groups according to the two data sets of observed and expected response scores. RESULTS: We found that (1) it is necessary to report CIs for reliability and skewness coefficients in papers; (2) data derived from expected response scores are preferable to making inferences; and (3) visual representations displaying the 95% CIs of skewness values applied to item-by-item analyses can provide a useful interpretation of quality-of-life outcomes. CONCLUSION: Reliability coefficients can be reported with 95% CIs by statistical software to evaluate the internal consistency of respondent scores on questionnaire items. The SPSS syntax procedures for estimating the reliability of the 95% CI, expected score generation and visual skewness analyses are demonstrated in this study. We recommend that effect sizes such as a 95% CI be reported along with p values reporting significant differences in quality-of-life studies.