Abstract
When people who advocate integrating conventional science-based medicine with complementary and alternative medicine (CAM) are confronted with the lack of evidence to support CAM they counter by calling for more research, diverting attention to the 'package of care' and its non-specific effects, and recommending unblinded 'pragmatic trials'. We explain why these responses cannot close the evidence gap, and focus on the risk of biased results from open (unblinded) pragmatic trials. These are clinical trials which compare a treatment with 'usual care' or no additional care. Their risk of bias has been overlooked because the components of outcome measurements have not been taken into account. The components of an outcome measure are the specific effect of the intervention and non-specific effects such as true placebo effects, cognitive measurement biases, and other effects (which tend to cancel out when similar groups are compared). Negative true placebo effects ('frustrebo effects') in the comparison group, and cognitive measurement biases in the comparison group and the experimental group make the non-specific effect look like a benefit for the intervention group. However, the clinical importance of these effects is often dismissed or ignored without justification. The bottom line is that, for results from open pragmatic trials to be trusted, research is required to measure the clinical importance of true placebo effects, cognitive bias effects, and specific effects of treatments.