Most readers should be aware that the Charity Commission has issued a consultation on the registration of complementary and alternative medicine (CAM) organisations as charities. This was after a five-year battle documented in other posts on this blog. In the interests of transparency I am posting my response here.
Question 1. What level and nature of evidence should the Commission require to establish the beneficial impact of CAM therapies?
The Commission should use exactly the same criteria as are used by the health care professions and other regulators. For example the Advertising Standards Authority is not a health care or scientific organisation, but requires evidence from randomised controlled trials (RCTs) when advertisers make claims that they can benefit health. The General Medical Council requires doctors to follow evidence based practice, and expects a similar level of robust evidence. No doubt CAM practitioners will respond to this consultation with large volumes of anecdotal data, but anecdotes are not robust evidence. A favourite is the paper by Spence et al on homeopathy. This was not a trial at all, but an unstructured satisfaction survey. There was no control group, and the journal was not a properly peer reviewed one.
It is important to differentiate between explanatory RCTs and pragmatic RCTs. As explained at Wikipedia:
Explanatory RCTs test efficacy in a research setting with highly selected participants and under highly controlled conditions. In contrast, pragmatic RCTs test effectiveness in everyday practice with relatively unselected participants and under flexible conditions; in this way, pragmatic RCTs can “inform decisions about practice”.
Many medical journals now require authors to state how their results might be generalised to the population of patients that present in routine clinical practice. Pragmatic RCTs attempt to address this question. But they are less useful for testing whether a treatment is intrinsically effective. That is the domain of the explanatory RCT. CAM researchers are often keen to skip the explanatory RCT stage and go straight to the pragmatic trial. This is dangerous. It is vital to know whether there is actually an intrinsic effect of the treatment, before asking whether such an effect has benefit for the general population.
Sitting above the level of even the RCT are systematic reviews and meta-analyses, whereby several trials are analysed together. This can strengthen the conclusions, but a very big problem, especially with CAM trials, is quality. In many systematic reviews and meta-analyses a majority of RCTs have to be excluded because of poor methodological quality. CAM proponents often ignore study quality, and it is rather telling that the effect size in homeopathy trials is inversely proportional to their quality. In other words, the worse the trial the better homeopathy looks.
It is not necessary for the Commission to become experts at clinical evidence. It could simply ask for robust evidence as set out above, as the ASA does. They engage outside experts where there is doubt.
Question 2: Can the benefit of the use or promotion of CAM therapies be established by general acceptance or recognition, without the need for further evidence of beneficial impact? If so, what level of recognition, and by whom, should the Commission consider as evidence?
Science is not a popularity contest, so `general acceptance’ cannot be contemplated. The idea of `recognition’ presupposes that some sort of respected body has already assessed the evidence. Only organisations which have done that rigorously could be considered. CAM professional bodies should never be asked. Their function is primarily to support their members. Not one of them asks any member to provide evidence for the health benefits they claim. Not even the statutorily regulated `professions’, chiropractic and osteopathy, do this. A good independent authority is NHS Choices, and there are good academic websites such as http://www.bandolier.org.uk/. The National Institute for Health and Care Excellence (NICE) provides some guidance but it is not comprehensive, because it depends on which treatments have been referred to it for appraisal.
The Commission’s guidance OG304 relies on the 2000 House of Lords report on CAM. This is a mistake as the enquiry specifically excluded evidence of efficacy from its terms of reference. There are more recent and authoritative reports, such as that of the Australian National Health and Medical Research Council in 2015, on homeopathy. There are some useful compendia, such as The Desktop Guide to CAM by Ernst, Pittler and Wider. However evidence continually develops so such works are not always the latest word.
But overall, the Commission could do no better than to emulate the approach of the ASA, which has for example adjudicated that no claims of efficacy can be made for homeopathy. The ASA always asks the advertiser for evidence to support their claims, whatever they may be. If applicants fail this test, it is not up to the Commission to investigate further, but simply to refuse registration.
Question 3: How should the Commission consider conflicting or inconsistent evidence of beneficial impact regarding CAM therapies?
There are no certainties in science. The consensus changes over time, as more and better evidence emerges. If no evidence has yet emerged to justify a practice, there might be two reasons for this. Either RTCs have been carried out, with no positive results, or no RCTs were done. In either case, there is no justification for claiming benefit. Later, better trials might show clear benefit, so the claims become valid. To protect the public the rule should be to reject claims of benefit that are doubtful because of conflicting evidence. Commonly a systematic review or meta-analysis can resolve such conflicts. Certainly, benefit cannot be assumed until evidence is available, and `cherry-picking’ positive trials is dangerous. The very existence of inconsistency in the evidence indicates that it is not robust. Homeopaths like to claim that there are many `positive trials’ in their favour, but they ignore the matter of trial quality. Among good quality trials, there is no controversy about homeopathy – it does not work. The Commission should beware of manufactured controversies.
Question 4: How, if at all, should the Commission’s approach be different in respect of CAM organisations which only use or promote therapies which are complementary, rather than alternative, to conventional treatments?
There should be no difference. The boundary between the two is extremely vague, and there is the danger of `bait and switch’. What a charity says in its application to the Commission may be different from what they tell vulnerable people with serious health problems. In any case, whatever is claimed must be justified with evidence, whether it is complementary or alternative.
Question 5: Is it appropriate to require a lesser degree of evidence of beneficial impact for CAM therapies which are claimed to relieve symptoms rather than to cure or diagnose conditions?
No. Symptoms may indicate a serious underlying condition. CAM practitioners however usually claim to treat the `root cause’ of illness, based on completely imaginary concepts of how the body works. These claims are the opposite of reality. Homeopaths base remedies on what they call the `symptom picture’, ie their knowledge is confined to symptoms and they know nothing about what is causing them. So what CAM practitioners say they do cannot be relied upon. Also, as described in 4 above, there is the `bait and switch’ problem, whereby a charity may claim to be only treating symptoms when they are actually seeing patients with serious conditions.
Question 6: Do you have any other comments about the Commission’s approach to registering CAM organisations as charities?
I appreciate that the Commission has a legacy problem. Many CAM charities were registered decades ago before evidence based health care really existed. It might seem harsh to tell these charities that they are to be removed from the register, but they are not being told to stop operating. They are simply being denied the tax benefits of a charity. They will of course have to comply with advertising and other regulation, but charitable status is seen as an imprimatur of validity and tends to reinforce their public credibility. This is undeserved.The first step should be to require robust evidence for any health claims made by organisations seeking charitable status. This would in effect impose a moratorium on new CAM charities. Existing CAM charities should be given notice that, in the absence of robust evidence for public benefit, they will be removed from the register at a specified date. Identifying CAM charities might be seen as a lot of work, so the public should be encouraged to report them to the Commission. Up to now the Commission has rejected all such reports, so this would be an important policy change.
The Commission stresses the point that a charitable purpose does not in itself imply a public benefit. The complementary or alternative health field is so called essentially because it lacks scientific validation. If robust evidence does emerge, the practice in question then ceases to be complementary or alternative and becomes orthodox treatment. Hence no CAM charity can pass the public benefit test. Stating that a charity’s aspiration or objective is to improve health falls within the scope of the charitable purpose, but not within that of a public benefit. A wish is not an outcome.
No doubt the CAM supporters will bombard the Commission with huge volumes of anecdotal `evidence’ and logical fallacies. A favourite is to talk up the many shortcomings of clinical trials on real medicines, but missing the point that efforts such as Retraction Watch are actually operated by scientists not CAM supporters. It is science that regulates itself, not the puffery of conspiracy theorists such as `What Doctors Don’t Tell You‘.
I hope that heavyweights like the medical royal colleges will submit responses. Have you?
