Statin trials
How robust is the evidence?
It probably started when medical authorities put the blame for heart disease (atherosclerosis) on circulating cholesterol. In the modern era, this goes back to the 1950s or before, when heart disease was still relatively rare, but increasing. Heart disease became publicly prominent in 1955, when US President Eisenhower suffered a heart attack (which he survived). The pressure was then on to find a cause for heart disease and to prevent it. Funding increased. Notably, Eisenhower had been a chain smoker nearly all his adult life (he was a 4 pack-a-day man), but that was ignored. The cholesterol hypothesis consolidated.
A number of drugs were available that could modestly lower cholesterol. These included high-dose niacin (vitamin B3), fibrates and bile acid sequestrates. These act by different mechanisms, but mostly they clear cholesterol that is already in the circulation either by drawing it into cells (whether cells want it or not), or eliminating it in faeces.
However, such was the focus on cholesterol that medical authorities wanted better drugs. Drug companies saw a market waiting to be had.
Early statin development
Industry turned its attention to blocking the naturally-occurring cholesterol synthesis pathway in cells, particularly the liver. The first such drug (triparanol) was approved in 1959, but was withdrawn in the 1960s due to emerging side effects. Triparanol targeted the last steps in the cholesterol pathway, and may have led to a damaging accumulation of intermediate molecules upstream, because the cell would detect a shortage of cholesterol and up-regulate the pathway.
The development of the first drugs acting at the start of the cholesterol synthesis pathway began in 1968 in Tokyo. Akira Endo was employed by Sankyo Research Laboratories, and given the freedom to work on a project of his choosing. He knew that antibiotics inhibited many different kinds of enzymes, and set about looking for an antibiotic that could inhibit the enzyme HMG-CoA reductase, the key regulator for cholesterol synthesis in the first steps of the pathway. He speculated that the place to start looking was fungi, on the basis that some fungi may target this enzyme to attack other microorganisms or for their own defense.
He worked heroically through 3,800 strains of fungi before finding a potential candidate that produced a chemical called citrinin. It was shown to be cholesterol-lowering in rats, but the research had to be suspended because of kidney toxicity.
He didn’t give up. In 1972 he found another active enzyme (compactin) in a culture broth of blue-green mold that was “isolated from a rice sample collected at a grain shop in Kyoto”. It was effective in inhibiting HMG-CoA reductase.
This was shown to work in cell cultures, then rats, hens, dogs and monkeys. There were many hurdles to overcome along the way. The first human study in a patient with familial hypercholesterolemia (a genetic condition causing elevated cholesterol) was trialled in 1978, and Phase II trials confirmed safety and effectiveness. However, in 1980, Sankyo discontinued the clinical development of compactin, over concerns about lymphomas in dogs. Later, this was shown to be an over-dosage problem, but Sankyo never revived compactin development.
Spare a thought for Akira Endo.
Nevertheless, compactin proved to be the catalyst for the development of a swathe of these drugs (statins), the first of significance being lovastatin and the modern statin era was underway: simvastatin (1988), pravastatin (1991), fluvastatin (1994), atorvastatin (1997), cerivastatin (1998), and rosuvastatin (2003). Most statins are now produced synthetically, rather than from fermentation byproducts.
Therefore, from the early 1990s and thereafter, medical authorities had their new, cholesterol-lowering drugs. Industry had done its job, and now it wanted its reward.
The tangled web
This put the pressure on medical scientists and authorities to use the statins and to show that the cholesterol hypothesis was correct and that these drugs ‘saved lives’. Remember that the cholesterol hypothesis had still not been proven, it remained an hypothesis. However, scientists were locked in — the cholesterol hypothesis now had to be right, there was too much money and industry pressure to allow anyone to back down. The drug companies started advertising campaigns around cholesterol, further locking in the scientists. The ‘Know Your Numbers’ campaign by Pfizer urged people to get their cholesterol measured, creating an anxiety that could only be relieved by taking a statin. Industry knew their drugs lowered cholesterol, now they wanted studies to show a clinical benefit.
Statin Trials
Most randomised controlled trials (RCTs) of statins have been drug-industry funded. RCTs are costly and time-consuming, therefore government agencies are willing to privatise this research, accept the outcomes and develop health policy on that basis.
The RCT design employed in any statin study tests whether statins reduce heart disease, which the drug company considers the primary purpose of the trial — the trial is about statins and clinical outcome. Note therefore, that these trials are not designed to test whether cholesterol causes heart disease.
The trials follow a common template that can be summarised like this: Recruit two large groups of people, one of which goes on a statin and the other on a placebo; decide what constitutes heart disease for the purpose of the study — for example, is it death from a heart attack (a hard endpoint), or is it a soft endpoint like angina, revascularisation (stenting) etc; decide on a followup period, usually 5 years; stop the study early when you have the result you want; inflate the effect size by using relative risk; publicise widely.
I have explained these steps in more detail in the Lipitor story.
If the study does not show a clinical effect, do another one until it does. For US Food and Drug Administration (FDA) approval of a new drug (or target population), the drug only has to be shown to be effective in two RCTs. It does not matter how many RCTs have to be carried out to get two that ‘work’ — the FDA has no rules about that. The trials that didn’t work are not likely to be publicised, or even published.
Statin RCT outcomes
This post confines itself to statin trials for primary prevention (preventing a first heart attack in otherwise healthy people). This is the most contentious application for statins and requires the greatest scrutiny.
The perception has been created that cholesterol-lowering trials strongly reduce the risk of heart disease. On the contrary, not only are their effect sizes small in real terms, and inflated for publicity purposes, they typically only show an effect on soft endpoints. They don’t improve mortality. On average, people don’t live longer on statins.
In fact, this applies to all means for lowering cholesterol. A recent 2017 paper listed 44 cholesterol-lowering RCTs (about a third were statin trials), all of which showed no effect on mortality despite successfully lowering cholesterol (in some studies substantially). Furthermore, less than a third of these studies could show a beneficial effect on soft endpoints. Six of the studies reported harm.
One wonders how many RCTs have to fail before the cholesterol hypothesis gets seriously questioned.
Pooling the analysis of RCTs
The secret of statin RCTs is how small the effect size really is, and how difficult it is even to see an effect. Because of this, scientists resort to pooling the results from multiple trials in what is referred to as a meta-analysis. It is hoped that the increase in participant numbers coming from pooling will reveal a statistically-significant effect. Meta-analyses are also used to reach a general conclusion when individual trials might report inconsistent findings.
Some such meta-analyses have been undertaken, however they have multiple confounds, not the least being which RCTs are included in the analysis to achieve significance (selectivity in RCT inclusion is a bias). Furthermore, they cannot re-analyse the actual RCT data, they can only use the results of the RCT data as analysed and published by the trial researchers. There are also issues of variable endpoints and methodological differences. Nevertheless, the meta-analyses that show statistical significance still report absolute effect sizes of well less than 1% (that is, it would be necessary to treat more than 100 people over whatever the duration of the studies were, for one to benefit).
A somewhat improved approach would be to get access to the RCT data in its un-analysed form from the researchers who undertook the studies, pool it across trials and then analyse the pooled data using a uniform method. Under normal circumstances, researchers would not hand over their data to a third party. However, in the case of statin RCTs, many have indeed done so.
There’s a catch though…
The CTT Collaboration
The Cholesterol Treatment Trialists’ (CTT) collaboration (setup in 1995 and ongoing) is a collaboration between principle investigators on RCTs that enabled them to share de-identified raw data, so that it could be centrally analysed by uniform methods.
To quote from their web page: “It was set up after it was recognized that no single lipid intervention trial would be likely to have a sufficient number of trial participants (and hence statistical power) to reliably assess mortality outcomes or look at events in particular types of patient.”
That’s my italic — that there was even a need for this type of analysis points to the weakness of the statin effect.
The Collaboration is jointly funded by the Clinical Trial Service Unit & Epidemiological Studies Unit (CTSU) in Oxford, UK and the NHMRC Clinical Trials Centre (CTC) in Sydney, Australia. It does not directly accept drug company sponsorship, nor would drug companies want to do that (see later). It is headed by Professor Rory Collins (now Sir Professor), and located at Oxford.
How did the CTT come about?
Remember that drug companies fund RCTs. They do this by funding a research group somewhere that will carry out the study under the direction of a senior scientist.
This mostly suits the drug company, because the scientists will analyse and publish the data under their institutional affiliation, and so the findings can appear independent of the company itself. This is only an illusion though — drug companies invest heavily in drugs, and in the RCTs needed to test them, so they are not likely to passively wait for whatever result emerges with their fingers crossed.
However, it was realised that it was possible to go further. While drug company sponsors of each RCT retain ownership of the data for that trial, statin manufacturers saw the benefit in releasing their data to a central location (the CTT) and allowing the CTT to perform various analyses on the pooled data. This gives the companies access to more powerful analyses that can promote the clinical significance of the statin class of drugs. It is also why they would not want to sponsor the CTT, the CTT is ultimately their independent mouthpiece. The affiliation with Oxford doesn’t do any harm either.
Central to the data sharing agreement is that the CTT data repository will never be disclosed or released to third parties. This means that CTT analyses have never been independently reviewed. Therein lies the controversy. They are a secret society of collaborators.
The reason this matters, is that medical authorities take results from CTT publications as ultimate and definitive, and CTT reviews form the basis of clinical practice guidelines and public health policy. This degree of influence, particularly over statins in primary care, has led to demands for more accountability and independent scrutiny of their data and analysis. Those demands have been firmly rebuffed by the CTT.
While drug companies retain ownership of trial data, a number of them claim to be open to sharing those data on application. This was tested recently by another meta-analysis group, the Cochrane Heart Group, that requested de-identified data from drug companies that were claiming data transparency. All declined (many didn’t reply). The excuses included that the trials had been performed before the disclosure policy had begun, that statin trials were specifically excluded in the disclosure policy or, that the data retention period had expired and the data were not available. The group also contacted government regulatory authorities in the UK, Ireland and the US that might have access to data as part of the drug approval process. Those that did have that access were not prepared to release those data.
Circa 2006
RCTs were not well regulated prior to 2006, not only for statins but for any drug. A controversy surrounding the anti-inflammatory drug Vioxx (e.g. for arthritis) changed that. It transpired that the drug company (Merck) had manipulated the data analysis and obscured the risk of secondary cardiovascular adverse events.
From 2006, the US and European regulators required all trials be registered, with a range of requirements that try to ensure openness in data interpretation, adverse effects and to minimise selective reporting — an admission that previous drug regulation procedures were not adequate. It follows that statin RCTs performed before this date are suspect.
There appear to be only two RCTs reported for primary prevention after this date, and they both tested rosuvastatin (Crestor). The Justification for the Use of Statins in Prevention: an Intervention Trial Evaluating Rosuvastatin (JUPITER) trial recruited participants on the basis of a marker of chronic inflammation (C-reactive protein, CRP) rather than low-density lipoprotein cholesterol (LDL-C). The sponsor (Merck) was looking for a new market. This trial triggered massive controversy when it was terminated prematurely at 1.9 years (of a planned 5 years). Early termination biases results. The difference between statin and placebo was statistically significant at 1.9 years, although it was showing signs of narrowing, which presumably was a motivator for early termination. The absolute effect size was a tiny 0.25% in favour of the statin, despite substantial reductions in LDL-C and CRP. The incidence of type 2 diabetes increased. Merck is the data custodian and the data has never been released. Merck employees collected the data.
Even with the new regulations, we still do not have transparency.
The Heart Outcomes Prevention Evaluation (HOPE)–3 is the second trial, and studied individuals of various ethnic backgrounds (20% white, 50% asian, 20% hispanic) who did not have cardiovascular disease and were at intermediate risk. It ran its full term (5 years). The statin reduced LDL-C by 0.9 mmol/L (from a baseline value of 3.3 mmol/L). The primary end-point was a composite of fatal cardiovascular causes, nonfatal heart attack and nonfatal stroke. There was a statistically-significant difference in this composite primary end-point in favour of the statin group, however the absolute effect size was just 1.1%. There was no significant difference between groups for all cause mortality, in keeping with other statin trials. There was no increase in diabetes in the statin group.
So, that’s all there is to the evidence justifying statins in primary prevention, in the new era of tighter RCT regulation.
The primary prevention debate
Contrary to public perception, the statin debate, particularly for primary care, is active at the highest academic levels.
In 2017, the senior editor for the BMJ (British Medical Journal) Professor Fiona Godlee, sent a letter to The Lancet (the journal of preference for the CTT). In part, it was in response to a demand from the CTT for the retraction of two papers, published in the BMJ, that were critical of the cholesterol hypothesis and statins in primary prevention. This, by the way, is outrageous — retraction is for data fraud, and not for an opinion they don’t agree with.
However, around the same time, the CTT also attempted to close down debate over statins in primary prevention by publishing an extensive (and tedious) review, of their own data, arguing that the case for statins was so compelling that there was no further case for argument.
Godlee raised these questions, that she considered inadequately answered, in her letter:
“… how strong is the evidence, how large is the benefit for individuals at lowest risk of heart disease, how well did the trials record common minor side-effects, how representative were the trials of women and the elderly, what was the effect of active run-in periods and composite endpoints, how does taking a statin affect a person’s diet and exercise patterns, why is there a discrepancy between the real-life experience of muscle pain and what was reported in the trials, why have the data for harms not yet been given the same levels of scrutiny as the data for benefits, and is cholesterol a reliable surrogate endpoint to guide prevention of cardiovascular disease?”
The last question is particularly pertinent, as it still doesn’t have an answer. Neither do the others, except perhaps the first, for which the answer is ‘feeble’. I haven’t mentioned diet, however some people believe they can eat whatever they want because they are protected by the statin. The active run-in period referred to means that before starting the RCT, the statin is given to participants to see who has an adverse effect, then these people are not included in the RCT thereby minimising adverse effects.
Concluding this topic, Professor Godlee writes:
“So despite … Collins and colleagues wanting to shut down the discussion and award themselves the final word, the debate about statins in primary prevention is alive and kicking. It is a debate that needs to be resolved as thoughtfully, objectively, and openly as possible, and not by eminence-based narrative reviews, however extensive, based on meta-analysis of data that only Collins, his fellow trialists, and industry sponsors have seen.”
Firm words, at the very highest levels of academia.
Disclaimer: I am not a medical doctor. Nothing herein is, nor should be taken to be, medical advice.
