Future directions of research into harms in randomised controlled trials

The Consort group includes many initiatives to improve reporting of results from randomised controlled trials (RCTs). The original Consort statement in 1996 was developed to provide a standardised way for authors to report their trial findings.1 Since then there have been two updates to the statement as well as many extensions including the 2004 Consort harms.2 The harms extension resulted, however, in limited improvements in? practice.3

The latest update aims to tackle this lack of impact by integrating the harm reporting guidelines into the main Consort checklist.4 This reflects the need for balance in reporting both harm and benefit. While this guidance can support improved reporting, as trialists, one of the major challenges we face is how to analyse and summarise complex harm data. We must now turn our energies to this unmet need which goes beyond Consort’s remit. We must improve the analysis of adverse events better to identify true signals of harm and promote more effective communication of this information. Only when we pair optimal analysis with good reporting will the full potential of harm data be realised.

To date, analysing adverse event data has been widely neglected and good analysis practice is often disregarded.5 There is over-reliance on simplistic approaches that do not make full use of the prospective, high quality data collected in RCTs. There is a growing understanding within the clinical trials community that how we analyse harm outcomes in trials needs to change.6 Efforts are being made, but these are overwhelmingly focused on analysis of prespecified harm outcomes.7 More attention is needed on the analysis of emerging harms, that are not defined at the outset but are obtained from reports throughout the study.

This poor practice prevails despite 2017 guidance for statistical analysis plans.8 Trialists need support to plan and apply suitable analysis methods. We need to give more in-depth consideration to harm outcomes at the design stage and develop analysis plans for prespecified and emerging events, as one would for primary and secondary outcomes. This should also include consideration and specification of the estimates of interest and how each analysis will be used to draw inference.9

Principled analysis approaches that are accepted good practice when analysing efficacy outcomes should be used when analysing harm outcomes. For example, by using information on recurrent events rather than presenting as those who experienced “at least one event” and reducing information loss by retaining continuous outcomes instead of dichotomising and analysing as binary outcomes. We should consider routinely adopting improvements such as reporting incident rate ratios with a measure of uncertainty to account for differences in exposure times, using linear mixed effects models to incorporate repeated continuous biological and clinical results, and plotting the mean cumulative function to account for recurrent events to explore the burden of harm over time. Underpowered hypothesis tests and using P values and measures of precision as proxy tests of null hypotheses should be avoided.5 Further research and consensus are needed around ideas such as reframing the research question around harm to one of detecting signals for investigation, using Bayesian methods to incorporate prior information and applying the estimand framework to harms.10 The estimand framework was developed to support trialists to precisely define the treatment effect they are aiming to estimate.1112 Work to date has predominantly focused on efficacy outcomes.

In addition to Consort reporting guidance, we need clear advice on best analysis practice for harm outcomes that is endorsed by key stakeholders and can be easily implemented. Increasing good practice around evaluation of harm outcomes with a more coherent and consistent approach would improve efficiency and synthesis of harm data across trials. Better reporting encouraged by the new Consort harm extension along with guidance for analysis would enable us to better assess the true benefit-risk of interventions.