Validity of population adjustment methods for disconnected networks of evidence

Network meta-analysis (NMA) is a method to pool published summary treatment effects from randomised controlled trials (RCTs) to obtain estimates of relative treatment effects between multiple treatments.

NMA is routinely used to inform decisions as to which treatments are effective or cost-effective, but requires that the RCT evidence forms a connected network of comparisons (the map of comparisons made in RCTs is a connected network).

Covariates such as age, biomarker status, or disease severity can be classified into (i) those that interact with relative treatment effects (Effect Modifiers), and (ii) those that predict outcomes but don't interact with treatment effects (Prognostic Factors).

NMA assumes that, if effect modifiers are present, their distribution is the same, or similar, in all the included trials. However, this may not hold.

Recently a multi-level network meta-regression (ML-NMR) method has been developed that relaxes this assumption, as long as individual patient data is available from one or more RCTs.

ML-NMR fits a model for individual-level treatment effects in studies where there is individual patient data, and then for each of the studies with aggregate-level data integrates the individual-level likelihood over the joint distribution of effect modifiers in each study to obtain an aggregate-level likelihood.

This is achieved using copulae to approximate the joint distribution of effect modifiers and quasi-Monte Carlo integration to obtain the aggregate-level likelihood.

Estimates can then be obtained in any population of interest (for example, the population represented by one of the included studies, or the UK population) by integrating over the relevant joint distribution of effect modifiers.

To date, the method has only been developed for the case where networks of evidence are connected (there is a path of RCT evidence joining any two treatments in the network).

However, it is becoming more common that health care policy makers are confronted with disconnected networks of evidence which may include single-arm (non-randomised) studies.

Population adjustment with disconnected networks of evidence requires that not only the effect modifying covariates are accounted for, but also all prognostic factors since absolute outcomes rather than relative effects are modelled.

The aim of this project is to extend the ML-NMR method for disconnected networks of evidence for a range of likelihoods, including likelihoods for survival outcomes, and to explore methods to assess the validity of the ML-NMR method in the context of disconnected networks of evidence.

This will include: (i) formulating extensions to the ML-NMR model for modelling absolute outcomes alongside relative effects, without introducing bias into the estimated relative treatment effects (ii) assessing the performance and properties of such approaches in a simulation study (iii) developing in-sample methods for assessing the validity of assumptions, such as cross-validation to estimate the proportion of variation explained by the model so that any unexplained variation will be largely due to missing prognostic variables or effect modifiers that have not been accounted for; and (iv) developing out-of-sample methods that aim to estimate prediction error by identifying external studies in a given population and comparing the absolute outcomes predicted by ML-NMR in this external population with the observed outcomes.