Improving the robustness of health technology funding decisions when evidence is from single-arm trials

Background Health technology assessment (HTA) is a multidisciplinary process to evaluate the clinical, economic and broader impact of the use of a health technology. Evidence on clinical effectiveness can arise from multiple sources.

Randomised controlled trials (RCTs) are considered to be the gold standard for evaluating an intervention because potential biases are minimised; other study designs may distort results.

However, RCTs may be infeasible in some disease areas, e.g., if it is unethical to allocate patients to a comparator arm, and single-arm trials have to be used instead. Estimates of treatment effects are at risk of bias if the data are from single-arm trials. Existing statistical methods can only correct for some of the biases.

Decision-makers such as the National Institute for Health and Care Excellence (NICE) have had to make decisions in cases when not all biases have been addressed.

Consequences of an incorrect funding decision could be extreme: patients may be harmed because they do not receive an effective treatment, and the wider society loses out because of an inefficient use of scarce healthcare budgets.

Aims and methods This fellowship aims to develop new methods to improve the robustness of healthcare funding decisions when evidence is from single-arm trials.

We will address two important issues: (i) bias due to difference between patients in different trials ('internal' bias); (ii) bias due to generalisability of the study results to clinical practice ('external' bias).

There are four related work packages: WP1 (months 1-17): develop a new analytical approach to adjust for imbalance in observed confounders; evaluate and compare the proposed approach to existing methods through simulation studies and real RCT data (objective 1 and 2) WP2 (months 18-31): develop bias adjustment models to adjust for residual biases due to the remaining internal and external bias; and develop methods for the formal elicitation of experts' opinion regarding potential residual biases (objective 3) WP3 (months 32-45): conduct three case studies using real-world examples to pilot the proposed methods; and determine the use value of information analysis for quantifying the impact of potential bias on decision uncertainty (objective 3 and 4) WP4 (months 46-48): formulate best practice guidance regarding the use of single-arm studies in estimating treatment effect in HTA (objective 5) Anticipated impact and dissemination The findings of this fellowship will benefit NICE by addressing gaps in the existing methods and providing a basis for more robust decision-making.

Making better decisions will improve outcomes for patients and the public. Health gains would be maximised in terms of the extent and quality of life of patients, given the available resources.

Healthcare resources would be distributed to patients and the public more effectively and fairly in the National Health Service (NHS).

It would also allow manufacturers to prepare more thorough submissions, which will improve the efficiency of the appraisal process.

The research outputs will be disseminated via publications in relevant open-access peer reviewed scientific journals, presentations at relevant UK and international conferences, an Impact Event with stakeholders, project webpage and social media channels.