The role of Biostatisticians, Bioinformaticians & other Data Experts in Clinical Research

As a medical researcher or a small enterprise in the life sciences industry, you are likely to encounter many experts using statistical and computational techniques to study biological, clinical and other health data. These experts can come from a variety of fields such as biostatistics, bioinformatics, biometrics, clinical data science and epidemiology. Although these fields do overlap in certain ways they differ in purpose, focus, and application. All four areas listed above focus on analysing and interpreting either biological, clinical data or public health data but they typically do so in different ways and with different goals in mind. Understanding these differences can help you choose the most appropriate specialists for your research project and get the most out of their expertise. This article will begin with a brief description of these disciplines for the sake of disambiguation, then focus on biostatistics and bioinformatics, with a particular overview of the roles of biostatisticians and bioinformatics scientists in clinical trials.

Biostatisticians

Biostatisticians use advanced biostatistical methods to design and analyse pre-clinical experiments, clinical trials, and observational studies predominantly in the medical and health sciences. They can also work in ecological or biological fields which will not be the focus of this article. Biostatisticians tend to work on varied data sets, including a combination of medical, public health and genetic data in the context of clinical studies. Biostatisticians are involved in every stage of a research project, from planning and designing the study, to collecting and analysing the data, to interpreting and communicating the results. They may also be involved in developing new statistical methods and software tools. In the UK the term “medical statistician” has been in common use over the past 40 years to describe a biostatistician, particularly one working in clinical trials, but it is becoming less used due to the global nature of the life sciences industry.

Bioinformaticians

Bioinformaticians use computational and statistical techniques to analyse and interpret large datasets in the life sciences. They often work with multi-omics data such as genomics, proteomics transcriptomics data and use tools such as large databases, algorithms, and specialised software programs to analyse and make sense of sequencing and other data. Bioinformaticians develop analysis pipelines and fine-tune methods and tools for analysing biological data to fit the evolving needs of researchers.

Clinical data scientists

Data scientists use statistical and computational modelling methods to make predictions and extract insights from a wide range of data. Often, data is real-world big data of which it might not be practical to analyse using other methods. In a clinical development context data sources could include medical records, epidemiological or public health data, prior clinical study data, or IOT and IOB sensor data. Data scientists may combine data from multiple sources and types. Using analysis pipelines, machine learning techniques, neural networks, and decision tree analysis this data can be made sense of. The better the quality of the input data the more precise and accurate any predictive algorithms can be.

Statistical programmers

Statistical programmers help statisticians to efficiently clean and prepare data sets and mock TFLs in preparation for analysis. They set up SDTM and ADaM data structures in preparation for clinical studies. Quality control of data and advanced macros for database management are also key skills.

Biometricians

Biometricians use statistical methods to analyse data related to the characteristics of living organisms. They may work on topics such as growth patterns, reproductive success, or the genetic basis of traits. Biometricians may also be involved in developing new statistical methods for analysing data in these areas. Some use the terms biostatistician and biometrician interchangeably however for the purpose of this article they remain distinct.

Epidemiologists

Epidemiologists study the distribution and determinants of diseases in populations. Using descriptive, analytical, or experimental techniques, such as cohort or case-control studies, they identify risk factors for diseases, evaluate the effectiveness of public health interventions, as well as track or model the spread of infectious diseases. Epidemiologists use data from laboratory testing, field studies, and publicly available health data. They can be involved in developing new public health policies and interventions to prevent or control the spread of diseases.

Clinical trials and the role of data experts

Clinical trials involve testing new treatments, interventions, or diagnostic tests in humans. These studies are an important step in the process of developing new medical therapies and understanding the effectiveness and safety of existing treatments.

Biostatisticians are crucial to the proper design and analysis of clinical trials. So that optimal study design can take place, they may first have to conduct extensive meta-analysis of previous clinical studies and RWE generation based on available real-world data sets or R&D results. They may also be responsible for managing the data and ensuring its quality, as well as interpreting and communicating the results of the trial. From developing the statistical analysis plan and contributing to the study protocol, to final analysis and reporting, biostatisticians have a role to play across the project time-line.

During a clinical trial, statistical programmers may prepare data sets to CDISC standards and pre-specified study requirements, maintain the database, as well as develop and implement standard SAS code and algorithms used to describe and analyse the study data.

Bioinformaticians may be involved in the design and analysis stages of clinical trials, particularly if the trial design involves the use of large data sets such as sequencing data for multi-omics analysis. They may be responsible for managing and analysing this data, as well as developing software tools and algorithms to support the analysis.

Data scientists may be involved in designing and analysing clinical trials at the planning stage, as well as in developing new tools and methods. The knowledge gleaned from data science models can be used to improve decision-making across various contexts, including life sciences R&D and clinical trials. Some applications include optimising the patient populations used in clinical trials; feasibility analysis using simulation of site performance, region, recruitment and other variables, to evaluate the impacts of different scenarios on project cost and timeline.

Biometricians and epidemiologists may also contribute to clinical trials, particularly if the trial is focused on a specific population or on understanding the factors that influence the incidence or severity of a disease. They may contribute to the design of the study, collecting and analysing the data, or interpreting the results.

Overall, the role of these experts in clinical trials is to use their varied expertise in statistical analysis, data management, and research design to help understand the safety and effectiveness of new treatments and interventions.

The role of biostatistician in clinical trials

Biostatisticians may be responsible for developing the study protocol, determining the sample size, producing the randomisation schedule, and selecting the appropriate statistical methods for analysing the data. They may also be responsible for managing the data and ensuring its quality, as well as interpreting and communicating the results of the trial.

SDTM data preparation

The Study Data Tabulation Model (SDTM) is a data standard that is used to structure and organize clinical study data in a standardized way. Depending on how a CRO is structured, either biostatisticians, statistical programmers, or both will be involved in mapping the data collected in a clinical trial to the SDTM data set, which involves defining the structure and format of the data and ensuring that it is consistent with the standard. This helps to ensure that the data is organised in a way that is universally interpretable. This process involves working with the research team to ensure the appropriate variables and categories are defined before reviewing and verifying the data to ensure that it is accurate, complete and in line with industry standards. Typically the SDTM data set will be established early at the protocol phase and populated later once trial data is accumulated.

Creating and analysing the ADaM dataset

In clinical trials, the Analysis Data Model (ADaM) is a data set model used to structure and organize clinical trial data in a standardized way for the purpose of statistical analysis. ADaM data sets are used to store the data that will be analysed as part of the clinical trial, and are typically created from the Study Data Tabulation Model (SDTM) data sets, which contain the raw data collected during the trial. This helps to ensure the reliability and integrity of the data, and makes it easier to analyse and interpret the results of the trial.

Biostatisticians and statistical programmers are responsible for developing ADaM data sets from the SDTM data, which involves selecting the relevant variables and organizing them in a way that is appropriate for the particular statistical analyses that will be conducted. While statistical programmers may create derived variables, produce summary statistics, TFLs, and organise the data into appropriate datasets and domains, biostatisticians are responsible for conducting detailed statistical analyses of the data and interpreting the results. This may include tasks such as testing hypotheses, identifying patterns and trends in the data, and developing statistical models to understand the relationships between the data and the research questions the trial seeks to answer.

The role of biostatisticians, specifically, in developing ADaM data sets from SDTM data is to use their expertise in statistical analysis and research design to guide statistical programmers in ensuring that the data is organised, structured, and formatted in a way that is appropriate for the analyses that will be conducted, and to help understand and interpret the results of the trial.

A Biostatistician’s role in study design & planning

Biostatisticians play a critical role in the design, analysis, and interpretation of clinical trials. The role of the biostatistician in a clinical trial is to use their expertise in statistical analysis and research design to help ensure that the trial is conducted in a scientifically rigorous and unbiased way, and to help understand and interpret the results of the trial. Here is a general overview of the tasks that a biostatistician might be involved in during the different stages of a clinical trial:

Clinical trial design: Biostatisticians may be involved in designing the clinical trial, including determining the study objectives, selecting the appropriate study population, and developing the study protocol. They are responsible for determining the sample size and selecting the appropriate statistical methods for analysing the data. Often in order to carry out these tasks, preparatory analysis will be necessary in the form of detailed meta-analysis or systematic review.

Sample size calculation: Biostatisticians are responsible for determining the required sample size for the clinical trial. This is an important step, as the sample size needs to be large enough to detect a statistically significant difference between the treatment and control groups, but not so large that the trial becomes unnecessarily expensive or time-consuming. Biostatisticians use statistical algorithms to determine the sample size based on the expected effect size, the desired level of precision, and the expected variability of the data. This information is informed by expert opinion and simulation of the data from previous comparable studies.

Randomisation schedules: Biostatisticians develop the randomisation schedule for the clinical trial, which is a plan for assigning subjects to the treatment and control groups in a random and unbiased way. This helps to ensure that the treatment and control groups are similar in terms of their characteristics, which helps to reduce bias or control for confounding factors that might affect the results of the trial.

Protocol development: Biostatisticians are involved in developing the statistical and methodological sections of the clinical trial protocol, which is a detailed plan that outlines the objectives, methods, and procedures of the study. In addition to outlining key research questions and operational procedures the protocol should include information on the study population, the interventions being tested, the outcome measures, and the data collection and analysis methods.

Data analysis: Biostatisticians are responsible for analysing the data from the clinical trial, including conducting interim analyses and making any necessary adjustments to the protocol. They play a crucial role in interpreting the results of the analysis and communicating the findings to the research team and other stakeholders.

Final analysis and reporting: Biostatisticians are responsible for conducting the final analysis of the data and preparing the final report of the clinical trial. This includes summarising the results, discussing the implications of the findings, and making recommendations for future research.

The role of bioinformatician in biomarker-guided clinical studies.

Biomarkers are biological characteristics that can be measured and used to predict the likelihood of a particular outcome, such as the response to a particular treatment. Biomarker-guided clinical trials use biomarkers as a key aspect of the study design and analysis. In biomarker-guided clinical trials where the biomarker is based on genomic sequence data, bioinformaticians may play a particularly important role in managing and analysing the data. Genomic and other omics data is often large and complex, and requires specialised software tools and algorithms to analyse and interpret. Bioinformaticians develop and implement these tools and algorithms, as well as for managing and analysing the data to identify patterns and relationships relevant to the trial. Bioinformaticians use their expertise in computational biology to to help understand the relationship between multi-omics data and the outcome of the trial, and to identify potential biomarkers that can be used to guide treatment decisions.

Processing sequencing data is a key skill of bioinformaticians that involves several steps, which may vary depending on the specific goals of the analysis and the type of data being processed. Here is a general overview of the steps that a bioinformatician might take to process sequencing data:

  1. Data pre-processing: Cleaning and formatting the data so that it is ready for analysis. This may include filtering out low-quality data, correcting errors, and standardizing the format of the data.
  2. Mapping: Aligning the sequenced reads to a reference genome or transcriptome in order to determine their genomic location. This can be done using specialized software tools such as Bowtie or BWA.
  3. Quality control: Checking the quality of the data and the alignment, and identifying and correcting any problems that may have occurred during the sequencing or mapping process. This may involve identifying and removing duplicate reads, or identifying and correcting errors in the data.
  4. Data analysis: Using statistical and computational techniques to identify patterns and relationships in the data such as identifying genetic variants, analysing gene expression levels, or identifying pathways or networks that are relevant to the study.
  5. Data visualization: Creating graphs, plots, and other visualizations to help understand and communicate the results of the analysis.

Once omics data has been analysed, the insights obtained can be used for tailoring therapeutic products to patient populations in a personalised medicine approach.

A changing role of data experts in life sciences R&D and clinical research

Due to the need for better therapies and health solutions, researchers are currently defining diseases at more granular levels using multi-omics insights from DNA sequencing data which allows differentiation between patients in the biomolecular presentation of their disease, demographic factors, and their response to treatment. As more and more of the resulting therapies reach the market the health care industry will need to catch up in order to provide these new treatment options to patients.

Even after a product receives regulatory approval, payers can opt not to reimburse patients, so financial benefit should be demonstrated in advance where possible. Patient reported outcomes and other health outcomes are becoming important sources of data to consider in evidence generation. Evidence provided to payers should aim to demonstrate financial as well as clinical benefit of the product.

In this context, regulators are becoming aware of the need for innovation in developing new ways of collecting treatment efficacy and other data used to assess novel products for regulatory approval. The value of observational studies and real-world-data sources as a supplement clinical trial data is being acknowledged as a legitimate and sometimes necessary part of the product approval process. Large scale digitisation now makes it easier to collect patient-centric data directly from clinical trial participants and users via devices and apps. Establishing clear evidence expectations from regulatory agencies then Collaborating with external stakeholders, data product experts, and service-providers to help build new evidence-building approaches.

Expert data governance and quality control is crucial to the success of any new methods to be implemented analytically. Data from different sources, such as IOT sensor data, electronic health records, sequencing data for multi-omics analysis, and other large data sets, has to be combined cautiously and with robust expert standards in place.

From biostatistics, bioinformatics, data science, CAS, and epidemiology for public heath or post-market modelling; a bespoke team of integrated data and analytics specialists is now as important to a product development project as the product itself to gaining competitiveness and therefore success in the marketplace. Such a team should be applying a combination of established data collection methodologies eg. clinical trials and systematic review, and innovative methods such as machine learning models that draw upon a variety of real world data sources to find a balance between advancing important innovation and mitigating risk.

Sex Differences in Clinical Trial Recruiting

The following article investigates several systematic reviews into sex and gender representation in individual clinical trial patient populations. In these studies sex ratios are assessed and evaluated by various factors such as clinical trial phase, disease type under investigation and disease burden in the population. Sex differences in the reporting of safety and efficacy outcomes are also investigated. In many cases safety and efficacy outcomes are pooled, rather than reported individually for each sex, which can be problematic when findings are generalised to the wider population. In order to get the dosage right for different body compositions and avoid unforeseen outcomes in off label use or when a novel therapeutic first reaches the market, it is important to report sex differences in clinical trials. Due to the unique nuances of disease types and clinical trial phases it is important to realise that a 50-50 ratio of male to female is not always the ideal or even appropriate in every clinical study design. Having the right sex balance in your clinical trial population will improve the efficiency and cost-effectiveness of your study. Based upon the collective findings a set of principles are put forth to guide the researcher in determining the appropriate sex ratio for their clinical trial design.

Sex difference by clinical trial phase

  • variation in sex enrolment ratios for clinical trial phases
  • females less likely to participate in early phases, due to increased risk of adverse events
  • under-representation of women in phase III when looking at disease prevalence

It has been argued that female representation in clinical trials is lacking, despite recent efforts to mitigate the gap. US data from 2000-2020 suggests that trial phase has the greatest variation in enrolment when compared to other factors, with median female enrolment being 42.9%, 44.8%, 51.7%, and 51.1% for phases I, I/II to II, II/III to III, and IV4. This shows that median female enrolment gradually increases as trials progress, with the difference in female enrolment between the final phases II/III to III and IV being <1%. Additional US data on FDA approved drugs including trials from as early as 1993 report that female participation in clinical trials is 22%, 48%, and 49% for trial phases I, II, and III respectively2. While the numbers for participating sexes are almost equal in phases II and III, women make up only approximately one fifth of phase I trial populations in this dataset2. The difference in reported participation for phase I trials between the datasets could be due to an increase in female participation in more recent years. The aim of a phase I trial is to evaluate safety and dosage, so it comes as no surprise that women, especially those of childbearing age, are often excluded due to potential risks posed to foetal development.

In theory, women can be included to a greater extent as trial phases progress and the potential risk of severe adverse events decreases. By the time a trial reaches phase III, it should ideally reflect the real-world disease population as much as possible. European data for phase III trials from 2011-2015 report 41% of participants being female1, which is slightly lower than female enrolment in US based trials. 26% of FDA approved drugs have a >20% difference between the proportion of women in phase II & III clinical trials and the prevalence of women in the US with the disease2, and only one of these drugs shows an over-representation of women.

Reporting of safety and efficacy by sex difference

  • Both safety and efficacy results tend to differ by sex.
  • Reporting these differences is inconsistent and often absent
  • Higher rates of adverse events in women are possibly caused by less involvement or non stratification in dose finding and safety studies.
  • There is a need to enforce analysis and reporting of sex differences in safety and efficacy data

Sex differences in response to treatment regarding both efficacy and safety have been widely reported. Gender subgroup analyses regarding efficacy can reveal whether a drug is more or less effective in one sex than the other. Gender subgroup analyses for efficacy are available for 71% of FDA approved drugs, and of these 11% were found to be more efficacious in men and 7% in women2. Alternatively, only 2 of 22 European Medicines Agency approved drugs examined were found to have efficacy differences between the sexes1. Nonetheless, it is important to study the efficacy of a new drug on all potential population subgroups that may end up taking that drug.

The safety of a treatment also differs between the sexes, with women having a slightly higher percentage (p<0.001) of reported adverse events (AE) than men for both treatment and placebo groups in clinical trials1. Gender subgroup analyses regarding safety can offer insights into the potential risks that women are subjected to during treatment. Despite this, gender specific safety analyses are available for only 45% of FDA approved drugs, with 53% of these reporting more side effects in women2. On average, women are at a 34% increased risk of severe toxicity for each cancer treatment domain, with the greatest increased risk being for immunotherapy (66%). Moreover, the risk of AE is greater in women across all AE types, including patient-reported symptomatic (female 33.3%, male 27.9%), haematologic (female 45.2%, male 39.1%) and objective non-haematologic (female 30.9%, male 29.0%)3. These findings highlight the importance of gender specific safety analyses and the fact that more gender subgroup safety reporting is needed. More reporting will increase our understanding of sex-related AE and could potentially allow for sex-specific interventions in the future.

Sex differences by disease type and burden

  • Several disease categories have recently been associated with lower female enrolment
  • Men are under-represented as often as women when comparing enrolment to disease burden proportions
  • There is a need for trial participants to be recruited on a case-by-case basis, depending on the disease.

Sex differences by disease type

When broken down by disease type, the sex ratio of clinical trial participation shows a more nuanced picture. Several disease categories have recently been associated with lower female enrolment, compared to other factors including trial phase, funding, blinding, etc4. Women comprised the smallest proportions of participants in US-based trials between 2000-2020 for cardiology (41.4%), sex-non-specific nephrology and genitourinary (41.7%), and haematology (41.7%) clinical trials4. Despite women being

proportionately represented in European phase III clinical studies between 2011-2015 for depression, epilepsy, thrombosis, and diabetes, they were significantly under-represented for hepatitis C, HIV, schizophrenia, hypercholesterolaemia, and heart failure and were not found to be overrepresented in trials for any of the disease categories examined1. This shows that the gap in gender representation exists even in later clinical trial phases when surveying disease prevalence, albeit to a lesser extent. Examining disease burden shows that the gap is even bigger than anticipated and includes the under-representation of both sexes.

Sex Differences by Disease Burden

It is not until the burden of disease is considered that men are shown to be under-represented as often as women. Including burden of disease can depict proportionality relative to the variety of disease manifestations between men and women. It can be measured as disability-adjusted life years (DALYs), which represent the number of healthy years of life lost due to the disease. Despite the sexes each making up approximately half of clinical trial participants overall in US-based trials between 2000-2020, all disease categories showed an under-representation of either women or men relative to disease burden, except for infectious disease and dermatologic clinical trials4. Women were under-represented in 7 of 17 disease categories, with the greatest under-representation being in oncology trials, where the difference between the number of female trial participants and corresponding DALYs is 3.6%. Men were under-represented compared with their disease burden in 8 of 17 disease categories, with the greatest difference being 11.3% for musculoskeletal disease and trauma trials.4 Men were found to be under-represented to a similar extent to women, suggesting that the under-representation of either sex could be by coincidence. Alternatively, male under-representation could potentially be due to the assumption of female under-representation leading to overcorrection in the opposite direction. It should be noted that these findings would benefit from statistical validation, although they illustrate the need for clinical trial participants to be recruited on a case-by-case basis, depending on the disease.

Takeaways to improve your patient sample in clinical trial recruiting:

  1. Know the disease burden/DALYs of your demographics for that disease.
  2. Try to balance the ratio of disease burden to the appropriate demographics for your disease
  3. Aim to recruit patients based on these proportions
  4. Stratify clinical trial data by the relevant demographics in your analysis. For example: toxicity, efficacy, adverse events etc should always be analyses separately for male and female to come up wit the respective estimates.
  5. Efficacy /toxicity etc should always be reported separately for male and female. reporting difference by ethnicity is also important as many diseases differentially affect certain ethnicity and the corresponding therapeutics can show differing degrees of efficacy and adverse events.

The end goal of these is that medication can be more personalised and any treatment given is more likely to help and less likely to harm the individual patient.

Conclusions

There is room for improvement in the proportional representation of both sexes in clinical trials and knowing a disease demographic is vital to planning a representative trial. Assuming the under-representation is on the side of female rather than male may lead to incorrect conclusions and actions to redress the balance. Taking demographic differences in disease burden into account when recruiting trial participants is needed. Trial populations that more accurately depict the real-world populations will allow a therapeutic to be tailored to the patient.

Efficacy and safety findings highlight the need for clinical study data to be stratified by sex, so that respective estimates can be determined. This enables more accurate, sex/age appropriate dosing that will maximise treatment efficacy and patient safety, as well as minimise the chance of adverse events. This also reduces the risks associated with later off label use of drugs and may avoid modern day tragedies resembling the thalidomide tragedy. Moreover, efficacy and adverse events should always be reported separately for men and women, as the evidence shows their distinct differences in response to therapeutics.

References:

1. Dekker M, de Vries S, Versantvoort C, Drost-van Velze E, Bhatt M, van Meer P et al. Sex Proportionality in Pre-clinical and Clinical Trials: An Evaluation of 22 Marketing Authorization Application Dossiers Submitted to the European Medicines Agency. Frontiers in Medicine. 2021;8.

2. Labots G, Jones A, de Visser S, Rissmann R, Burggraaf J. Gender differences in clinical registration trials: is there a real problem?. British Journal of Clinical Pharmacology. 2018;84(4):700-707.

3. Unger J, Vaidya R, Albain K, LeBlanc M, Minasian L, Gotay C et al. Sex Differences in Risk of Severe Adverse Events in Patients Receiving Immunotherapy, Targeted Therapy, or Chemotherapy in Cancer Clinical Trials. Journal of Clinical Oncology. 2022;40(13):1474-1486.

4. Steinberg J, Turner B, Weeks B, Magnani C, Wong B, Rodriguez F et al. Analysis of Female Enrollment and Participant Sex by Burden of Disease in US Clinical Trials Between 2000 and 2020. JAMA Network Open. 2021;4(6):e2113749.

Estimating the Costs Associated with Novel Pharmaceutical development: Methods and Limitations.

Data sources for cost analysis of drug development R&D and clinical trials

Cost estimates for pre-clinical and clinical development across the pharmaceutical industry differ based on several factors. One of these is the source of data used by each costing study to inform these estimates. Several studies use private data, which can include confidential surveys filled out by pharmaceutical firms/clinical trial units and random samples from private databases3,9,10,14,15,16. Other studies have based their cost estimates upon publicly available data, such as data from the FDA/national drug regulatory agencies, published peer-reviewed studies, and other online public databases1,2,12,13,17.

Some have questioned the validity of using private surveys from large multinational pharmaceutical companies to inform cost estimates, saying that survey data may be artificially inflated by pharmaceutical companies to justify high therapeutic prices 18,19,20. Another concern is that per trial spending by larger pharmaceutical companies and multinational firms would far exceed the spending of start-ups and smaller firms, meaning cost estimates made based on data from these larger companies would not be representative of smaller firms.

Failure rate of R&D and clinical trial pipelines

Many estimates include the cost of failures, which is especially the case for cost estimates “per approved drug”. As many compounds enter the clinical trial pipeline, the cost to develop one approved drug/compound includes cost of failures by considering the clinical trial success rate and cost of failed compounds. For example, if 100 compounds enter phase I trials, and 2 compounds are approved, the clinical cost per approved drug would include the amount spent on 50 compounds.

The rate of success used can massively impact cost estimates, where a low success rate or high failure rate will lead to much higher costs per approved drug. The overall probability of clinical success may vary by year and has been estimated at a range of values including 7.9%21, 11.83%10, and 13.8%22. There are concerns that some studies suggesting lower success rates have relied on small samples from industry curated databases and are thereby vulnerable to selection bias12,22.

Success rates per phase transition also affects overall costs. When more ultimately unsuccessful compounds enter late clinical trial stages, the higher the costs are per approved compound. In addition, success rates are also dependent on therapeutic area and patient stratification by biomarkers, among other factors. For example, one study estimated the lowest success rate at 1.6% for oncological trials without biomarker use compared with a peak success rate of 85.7% for cardiovascular trials utilising biomarkers22. While aggregate success rates can be used to estimate costs, using specific success rates will be more accurate to estimate the cost of a specific upcoming trial, which could help with budgeting and funding decisions.

Out-of-pocket costs vs capitalised costs & interest rates

Cost estimates also differ due to reporting of out-of-pocket and capitalised costs. An out-of-pocket cost refers to the amount of money spent or expensed on the R&D of a therapeutic. This can include all aspects of setting up therapeutic development, from initial funding in drug discovery/device design, to staff and site costs during clinical trials, and regulatory approval expenses.

The capitalised cost of a new therapeutic refers to the addition of out-of-pocket costs to a yearly interest rate applied to the financial investments funding the development of a new drug. This interest rate, referred to as the discount rate, is determined by (and is typically greater than) the cost of capital for the relevant industry.

Discount rates for the pharmaceutical industry vary between sources and can dramatically alter estimates for capitalised cost, where a higher discount rate will increase capitalised cost. Most studies place the private cost of capital for the pharmaceutical industry to be 8% or higher23,24, while the cost of capital for government is lower at around 3% to 7% for developed countries23,25. Other sources have suggested rates from as high as 13% to as low as zero13,23,26, where the zero cost of capital has been justified by the idea that pharmaceutical firms have no choice but to invest in R&D. However, the mathematical model used in many estimations for the cost of industry capital, the CAPM model, tends to give more conservative estimates23. This would mean the 10.5% discount rate widely used in capitalised cost estimates may in fact result in underestimation.

While there is not a consensus on what discount rate to use, capitalised costs do represent the risks undertaken by research firms and investors. A good approach may be to present both out-of-pocket and capitalised estimated costs, in addition to rates used, justification for the rate used, and the estimates using alternative rates in a sensitivity analysis26.

Costs variation over time

The increase in therapeutic development costs

Generally, there has been a significant increase in the estimated costs to develop a new therapeutic over time26. One study reported an exponential increase of capitalised costs from the 1970s to the mid-2010s, where the total capitalised costs rose annually 8.5% above general inflation from 1990 to 201310. Recent data has suggested that average development costs reached a peak in 2019 and had decreased the following two years9. This recent decrease in costs was associated with slightly reduced cycle times and an increased proportion of infectious disease research, likely in response to the rapid response needed for COVID-19.

Recent cost estimates

Costs can range with more than 100-fold differences for phase III/pivotal trials alone1. One of the more widely cited studies on drug costs used confidential survey data from ten multinational pharmaceutical firms and a random sample from a database of publicly available data10. In 2013, this study estimated the total pre-approval cost at $2.6 billion USD per approved new compound. This was a capitalised cost, and the addition of post-approval R&D costs increased this estimate to $2.87 billion (2013 USD). The out-of-pocket cost per approved new compound was reported at $1.395 billion, of which $965 million were clinical costs and the remaining $430 million were pre-clinical.

Another estimate reported the average cost to develop an asset at $1.296 billion in 20139. Furthermore, it reported that this cost had increased until 2019 at $2.431 billion and had since decreased to $2.376 billion in 2020 and $2.006 billion in 2021. While comparable to the previous out-of-pocket estimate for 2013, this study does not state whether their estimates are out-of-pocket or capitalised, making it difficult to meaningfully compare these estimates.

Figure 1: Recent cost estimates for drug development per approved new compound. “Clinical only” costs include only the costs of phase 0-III clinical trials, while “full” costs include pre-clinical costs. The colour of each bubble indicates the study, while bubble size indicated relative cost. A dashed border indicated the study used private data for their estimations, while a solid border indicates the study utilised publicly available data. Figure represents studies 9, 10, 12, 13 and 17 from the reference list in this report.

Publicly available data of 63 FDA-approved new biologics from 2009-2018 was used to estimate the capitalised (at 10.5%) R&D investment to bring a new drug to market at median of $985.3 million and a mean of $1.3359 billion (inflation adjusted to 2018 USD)12. These data were mostly accessible from smaller firms, smaller trials, first-in-class drugs, and further specific areas. The variation in estimated cost was, through sensitivity analysis, mostly explained by success/failure rates, preclinical expenditures, and cost of capital.

Publicly available data of 10 companies with no other drugs on the market in 2017 was used to estimate out-of-pocket costs for the development of a single cancer drug. This was reported at a median of $648 million and a mean of $719.8 million13. Capitalised costs were also reported using a 7% discount rate, with a median of $754.4 million and mean of $969.4 million. By focusing on data from companies without other drugs on the market, these estimates may better represent the development costs per new molecular entity (NME) for start-ups as the cost of failure of other drugs in the pipeline were included while any costs related to supporting existing on-market drugs were systematically impossible to include.

One study estimated the clinical costs per approved non-orphan drug at $291 million (out-of-pocket) and $412 million (capitalised 10.5%)17. The capitalised cost estimate increased to $489 million when only considering non-orphan NMEs. The difference between these estimates for clinical costs and the previously mentioned estimates for total development costs puts into perspective the amount

spent on pre-clinical trials and early drug development, with one studynoting their pre-clinical estimates comprised 32% of out-of-pocket and 42% of capitalised costs10.

Things to consider about cost estimates

The issue with these estimates is that there are so many differing factors affecting each study. This complicates cost-based pricing discussions, especially when R&D cost estimates can differ orders of magnitude apart. The methodologies used to calculate out-of-pocket costs differ between studies9,17, and the use of differing data sources (public data vs confidential surveys) seem to impact these estimates considerably.

There is also an issue with the transparency of data and methods from various sources in cost estimates. Some of this is a result of using confidential data, where some analyses are not available for public scrutiny8. This study in particular raised questions as estimates were stated without any information about the methodology or data used in the calculation of estimates. The use of confidential surveys of larger companies has also been criticised as the confidential data voluntarily submitted would have been submitted anonymously without independent verification12.

Due to the limited amount of comprehensive and published cost data17, many estimates have no option but to rely on using a limited data set and making some assumptions to arrive at a reasonable estimate. This includes a lack of transparent available data for randomised control trials, where one study reported that only 18% of FDA-approved drugs had publicly available cost data18. Therefore, there is a need for transparent and replicable data in this field to allow for more plausible cost estimates to be made, which in turn could be used to support budget planning and help trial sustainability18,26.

Despite the issues between studies, the findings within each study can be used to gather an idea of trends, cost drivers, and costs specific to company/drug types. For example, studies suggest an increasing overall cost of drug development from 1970 to peak in 201910, with a subsequent decrease in 2020 and 20219.

For a full list of references used in this article, please see the main report here: https://anatomisebiostats.com/biostatistics-blog/how-much-does-developing-a-novel-therapeutic-cost-factors-affecting-drug-development-costs-in-the-pharma-industry-a-mini-report/

How much does developing a novel therapeutic cost? – Factors Affecting Drug Development Costs across the Pharma Industry: A mini-Report

Introduction

Data evaluating the costs associated with developing novel therapeutics within the pharmaceutical industry can be used to identify trends over time and can inform more accurate budgeting for future research projects. However, the cost to develop a drug therapeutic is difficult to accurately evaluate, resulting in varying estimates ranging from hundreds of millions to billions of US dollars between studies. The high cost of drug development is not purely because of clinical trial expenses. Drug discovery, pre-clinical trials, and commercialisation also need to be factored into estimates of drug development costs.

There are limitations in trying to accurately assess these costs. The sheer number of factors that affect estimated and real costs means that studies often take a more specific approach. For example, costs will differ between large multinational companies with multiple candidates in their pipeline and start-ups/SMEs developing their first pharmaceutical. Due to the amount and quality of available data, many studies work mostly with data from larger multinational pharmaceutical companies with multiple molecules in the pipeline. When taken out of context, the “$2.6 billion USD cost for getting a single drug to market” can seem daunting for SMEs. It is very important to clarify what scale these cost estimates represent, but cost data from large pharma companies are still relevant for SMEs as they can used to infer costs for different scales of therapeutic development.

This mini-report includes what drives clinical trial costs, methods to reduce these costs, and then explores what can be learned from varying cost estimates.

What drives clinical trial costs?

There is an ongoing effort to streamline the clinical trial process to be more cost and time efficient. Several studies report on cost drivers of clinical trials, which should be considered when designing and budgeting a trial. Some of these drivers are described below:

Study size

Trial costs rise exponentially with an increasing study size, which some studies have found to be the single largest driver in trial costs1,2,3. There are several reasons for varying sample sizes between trials. For example, study size increases with trial phase progression as phases require different study sizes based on the number of patients needed to establish the safety and/or effectiveness of a treatment. Failure to recruit sufficient patients can result in trial delays which also increases costs4.

Trial site visits

A large study size is also correlated with a larger overall number of patient visits during a trial, which is associated with a significant increase in total trial costs2,3. Trial clinic visits are necessary for patient screening, treatment and treatment assessment but include significant costs for staff, site hosting, equipment, treatment, and in some cases reimbursement for patient travel costs. The number of trial site visits per patient varies between trials where more visits may indicate longer and/or more intense treatment sessions. One estimate for the number of trial visits per person was a median of 11 in a phase III trial, with $2 million added to estimated trial costs for every +1 to the median2.

Number & location of clinical trial sites

A higher number of clinical trial study sites has been associated with significant increase in total trial cost3. This is a result of increased site costs, as well as associated staffing and equipment costs. These will vary with the size of each site, where larger trials with more patients often use more sites or larger sites.

Due to the lower cost and shorter timelines of overseas clinical research5,6, there has been a shift to the globalisation of trials, with only 43% of study sites in US FDA-approved pivotal trials being in North America7. In fact, 71% of these trials had sites in lower cost regions where median regional costs were 49%-97% of site costs in North America. Most patients in these trials were either in North America (39.7%), Western Europe (21%), or Central Europe (20.4%).

Median cost per regional site as a percentage of North American median cost for comparison.

However, trials can face increased difficulties in managing and coordinating multiple sites across different regions, with concerns of adherence to the ethical and scientific regulations of the trial centre’s region5,6. Some studies have reported that multiregional trials are associated with a significant increase in total trial costs, especially those with sites in emerging markets3. It is unclear if this reported increase is a result of lower site efficiency, multiregional management costs, or outsourcing being more common among larger trials.

Clinical Trial duration

Longer trial duration has been associated with a significant increase in total trial costs3,4, where many studies have estimated the clinical period between 6-8 years8,9,10,11,12,13. Longer trials are sometimes necessary, such as in evaluating the safety and efficacy of long-term drug use in the management of chronic and degenerative disease. Otherwise, delays to starting up a trial contribute to longer trials, where delays can consume budget and diminish the relevance of research4. Such delays may occur as a result of site complications or poor patient accrual.

Another aspect to consider is that the longer it takes to get a therapeutic to market (as impacted by longer trials), the longer the wait is before a return of investment is seen by both the research organisation and investors. The period from development to on-market, often referred to as cycle time, can drive costs per therapeutic as interest based on the industry’s cost of capital can be applied to investments.

Therapeutic area under investigation

The cost to develop a therapeutic is also dependent on the therapeutic area, where some areas such as oncology and cardiovascular treatments are more cost intensive compared with others1,2,5,6,12,14. This is in part due to variation in treatment intensity, from low intensity treatments such as skin creams to high intensity treatments such as multiple infusions of high-cost anti-cancer drugs2. An estimate for the highest mean cost for pivotal trials per therapeutic area was $157.2M in cardiovascular trials compared to $45.4M in oncology, and a lowest of $20.8M in endocrine, metabolic, and respiratory disease trials1. This was compared to an overall median of $19M. Clinical

trial costs per therapeutic area also varied by clinical trial phase. For example, trials in pain and anaesthesia have been found to have the lowest average cost of a phase I study while having the highest average cost of a phase III study6.

It is important to note that some therapeutic areas will have far lower per patient costs when compared to others and are not always indicative of total trial costs. For example, infectious disease trials generally have larger sample sizes which will lead to relatively low per patient costs, whereas trials for rare disease treatment are often limited to smaller sample sizes with relatively high per patient costs. Despite this, trials for rare disease are estimated to have significantly lower drug to market costs.

Drug type being evaluated

As mentioned in the therapeutic areas section above, treatments may vary in intensity from skin creams to multiple rounds of treatment with several anti-cancer drugs. This can drive total trial costs due to additional manufacturing and the need for specially trained staff to administer treatments.

In the case of vaccine development, phase III/pivotal trials for vaccine efficacy can be very difficult to run unless there are ongoing epidemics for the targeted infectious disease. Therefore, some cost estimates of vaccine development include from the pre-clinical stages to the end of phase IIa, with the average cost for one approved vaccine estimated at $319-469 million USD in 201815.

Study design & trial control type used

Phase III trial costs vary based on the type of control group used in the trial1. Uncontrolled trials were the least expensive with an estimated mean of $13.5 million per trial. Placebo controlled trials had an estimated mean of $28.8 million, and trials with active drug comparators had an estimated mean cost of $48.9 million. This dramatic increase in costs is in part due to manufacturing and staffing to administer a placebo or active drug. In addition, drug-controlled trials require more patients compared to placebo-controlled, which also requires more patients than uncontrolled trials2.

Reducing therapeutic development costs

Development costs can be reduced through several approaches. Many articles recommend improvements to operational efficiency and accrual, as well as deploying standardised trial management metrics4. This could include streamlining trial administration, hiring experienced trial staff, and ensuring ample patient recruitment to reduce delays in starting and carrying out a study.

Another way to reduce development costs can take place in the thorough planning of clinical trial design by a biostatistician, whether in-house or external. Statistics consulting throughout a trial can help to determine suitable early stopping conditions and the most appropriate sample size. Sample size calculation is particularly important as underestimation undermines experimental results, whereas overestimation leads to unnecessary costs. Statisticians can also be useful during the pre-clinical stage to audit R&D data to select the best available candidates, ensure accurate R&D data analysis, and avoid pursuing unsuccessful compounds.

Other ways to reduce development costs include the use of personalised medicine, clinical trial digitisation, and the integration of AI. Clinical trial digitisation would lead to the streamlining of clinical trial administration and would also allow for the integration of artificial intelligence into clinical trials. There have been many promising applications for AI in clinical trials, including the use of electronic health records to enhance the enrolment and monitoring of patients, and the potential use of AI in trial diagnostics. More information about this topic can be found in our blog “Emerging use-cases for AI in clinical trials”.

For more information on the methodology by which pharmaceutical development and clinical trials costs are estimated and what data has been used please see the article: https://anatomisebiostats.com/biostatistics-blog/estimating-the-costs-associated-with-novel-pharmaceutical-development-methods-and-limitations/

Cost breakdown in more detail: How is a clinical trial budget spent?

Clinical trial costs can be broken down and divided into several categories, such as staff and non-staff costs. In a sample of phase III studies, personnel costs were found to be the single largest component of trial costs, consisting of 37% of the total, whereas outsourcing costs made up 22%, grants and contracting costs at 21%, and other expenses at 21%3.

From a CRO’s perspective, there are many factors that are considered in the cost of a pivotal trial quotation, including regulatory affairs, site costs, management costs, the cost of statistics and medical writing, and pass-through costs27. Another analysis of clinical trial cost factors determined clinical procedure costs made up 15-22% of the total budget, with administrative staff costs at 11-29%, site monitoring costs at 9-14%, site retention costs at 9-16%, and central laboratory costs at 4-12%5,6. In a study of multinational trials, 66% of total estimated trial costs were spent on regional tasks, of which 53.3% was used in trial sites and the remainder on other management7.

Therapeutic areas and shifting trends

Therapeutic area had previously been mentioned as a cost driver of trials due to differences in sample sizes and/or treatment intensity. It is however worth mentioning that, in 2013, the largest number of US industry-sponsored clinical trials were in oncology (2,560/6,199 active clinical trials with 215,176/1,148,340 patients enrolled)4,14. More recently, there has been a shift to infectious disease trials, in part due to the needed COVID-19 trials9.

Clinical trial phases

Due to the expanding sample size as a trial progresses, average costs per phase increase from phase I through III. Median costs per phase were estimated in 2016 at $3.4 million for phase I, $8.6 million for phase II, and $21.4 million for phase III3. Estimations of costs per patient were similarly most expensive in phase III at $42,000, followed by phase II at $40,000 and phase I at $38,50014. The combination of an increasing sample size and increasing per patient costs per phase leads to the drastic increase in phase costs with trial progression.

In addition, studies may have multiple phase III trials, meaning the median estimated cost of phase III trials per approved drug is higher than per trial costs ($48 million and $19 million respectively)2. Multiple phase III trials can be used to better support marketing approval or can be used for therapeutics which seek approval for combination/adjuvant therapy.

There are fewer cost data analyses available on phase 0 and phase IV on clinical trials. Others report that average Phase IV costs are equivalent to Phase III but much more variable5,6.

Orphan drugs

Drugs developed for the treatment of rare diseases are often referred to as orphan drugs. Orphan drugs have been estimated to have lower clinical costs per approved drug, where capitalised costs per non-orphan and orphan drugs were $412 million and $291 million respectively17. This is in part due to the limit to sample size imposed upon orphan drug trials by the rarity of the target disease and the higher success rate for each compound. However, orphan drug trials are often longer when compared to non-orphan drug trials, with an average study duration of 1417 days and 774 days respectively.

NMEs

New molecular entities (NMEs) are drugs which do not contain any previously approved active molecules. Both clinical and total costs of NMEs are estimated to be higher when compared to next in class drugs13,17. NMEs are thought to be more expensive to develop due to the increased amount of pre-clinical research to determine the activity of a new molecule and the increased intensity of clinical research to prove safety/efficacy and reach approval.

Conclusion & take-aways

There is no one answer to the cost of drug or device development, as it varies considerably by several cost drivers including study size, therapeutic area, and trial duration. Estimates of total drug development costs per approved new compound have ranged from $754 million12 to $2.6 billion10 USD over the past 10 years. These estimates do not only differ based on the data used, but also due to methodological differences between studies. The limited availability of comprehensive cost data for approved drugs also means that many studies rely on limited data sets and must make assumptions to arrive at a reasonable estimate.

There are still multiple practical ways that can be used to reduce study costs, including expert trial design planning by statisticians, implementation of biomarker-guided trials to reduce the risk of failure, AI integration and digitisation of trials, improving operational efficiency, improving accrual, and introducing standardised trial management metrics.

References

Moore T, Zhang H, Anderson G, Alexander G. Estimated Costs of Pivotal Trials for Novel Therapeutic Agents Approved by the US Food and Drug Administration, 2015-2016. JAMA Internal Medicine. 2018;178(11):1451-1457.

.1 Moore T, Zhang H, Anderson G, Alexander G. Estimated Costs of Pivotal Trials for Novel Therapeutic Agents Approved by the US Food and Drug Administration, 2015-2016. JAMA Internal Medicine. 2018;178(11):1451-1457.

2. Moore T, Heyward J, Anderson G, Alexander G. Variation in the estimated costs of pivotal clinical benefit trials supporting the US approval of new therapeutic agents, 2015–2017: a cross-sectional study. BMJ Open. 2020;10(6):e038863.

3. Martin L, Hutchens M, Hawkins C, Radnov A. How much do clinical trials cost?. Nature Reviews Drug Discovery. 2017;16(6):381-382.

4. Bentley C, Cressman S, van der Hoek K, Arts K, Dancey J, Peacock S. Conducting clinical trials—costs, impacts, and the value of clinical trials networks: A scoping review. Clinical Trials. 2019;16(2):183-193.

5. Sertkaya A, Birkenbach A, Berlind A, Eyraud J. Examination of Clinical Trial Costs and Barriers for Drug Development [Internet]. ASPE; 2014. Available from: https://aspe.hhs.gov/reports/examination-clinical-trial-costs-barriers-drug-development-0

6. Sertkaya A, Wong H, Jessup A, Beleche T. Key cost drivers of pharmaceutical clinical trials in the United States. Clinical Trials. 2016;13(2):117-126.

7. Qiao Y, Alexander G, Moore T. Globalization of clinical trials: Variation in estimated regional costs of pivotal trials, 2015–2016. Clinical Trials. 2019;16(3):329-333.

8. Monitor Deloitte. Early Value Assessment: Optimising the upside value potential of your asset [Internet]. Deloitte; 2020 p. 1-14. Available from: https://www2.deloitte.com/content/dam/Deloitte/be/Documents/life-sciences-health-care/Deloitte%20Belgium_Early%20Value%20Assessment.pdf

9. May E, Taylor K, Cruz M, Shah S, Miranda W. Nurturing growth: Measuring the return from pharmaceutical innovation 2021 [Internet]. Deloitte; 2022 p. 1-28. Available from: https://www2.deloitte.com/content/dam/Deloitte/uk/Documents/life-sciences-health-care/Measuring-the-return-of-pharmaceutical-innovation-2021-Deloitte.pdf

10. DiMasi J, Grabowski H, Hansen R. Innovation in the pharmaceutical industry: New estimates of R&D costs. Journal of Health Economics. 2016;47:20-33.

11. Farid S, Baron M, Stamatis C, Nie W, Coffman J. Benchmarking biopharmaceutical process development and manufacturing cost contributions to R&D. mAbs. 2020;12(1):e1754999.

12. Wouters O, McKee M, Luyten J. Estimated Research and Development Investment Needed to Bring a New Medicine to Market, 2009-2018. JAMA. 2020;323(9):844-853.

13. Prasad V, Mailankody S. Research and Development Spending to Bring a Single Cancer Drug to Market and Revenues After Approval. JAMA Internal Medicine. 2017;177(11):1569-1575.

14. Battelle Technology Partnership Practice. Biopharmaceutical Industry-Sponsored Clinical Trials: Impact on State Economies [Internet]. Pharmaceutical Research and Manufacturers of America; 2015. Available from: http://phrma-docs.phrma.org/sites/default/files/pdf/biopharmaceutical-industry-sponsored-clinical-trials-impact-on-state-economies.pdf

15. Gouglas D, Thanh Le T, Henderson K, Kaloudis A, Danielsen T, Hammersland N et al. Estimating the cost of vaccine development against epidemic infectious diseases: a cost minimisation study. The Lancet Global Health. 2018;6(12):e1386-e1396.
16. Hind D, Reeves B, Bathers S, Bray C, Corkhill A, Hayward C et al. Comparative costs and activity from a sample of UK clinical trials units. Trials. 2017;18(1).

17.Jayasundara K, Hollis A, Krahn M, Mamdani M, Hoch J, Grootendorst P. Estimating the clinical cost of drug development for orphan versus non-orphan drugs. Orphanet Journal of Rare Diseases. 2019;14(1).

19. Speich B, von Niederhäusern B, Schur N, Hemkens L, Fürst T, Bhatnagar N et al. Systematic review on costs and resource use of randomized clinical trials shows a lack of transparent and comprehensive data. Journal of Clinical Epidemiology. 2018;96:1-11.

20. Light D, Warburton R. Demythologizing the high costs of pharmaceutical research. BioSocieties. 2011;6(1):34-50.

21. Adams C, Brantner V. Estimating The Cost Of New Drug Development: Is It Really $802 Million?. Health Affairs. 2006;25(2):420-428.

22. Thomas D, Chancellor D, Micklus A, LaFever S, Hay M, Chaudhuri S et al. Clinical Development Success Rates and Contributing Factors 2011–2020 [Internet]. BIO|QLS Advisors|Informa UK; 2021. Available from: https://pharmaintelligence.informa.com/~/media/informa-shop-window/pharma/2021/files/reports/2021-clinical-development-success-rates-2011-2020-v17.pdf

23. Wong C, Siah K, Lo A. Estimation of clinical trial success rates and related parameters. Biostatistics. 2019;20(2):273-286.
24. Chit A, Chit A, Papadimitropoulos M, Krahn M, Parker J, Grootendorst P. The Opportunity Cost of Capital: Development of New Pharmaceuticals. INQUIRY: The Journal of Health Care Organization, Provision, and Financing. 2015;52:1-5.
25. Harrington, S.E. Cost of Capital for Pharmaceutical, Biotechnology, and Medical Device Firms. In Danzon, P.M. & Nicholson, S. (Eds.), The Oxford Handbook of the Economics of the Biopharmaceutical Industry, (pp. 75-99). New York: Oxford University Press. 2012.
26. Zhuang J, Liang Z, Lin T, De Guzman F. Theory and Practice in the Choice of Social Discount Rate for Cost-Benefit Analysis: A Survey [Internet]. Manila, Philippines: Asian Development Bank; 2007. Available from: https://www.adb.org/sites/default/files/publication/28360/wp094.pdf
27. Rennane S, Baker L, Mulcahy A. Estimating the Cost of Industry Investment in Drug Research and Development: A Review of Methods and Results. INQUIRY: The Journal of Health Care Organization, Provision, and Financing. 2021;58:1-11.
28. Ledesma P. How Much Does a Clinical Trial Cost? [Internet]. Sofpromed. 2020 [cited 26 June 2022]. Available from: https://www.sofpromed.com/how-much-does-a-clinical-trial-cost


Medical Device Categorisation, Classification and Regulation in the United Kingdom

Contributor: Sana Shaikh

In this article

  • Overview of medical device categorisations and classifications for regulatory purposes in the United Kingdom
  • Summary of medical devices categorisations based on type, usage and risk potential during use as specified in the MDR and IVDR.
  • The class of medical device and its purpose determines the criteria required to meet regulatory approval. All medical devices in the UK must have a UKCA or CE marking depending on the legislation the device has been certified under.
  • Explanation of risk classifications for general medical devices and active implantable devices
  • Explanation of risk classifications for in vitro diagnostics

In the UK and EU medical devices are regulated under the Medical Devices Regulation (MDR) or In Vitro Diagnostics Regulation (IVDR) depending upon which category they fall under. In the UK it is the Medicines and Healthcare Products Regulatory Agency (MHRA) that is responsible for new product approval and market surveillance activities related to medical devices and other therapeutics, such as pharmaceuticals, intended for use in patients within the UK. The equivalent regulatory agency in the EU is the European Regulatory Agency (EMA). The MHRA also manages the Early Access to Medicines Scheme (EAMS) to enable patients access to pre-market therapeutics that are yet to receive regulatory approval where their medical needs are currently unmet by existing options. To qualify for EAMS a medicine must be designated as a Promising Innovative Medicine (PIM) based on early clinical data.

Having a thorough understanding of the classification and class of your medical device is vital for it to undergo the appropriate assessment route and be approved and ready for market. While the scope of medical devices is incredibly broad, for regulatory purposes they tend to be classified based on device type, duration of use and level of risk. Which risk class a device falls into will be determined in a large part by device type and duration of use, as both of these factors influence the level of risk to the patient. All medical devices in the UK must be designated a category and a risk classification in order to undertake the regulatory approval process.

Category (type) of Medical Device

The MHRA categorises medical devices into the following 5 categories:

  • Non-invasive – Devices which do not enter the body
  • Invasive – Devices which in whole or part are inserted into the body’s orifices (including the external eyeball surface) or through the surface or the body such as the skin.
  • Surgically invasive – Devices used or inserted surgically that penetrate the body through the surface of the body, such as through the skin.
  • Active – Devices requiring an external source of power, including stand-alone software.
  • Implantable – Devices intended to be totally or partially introduced into the human body (including to replace an epithelial surface or the surface of the eye) by surgical intervention and to remain in place for a period of time.

Duration of use category

Medical devices are then further categorised based upon their intended duration of use under normal circumstances.

  • Transient – intended for less than 60 minutes of continuous use.
  • Short term – intended for between 60 minutes to 30 days of continuous use.
  • Long term – intended for more than 30 days continuous use.

More information to aid accurate medical device categorisation in the UK and EU can be downloaded here: Medical devices: how to comply with the legal requirements in Great Britain – GOV.UK (www.gov.uk)

UKCA Mark & Conformity Assessment

Further to these use, duration and risk categories the HPRA designates 3 additional categories for the purposes of UKCA Mark and conformity assessment. These categories for the are:

  • General medical devices – most medical devices fall into this category.
  • Active implantable devices – devices powered by implants or partial implants intended to remain in the human body after a procedure.
  • In vitro diagnostics medical devices (IVDs) – equipment or system used in vitro to examine specimens from the human body.

UKCA mark and conformity assessment and subsequent labelling is a crucial procedure for a device to enter the UK market for use by patients. It should be noted that the UKCA mark is not recognised in the EU or Northern Ireland, who instead recognise the CE mark. Great Britain will not recognise the CE mark after 30 June 2023, thus it will be important to have both the UKCA and CE mark for widespread distribution of a medical device. These incompatibilities seem to have arisen largely as a result of Brexit.

Risk classification categories for general medical devices and active implantable devices

In The UK and EU there are 4 official risk-related classes for medical devices. These classes apply to both general medical devices and active implantable devices. As noted previously, the class a device falls into is largely informed by the category and the intended duration of use for the device.

  • Class I , which includes the subclasses Class Is (sterile no measuring function), Class Im (measuring function), and Class Ir (devices to be reprocessed or reused). Low risk of illness/injury resulting from use. Only self-assessment required to meet regulatory approval.
  • Class IIa Low to medium risk of illness/injury resulting from use. Notified Body approval required.
  • Class IIb Medium to high risk of illness/injury resulting from use. Notified Body approval required.
  • Class III high potential risk of illness/injury resulting from use. Notified Body approval required.

More details on these classes can be found below.

In Vitro Diagnostic Medical Devices (IVDs)

The IVDR categorise IVDs in to the following categories for the purpose of obtaining regulatory approval in Great Britain. IVDs do not harm patients directly in the same way that other medical devices can and are thus subject to different risk assessment.

  • General IVD medical devices
  • IVDs for self-testing – intended to be using by an individual at home.
  • IVDs stated in Part IV of the UK MDR 2002, Annex II List B
  • IVDs stated in Part IV of the UK MDR 2002, Annex II List A

A more detailed explanation of these categories can be found towards the end of this article.

The EU and Northern Ireland has moved away from this list style of classification and has recently implemented the following risk classes. There are 4 IVDR risk classes outlined in Annex VIII. It seems likely that Great Britain may follow this in future.

Risk Classes for IVDs

  • Class A – Laboratory devices, instruments and receptacles.
  • Class B – All devices not covered in the other classes.
  • Class C – High risk devices presenting a lower direct risk to the patient population. Includes diagnostic devices where failure to accurately diagnose could be life-threatening. Covers companion diagnostics, genetic screening and some self-testing.
  • Class D – Devices that pose a high direct risk to the patient population, and in some cases the wider population, relating to life threatening conditions, transmissible agents in blood, biological materials for transplantation in to the human body and other similar materials.

Risk categories for general medical devices and active implantable medical devices in detail

Class I devices

These are generally regarded as low risk devices and pose little risk of illness and injury. Such devices have minimal contact with patients and the lowest impact on patient health outcomes. To self-certify your product, you must confirm that it is a class I device1,3. This may involve carrying out clinical evaluations, notifying the Medicines and Healthcare products Regulatory Agency (MHRA) of proposals to perform clinical investigations, preparing technical documentation and drawing up a declaration of conformity1. In cases where the device includes sterile products or measuring functions, approval from a UK Approved Body may still be necessary3. Devices in this category include thermometers, stethoscopes, bandages and surgical masks.

Class IIa & IIb devices

Class IIa devices are generally regarded as medium risk devices and pose moderate risk of illness and injury. Both class IIa and IIb devices must be declared as such by applying to a UK Approved Body and performing a conformity assessment3, 4. For class IIa and IIb devices, there are several assessments. These include examining and testing the product or a homogenous batch of products, auditing the production quality assurance system, auditing the final inspection and testing or auditing the full quality assurance system3. include dental fillings, surgical clamps and tracheotomy tubes4 Class IIb devices include lung ventilators and bone fixation plates4.

Class III devices

These are considered high risk devices and pose substantial risk of illness and injury. Devices in this category are essential for sustaining human life and Due to the high-risk associated with class III devices, they are subject to the strictest regulations. In addition to the class IIa and IIb assessments, class III devices require a design dossier examination3. include pacemakers, ventilators, drug-coated stents and spinal disc cages.

Risk Categories for In Vitro Diagnostics in detail

These include but are not limited to reagents, instruments, software and systems intended for in vitro examination of specimens such as tissue donations and blood4. Most IVDs do not require intervention from a UK Approved Body5. However, for IVDs that are considered essential to health, involvement of a UK Approved Body is necessary5. The specific conformity assessment procedure depends on the category of IVD concerned5.

General IVDs

These are considered a low risk to patients and include clinical chemistry analysers, specimen receptacles and prepared selective culture media4. For general IVDs, involvement from a UK Approved Body is not required5. Instead, relevant provisions in the UK MDR 2002 must be met and self-declared prior to adding a UKCA mark to the device5,6.

IVDs for self-testing

These represent a low-to-medium risk to patients and include pregnancy self-testing, urine test strips and cholesterol self-testing4. In addition to conforming to requirements for general IVDs, applications for IVDs involved in self-testing must be sent to a UK Approved Body5. This enables examination of the design of the device, such as how suitable it is for non-professional users5.

IVDs stated in Part IV of the UK MDR 2002, Annex II List B

These represent medium-to-high risk to patients and include blood glucose self-testing, PSA screening and HLA typing4. Applications for devices in this category must be sent to a UK Approved Body5. This can enable auditing of technical documentation and the quality management system6.

IVDs stated in Part IV of the UK MDR 2002, Annex II List A.

These represent the highest risk to patients and include Hepatitis B blood-donor screening, ABO blood grouping and HIV blood diagnostic tests4. Due to the high risk associated with IVDs in this category, applications for devices in this category must be sent to a UK Approved Body5. By doing so, an audit of the quality management system can be performed as well as a design dossier review6. In addition, the UK Approved Body must verify each product or batch of products prior to being placed on the market5,6.

Proposed updates to medical device categories in the UK

Due to the quickly evolving state of medical technology, many items that did not previously count as a medical device, such as software and AI, are now needing to be considered as such. New proposals have been put forward as potential amendments to the existing regulations and risk classifications to accommodate newer technologies and devices. Among other proposed changes the following list of novel devices has been recommended for upgrade to the classification of highest risk Class III.

  • Active implantable medical devices and their accessories
  • in vitro fertilisation (IVF) and Assisted reproduction technologies (ART)
  • Surgical meshes
  • total or partial joint replacements
  • spinal disc replacements and other medical devices that come into contact with the spinal column
  • medical devices containing nano-materials
  • medical devices containing substances that will be introduced to the human body by one of various methods of absorption in order to achieve their intended function.
  • Active therapeutic devices with an integrated diagnostic function determining patient management such as closed loop or automated systems.

With the shift to a higher risk classification will come increased demand of clinical evidence and clinical testing, including clinical trials, in order for these devices to meet regulatory approval and reach the market. While an increased burden for the manufacturer this will be to the benefit patient safety and satisfaction for the end users. A full list of the proposed changes, including those outside of Class III, can be found here: Chapter 2: Classification – GOV.UK (www.gov.uk)

Medical devices are incredibly heterogenous, ranging from therapeutics and surgical tools to diagnostics and medical imaging software including machine learning and AI. Accordingly, medical device research and development often requires an interdisciplinary approach. During R&D, it is important to consider for whom the device is intended, how it will be used, and under what circumstances. Similarly, it is crucial to understand the risk status of the device. By considering these attributes, the device can be successfully assessed through the appropriate regulatory approval pathway.

References

Factsheet: medical devices overview – GOV.UK (www.gov.uk)

[1] https://www.gov.uk/government/collections/guidance-on-class-1-medical-devices

[2] https://www.gov.uk/guidance/medical-devices-how-to-comply-with-the-legal-requirements

[3] https://www.gov.uk/guidance/medical-devices-conformity-assessment-and-the-ukca-mark

[4] https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/640404/MDR_IVDR_guidance_Print_13.pdf[5] https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/946260/IVDD_legislation_guidance_-_PDF.pdf

[5] https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/946260/IVDD_legislation_guidance_-_PDF.pdf

[6] https://www.bsigroup.com/meddev/LocalFiles/en-IN/Technologies/BSI-md-ivd-diagnostic-directive-guide-brochure-UK-EN.pdf

Regulation of Connected Medical Devices and IOmT

Collection and transmission of personal biologic and health information via IOmT connected medical devices requires regulatory oversight and has cybersecurity implications.

Connected medical devices (CMDs) can produce and transmitting patient data, allowing their condition to be monitored by healthcare professionals. They are often used in decentralised clinical trials (DCTs) outside of the clinical trial site, allowing for participants who wouldn’t usually be able to attend. CMDs have led to the Internet of Medical things, a connected network of systems and which produce, transmit and analyse patient data.

CMDs and IoMT have countless applications in the healthcare and medical technology (Medtech) industries, however these devices are susceptible to cyber-attacks and data leaks. These attacks include stealing and selling private patient data to third parties, denial of service (DOS) attacks, and altering medical data which can lead to improper diagnoses and treatments.

It has been suggested by multiple authors that CMDs and other wearable activity trackers are prone to cyber-attack is that data security and privacy issues are often not considered during their development (1). Regulations for the development of CMDs in the UK fall under two categories: regulations concerning medical devices in general, and regulations concerning IoMT including data protection and cybersecurity. Medtech companies must follow both types of regulations if they wish to sell CMDs in the UK and abroad. Here we discuss the current regulations for CMDs in the UK, how they may change in response to these security issues, and how this will impact clinical trials and the approval of CMDs.

Current Device Regulations

Regulations for medical devices in the UK need to be updated to better cover the risks associated with CMDs, as many of these devices can enter the UK market with little-to-no regulatory approval especially in terms of data security. Manufacturers currently need only a Conformité Européenne (CE) mark to be sold in the EU (1). With CE marking, devices are classified according to risk from lowest (Class I) to highest (Class III), with class I devices allowed to enter the market without prior data regarding their safety in the US, EU and Japan. Devices placed in class IIb or III must carry out an audit of the whole quality assurance system or undergo an “Annex III” examination which can include examination of each product/batch, audit of the final inspection, or an audit of the production quality assurance system (2). Clinical trials to evaluate the conformity of CMDs to medical device regulations will have at least one of the following aims: (a) to verify that under normal usage, the device achieves the performance intended by the manufacturer, (b) to establish its clinical benefit as specified by the manufacturer, and (c) to establish its clinical safety (3). Many wearable devices e.g. smartwatches and activity trackers can skip regulatory approval as they aren’t currently classed as CMDs, however to be utilised in DCTs, they will need to be approved as medical devices (4).

In the UK and EU, the General Data Protection Regulation (GDPR) covers the use of medical data, as well as the Data Protection Act 2018 (DPA) in the UK as of 1 January 2021 (5). These regulations prohibit the disclosure of private data to third parties without the patient’s consent and can only be used without consent in the case of direct care and healthcare quality improvement projects. On the 24th of November 2021, the UK government issued the Product Security and Telecommunications Infrastructure (PSTI) Bill to place increased cybersecurity standards on technology companies (6). Requirements of PSTI include banning default and weak passwords, investigation of compliance failures and being transparent about fixes to security issues, with hefty fines in place if these rules aren’t followed. These regulations will force Medtech companies to constantly update devices and software found to be at risk of cyber-attack, as well as keeping the public informed on the updates. In addition, NHS-contracted organisations need to follow the NHS Code of Confidentiality and Code of Practice (5). Medtech companies hoping to sell in the UK should ensure their device meets these NHS requirements, and the NHS Data Security and Protection Toolkit 2021 states that healthcare organisations must keep an inventory of CMDs in their network (7). While these regulations prevent CMD developers from directly releasing data to third parties, they will not prevent cyber-attacks.

On the 26th of June 2022, the UK Government had a press release in which they discussed future regulatory changes regarding CMDs and data security (8). As of the 30th of June 2023, CMDs will need to carry a UK Conformity Assessed (UKCA) marking to be sold in the UK instead of the current CE markings. The UKCA marking is not recognised by the EU market as it only complies to the UK Supply of Machinery (Safety) Regulations 2008 (9), meaning Medtech companies hoping to enter both markets will need to follow the regulations of both markings. In addition, the government intends to introduce pre-market regulations similar to the EU MDR General Safety and Performance Requirement (GSPR) 17.4 regarding cyber security for medical devices. Following this regulation, hardware, IT networks and security measures must meet minimum requirements including protection against unauthorised access needed to allow the software to run efficiently (10).

Potential future intersection between regulations for cybersecurity & medical devices.

Where regulation may fall short of innovation in the changing landscape and possible solutions

Currently, medical device regulations such as the Conformité Européenne (CE) and UKCA markings don’t intersect with cybersecurity and data protection regulations, meaning CMDs can currently be sold in the UK despite being susceptible to data leaks. There is no evidence to suggest that this will change soon, however possible future rules to combine these types of regulation may include classing data security as a component of patient safety in clinical trials. In addition, pre-market trials of CMD cybersecurity could be performed using simulated malware to test for vulnerabilities in CMDs, including software and AI networks (1). These regulations will force Medtech companies to consider the cybersecurity of their devices more strongly during the design and production stages of development, preventing cyber-attacks instead of retroactive changes following data leaks.

CMDs have revolutionised modern healthcare, however IoMT is still in its infancy and cybersecurity risks and subsequent regulatory changes are to be expected. These changes will likely stall the development and sale of CMDs due to increased care during development and stricter pre-market trials, however regulations are necessary to ensure patient data remains private for the safety and security of the public.

References:

1)     Hernández-Álvarez L, Bullón Pérez JJ, Batista FK, Queiruga-Dios A. Security Threats and Cryptographic Protocols for Medical Wearables. Mathematics. 2022 Mar 10;10(6):886. – Available from: https://doi.org/10.3390/math10060886

2)     CE Marking – Medical Devices Class III [Internet] 2021 – Available from: http://www.ce-marking.com/medical-devices-class-iii.html

3)     Reuschlaw – Need for clinical trials in accordance with the MDR [Internet] 2021 – Available from: https://www.reuschlaw.de/en/news/need-for-clinical-trials-in-accordance-with-the-mdr/

4)     Sato T, Ishimaru H, Takata T, Sasaki H, Shikano M. Application of Internet of Medical/Health Things to Decentralized Clinical Trials: Development Status and Regulatory Considerations. Frontiers in Medicine. 2022;9. doi: 10.3389/fmed.2022.903188

5)     TaylorWessing – Medical devices in the UK – the data protection angle [Internet] 2020 – Available from: https://globaldatahub.taylorwessing.com/article/medical-devices-in-the-uk-the-data-protection-angle

6)     Info Security Magazine – UK Introduces New Cybersecurity Legislation for IoT Devices [Internet] 2021 – Available from: https://www.infosecurity-magazine.com/news/uk-cybersecurity-legislation-iot/

7)     Core to Cloud – New mandatory cybersecurity requirements for medical devices [Internet] 2021 – Available from: https://www.coretocloud.co.uk/new-mandatory-cybersecurity-requirements-for-medical-devices/

8)     UK Government press release – UK to strengthen regulation of medical devices to protect patients [Internet] 2022 – Available from: https://www.gov.uk/government/news/uk-to-strengthen-regulation-of-medical-devices-to-protect-patients

9)     Make UK – CE Marking vs UKCA Marking – What does it mean? [Internet] 2020 – Available from: https://www.makeuk.org/insights/blogs/ce-marking-vs-ukca-marking

10)  EU Medical Device Regulation – ANNEX I – General safety and performance requirements [Internet] 2019 – Available from: https://www.medical-device-regulation.eu/2019/07/23/annex-i-general-safety-and-performance-requirements/

Cybersecurity Considerations for Connected Medical Devices and the “Internet of Medical Things”

Cybersecurity for IOmT connected medical devices.

Advancements in technology of the past few decades has led to the development of devices capable of connecting to one another via networks such as Wi-Fi and Bluetooth, allowing them to create, transmit and receive data between one another. Medical technology (Medtech) companies have utilised these features to develop connected medical devices. These devices can transmit patient data such as heart rate, blood glucose levels and sleep patterns, which can be monitored by healthcare professionals and clinical trials companies, allowing for accurate remote oversight of a patient’s condition for quick and accurate diagnoses and treatment from anywhere.

The existence of connected medical devices has led to the Internet of Medical Things (IoMT), the connected network of health systems and services able to produce, transmit and analyse clinical data, which is changing the shape of healthcare and clinical trials globally.

Despite the clear potential of IoMTs in the healthcare system, there are several factors affecting the development of connected medical devices and their uptake by the public. Worries regarding the security of their private clinical data in the light of cybersecurity attacks over the past decade, and subsequent data protection regulations put in place to prevent further leaks and their potential impact on future innovations in the medtech industry.

Connected Medical Devices and the Internet of Medical Things (IoMT)

There are over 500,000 connected medical devices (CMDs) currently on the market (1), which can be split into three key groups; stationary medical devices typically found in hospitals such as CT and MRI scanners, implanted medical devices such as pacemakers and defibrillators to monitor a patient’s condition more closely, and wearable medical devices such as smartwatches that track patient activity and insulin pumps (1). Many technology companies, including those which wouldn’t be classified as Medtech (Apple, Nike, Huawei) produce smart devices which produce data surrounding user activity such as exercise, heart rate and quality of sleep. In November 2021, the FDA authorised the first prescription-use VR system for chronic lower back pain, further highlighting the increasing opportunities for CMDs in healthcare (2). Artificial intelligence (AI) and machine learning (ML) algorithms can also be classed under CMDs, capable of automated learning using neural networks to search and analyse data much faster (3). These AI are commonly used to search for novel patterns in data, diagnoses and predicting outcomes, and optimising patient treatments and are commonly used in clinical trials (3).

These devices, the data they produce and the development of software capable of compiling and analysing this data has led to the creation of the Internet of Medical Things (IoMT), which has the potential to revolutionise healthcare (1). IoMT allows healthcare professionals to monitor patients in real time from anywhere, increasing the speed and accuracy of diagnoses and treatment. General uptake of IoMT in healthcare may improve disease and drug management, leading to better patient outcomes and decreased costs to healthcare providers.

Medical Devices and Clinical Trials

CMDs have allowed for hybrid and decentralised clinical trials (DCTs), in which trials take place remotely from patient’s homes and during their daily lives instead of on a trial site. The prevalence of DCTs have increased significantly since the start of the COVID-19 pandemic, in which patient access to clinical trials was reduced by 80% and monthly trial starts decreased by 50% (4).

DCTs allow patients to take part who would usually be unable to participate due to geographical or time limitations, while reducing time spent on-site. According to a study by CISCRP, 60% of patients see the location and time spent in a clinical site as important factors when considering clinical trials (5). CMDs can include telemedicines, smart phone apps and AI capable of analysing patient data. As a result of this, there has been ~34% annual compound growth of CMD use in clinical trials (6).These benefits are best portrayed by the significant growth in the IoMT market, which is expected to grow from ~$31 billion in 2021 to a predicted ~$188 billion in 2028 (7), with CMDs and wearable smart devices increasingly used in the home as well as healthcare institutions.

Cybersecurity Issues

Despite the advantages of the IoMT, the adoption of CMDs is hampered by concerns regarding the security of clinical data stored in the cloud, instead of traditional medical records stored on paper or in internal servers which are less susceptible to being leaked. IoMT devices are vulnerable to many types of attack which can interfere with patient monitoring and care. Examples of these include eavesdropping, in which an attacker gains access to private medical records which can then be used to unlock the CMD, gaining further access to unauthorised data and allowing them to tamper with private medical records (8). While the common aim of these attacks is to sell this data to a third party, attacks on IoMT devices could include changing medical data leading to improper diagnoses of patients, the prescription of medication leading to an allergic response, and inaccurate monitoring of medical conditions which would impact patient welfare and have potentially significant financial impacts (8).

There have been many instances of attacks on large technology companies in recent years. Fitbit, one of the largest producers of wearable activity tracking watches, has been revealed to be vulnerable to data leakage via network connection (9), and the Nike+ Fuelband is prone to attack due to its USB connector (10). Technology companies such as Huawei, Xiaomi and Jawbone have suffered data leaks (9).

These incidents have negatively impacted public trust in CMDs collecting medical data, with people typically not wishing to share medical information with non-NHS businesses for reasons other than direct care. While trust was shown to increase after a deliberative workshop, it remained low (<50%) (11). As shown here, public distrust towards CMDs amid cybersecurity scandals will halt the potential growth of IoMT and its applications in healthcare.

CMDs and IoMT provide a promising avenue for quick, efficient diagnoses and treatment of a variety of conditions and allow for DCTs which increases the number of willing participants and allows for remote accurate monitoring of conditions. However, cybersecurity issues halt the progress and uptake of CMDs due to public distrust and misuse of the technology by cyber attackers. Unfortunately, cybersecurity issues can typically only be addressed after the incident occurs, however updates to UK regulations regarding CMDs will help prevent future attacks and data leaks.

Cybersecurity breaches can have a variety of goals.

1)     Deloitte – Medtech and the Internet of Medical Things [Internet] 2018 – Available from: https://www2.deloitte.com/global/en/pages/life-sciences-and-healthcare/articles/medtech-internet-of-medical-things.html

2)     Sato T, Ishimaru H, Takata T, Sasaki H, Shikano M. Application of Internet of Medical/Health Things to Decentralized Clinical Trials: Development Status and Regulatory Considerations. Frontiers in Medicine. 2022;9. – Available from: https://doi.org/10.3389%2Ffmed.2022.903188

3)     Angus DC. Randomized clinical trials of artificial intelligence. Jama. 2020 Mar 17;323(11):1043-5. – Available from: doi:10.1001/jama.2020.1039

4)     McKinsey & Company – No place like home? Stepping up the decentralization of clinical trials [Internet] 2021 – Available from: https://www.mckinsey.com/industries/life-sciences/our-insights/no-place-like-home-stepping-up-the-decentralization-of-clinical-trials

5)     Anderson A, Borfitz D, Getz K. Global public attitudes about clinical research and patient experiences with clinical trials. JAMA Network Open. 2018 Oct 5;1(6):e182969-. Available from: doi:10.1001/jamanetworkopen.2018.2969

6)     Marra C, Chen JL, Coravos A, Stern AD. Quantifying the use of connected digital products in clinical research. NPJ digital medicine. 2020 Apr 3;3(1):1-5. – Available from: https://doi.org/10.1038/s41746-020-0259-x

7)     Fortune Business Insights – Internet of Medical Things (IoMT) Market [Internet] – Available from: https://www.fortunebusinessinsights.com/industry-reports/internet-of-medical-things-iomt-market-101844

8)     Hasan MK, Ghazal TM, Saeed RA, Pandey B, Gohel H, Eshmawi AA, Abdel‐Khalek S, Alkhassawneh HM. A review on security threats, vulnerabilities, and counter measures of 5G enabled Internet‐of‐Medical‐Things. IET Communications. 2022 Mar;16(5):421-32. – Available from: https://doi.org/10.1049/cmu2.12301

9)     Jiang D, Shi G. Research on data security and privacy protection of wearable equipment in healthcare. Journal of Healthcare Engineering. 2021 Feb 5;2021. – Available from: https://doi.org/10.1155/2021/6656204

10)  Arias O, Wurm J, Hoang K, Jin Y. Privacy and security in internet of things and wearable devices. IEEE Transactions on Multi-Scale Computing Systems. 2015 Nov 6;1(2):99-109. DOI: 10.1109/TMSCS.2015.2498605

11)  Chico V, Hunn A, Taylor M. Public views on sharing anonymised patient-level data where there is a mixed public and private benefit. NHS Health Research Authority, University of Sheffield School of Law. 2019 Sep. – Available from: https://s3.eu-west-2.amazonaws.com/www.hra.nhs.uk/media/documents/Sharing_anonymised_patient-level_data_where_there_is_a_mixed_public_and_privat_Pab71UW.pdf

The Role of Precision Medicine in Drug Development and Clinical Trials

With the help of precision medicine, or personalised medicine, modern medicine has moved away from a ‘one size fits all’ approach to treating disease and towards therapeutic approaches that are tailored to individuals and subgroups. These treatments are designed to be more efficacious due to targeting population subgroups based on their genetic or molecular nuances, rather than operating on the assumption that all bodies function and respond the same way and to the same degree to a given treatment. Molecular knowledge can now be utilised to tailor treatment to the patient at the correct dosage and time point, usually with the aid of pharmacogenomic approaches and molecular biomarkers.

Information about an individual’s genetic makeup, such as genetic variants that may influence treatment efficacy, toxicity, and adverse events can help to determine how patients will respond to a certain treatment. In addition to genomic, recent technological advances have led to the identification of many transcriptomic and proteomic biomarkers. This knowledge is useful in all stages of therapeutic development and can influence both the design of the therapeutic itself and of the clinical trial.


Drug Development

Inter-individual variations in drug response can result from polymorphisms in drug metabolizing enzymes. Thorough examination of gene expression and mutations in disease populations can lead to the identification of distinct disease subpopulations that share certain characteristics.  Further exploration of these genes and their interactions can uncover possible drug target genes for the treatment of a disease subpopulation.

Furthermore, an awareness of genetic variation in disease subpopulations means that the involved pathways and components can be more accurately recreated in pre-clinical studies. Bringing the gap between animal and human toxicity findings allows for more representative disease models. This allows variability in treatment response and optimal dosage to be explored more precisely.


Clinical Trial Design

Originally, clinical trials were designed to evaluate differences between novel treatments and standard treatments or controls, not among individual responses in treated groups. As a result, a therapeutic that was very effective in only a subgroup of the trial population may not have produced significant results and a therapeutic that caused adverse events in only a small subgroup could have been deemed too toxic for overall use.

The goal of clinical trials to gain regulatory approval remains unchanged. With the emergence of precision medicine come biomarker-driven trials that include patient subgroups in their design. Master protocols for trials enable the application of one treatment to multiple diseases, or multiple treatments to one disease, allowing a trial to adapt during its course. This room for adaptation can reduce financial impact due to ineffective treatments being abandoned earlier and targeting the most suitable groups. Incorporating a diagnostic assay in trial design can offer multiple advantages and prevent research from straying in the wrong direction.

Targeted therapies can be tested in the most appropriate patient groups likely to benefit by biomarker testing of patients prior to clinical trial participation. Screening patients for those more likely to respond well to treatment gives a greater estimate of treatment effect in the subgroup.  This increases the likelihood of demonstrating efficacy in a clinical trial. It also reduces the size of the sample population required to see statistically significant results, which can speed up the process.

Identifying responders before enrolment in such a manner minimises the number of exposed patients who would not benefit from treatment. Decreasing the risk of exposing non-responders to potential adverse events can improve the benefit/risk analysis.

Patient stratification is another aspect of trial design that utilises patient’s molecular biomarker profiles. Stratifying trial participants into subgroups can classify disease subtypes. Particularly in oncology, genomic approaches can guide the stratification of patients by their tumour mutations. It is notably useful in umbrella, basket, and platform trials and can reduce the financial impact by allowing for adaptive trials.

Umbrella trials test multiple targeted therapeutics in different biomarker cohorts of a single disease. Basket trials, on the other hand, test one or more targeted therapeutics in a patient cohort with matched biomarkers. Platform trials have a randomised structure and allow the evaluation of multiple targeted therapeutics in multiple biomarker-selected populations.


Application in developed therapeutics

While precision medicine approaches are most beneficial when included throughout the drug development process, their application can also improve or salvage existing treatments and prevent a clinical trial from failing. For example, a developed drug may cause severe adverse events in a small disease subpopulation.  Upon investigation it is found that the drug has a secondary target, which is only present in that subgroup.  With this knowledge, patients can be screened for presence/absence of the safety biomarker and intervention with said drug can be avoided in that subgroup while continued in the remaining population.

Alternatively, a drug may have clinically meaningful results in only a small number of patients. The responsive subgroup can be explored for potential biomarkers associated with degree of responsiveness to treatment. The clinical trial can then resume with a focus on patients likely to respond well to the therapeutic.


Response Monitoring

Throughout and after a clinical trial, biomarkers can be used as a means of observing patient response to intervention, and account for variability in response. Safety and efficacy monitoring markers will reveal individual cases where treatment is working effectively or needs to be halted due to adverse events. For example, a cancer-related gene mutation or protein detected in blood may no longer be present after successful treatment has been administered, showing that the treatment has worked.

Identified responders or non-responders can be further stratified into subgroups and studied.  Genomic information can aid in the understanding of outliers and changes to treatment response. This will contribute to disease and therapeutic understanding, so that the right patients can be given the right dose, getting the most benefit out of treatment.


Challenges of Precision Medicine

It should be noted that the development of a targeted therapy requires the right data, both for the identification of the drug target and suitable patients. Molecular data from disease populations in previous studies may not always be available during drug development. If available, it may not be the correct type of data or generated by the most appropriate assay. Developing a targeted therapy is not possible without suitable data to understand disease mechanisms and identify putative drug targets.

Biomarker-driven therapies require genetic tests and companion diagnostics to identify and distinguish suitable patients. Incorporating diagnostic methods in a clinical trial is an added cost and the process can be burdensome as it can make participant recruitment harder. Clinical intervention according to the results of stratification should also be well-defined before a trial phase commences.
 


References
Di Liello, R., Piccirillo, M., Arenare, L., Gargiulo, P., Schettino, C., Gravina, A. and Perrone, F., 2021. Master Protocols for Precision Medicine in Oncology: Overcoming Methodology of Randomized Clinical Trials. Life, 11(11), p.1253.
Dugger, S., Platt, A. and Goldstein, D., 2017. Drug development in the era of precision medicine. Nature Reviews Drug Discovery, 17(3), pp.183-196.
Mirsadeghi, S. and Larijani, B., 2017. Personalized Medicine: Pharmacogenomics and Drug Development. Acta Med Iran, 55(3), pp.150-165.
Woodcock, J., 2007. The Prospects for “Personalized Medicine” in Drug Development and Drug Therapy. Clinical Pharmacology & Therapeutics, 81(2), pp.164-169.
 

Mini Report: Why do clinical trials fail? 

Overview

Clinical studies are time-consuming, expensive, and frequently pose challenges for participants and sponsors alike. This article explores some of the numerous factors contributing the failure of a clinical study and offers suggestions on how to increase the likelihood of designing and carrying out effective clinical trials.

Pharmaceutical and medical device clinical trials present several chances for failure. Failures can occur due to lack of treatment effectiveness, problems with safety, or a failure to demonstrate either of these through use of the appropriate study design, as well as though budgetary constraint [1]. Other factors include not adhering to MHRA or FDA guidelines, failure to maintain acceptable study protocol, or issues with patient recruiting, enrolment, and retention. [2] In order to decide whether or not a clinical trial should continue, it is crucial to produce accurate and sufficient results and insights from data.

Briefly, some common reasons for failure at any phase include:

  1. The treatment is not effective: The treatment may not work as well as expected, or may not work for a significant portion of the study population.
  2. The treatment has unacceptable side effects: The treatment may have serious or unacceptable side effects that outweigh its potential benefits.
  3. The treatment is not better than existing treatments: The treatment may not be significantly better than existing treatments, or may not offer a clear advantage in terms of efficacy or convenience. If the study design is a non-inferiority or equivalence design this would not be a concern, thus it is important to choose the optimal study design for a specific context.
  4. The study design is flawed: The study may be poorly designed or executed, leading to inaccurate or misleading results.
  5. The study population is not representative: The study population may not accurately reflect the population for which the treatment is intended, leading to results that do not generalize to the wider population or to the study population of the subsequent phase of the clinical trial.

Overall, the high failure rate of clinical trials is a reflection of the complex and uncertain nature of medical research and the difficulty of developing new treatments that are both effective and safe.

One of the most expensive stages for a therapeutic clinical trial to fail in at the Phase III or phase IV stage. This is because so much investment of time, money and resources has already taken place across the previous clinical trial phases, as well as at the R&D and pre-clinical stages Unfortunately one of the most common phases for a clinical trial to fail is at phase III.

A 2016 review found that  46% of orphan-designated drugs were approved by regulatory agencies compared to 34% of non-orphan drugs (OR=2.3; 95% CI: 1.4-3.7); 27% of oncological agents were approved compared to 39% of agents targeted at other diseases (non-oncological) (OR=0.5; 95% CI: 0.3-0.9) and; 28% of agents sponsored by small and medium-size companies were approved vs 42% for those sponsored by larger firms (OR=0.4; 95% CI: 0.3-0.7). [5]

Earlier reviews of clinical trial results have found that on average there is a trend towards increased response rates in phase II trials than in the subsequent phase III trials of the same therapeutics. (zia et al, Clinical oncology 2005>. Further to this, 2019 study found a ratio of the hazard ratios for overall survival (OS) and progression free survival (PFS) of phase II compared to phase III trials of .074 indicating a significant reduction in observed hazard ratios in both the positive or negative direction (away from 1) for the phase III trials.[13]

Phase III clinical trials are the final stage of testing before a potential new treatment is submitted for approval to regulatory authorities. These trials are conducted on large groups of people, usually several hundred to several thousand, to confirm the effectiveness of the treatment, monitor side effects, and compare it to existing treatments. Despite careful planning and design, phase III clinical trials can fail for a variety of reasons.

Factors associated with patient eligibility: exclusion and inclusion criteria

The inclusion and exclusion criteria of a clinical study should ideally result in a population that proportionally matches the general patient population who experiences the disease, or would benefit from the therapeutic under investigation, to the extent that that is practical from a recruitment and safety stand-point.[4] Inclusion criteria may vary across studies or across sites of a single study and this has the potential impact the clarity of results. Having inclusion criteria that is too specific or narrow can also reduce the pool of eligible participants which typically translates to longer recruitment time horizons. On study found that 16% of protocol amendments are due to changes in inclusion or exclusion criteria,[5] this can lead to differences in the patient populations before and after the amendments.

Having a long list of exclusion criteria for a clinical trial can becomes problematic when it limits the resulting sample of patients too severely. It can artificially reduce the variance to the extent that the results of the trial are not generalisable. This creates problems at later trial phases and may have a negative impact on the chances of gaining regulatory approval. It could prevent the sponsor from acquiring timely knowledge regarding patient populations that are likely to be more reflective of real world use cases for the therapeutic under investigation.[1] In some cases this act of limitation in phase II of clinical development in particular, has been justified as a means of reducing variability;[2] and as a counter-point there is little published evidence to state universally increasing the diversity of the patient population with other criteria will inevitably increase the variability of the primary endpoint. This having narrower criteria could conceivably be justified in some contexts if carefully considered.[3]

Patient factors related to recruitment and retention

A failure to enrol a sufficient number of patients is a long standing problem with a UK study of 114 trials 10 indicated that only 31% met enrolment goals.[1] Bottlenecks or other problems with patient recruitment can hamper the success of clinical trials. If too many companies are using the same preferred trial site this can dilute to potential subject pool, and where the target disease is rare this can pose particular problems with recruiting.

Logistical constraints may also occur where patients who are ill cannot easily travel to designated hospitals. Some companies are trying to address this problem by potentially bringing the trial to people’s homes, however this could present further issues.[2] Some studies offer certain remunerations to patients to cover expenses in the hope that recruitment could be improved, however evidence suggests that paying patients to participate does not tend to generate better recruitment as it does scant to overcome the logistical challenges.[3] Although financial incentives did not result in better recruitment, it was reported that financial incentives did increase participant’s response to questionnaires for the trial.[4]

Biological effects of the drug do not translate across species or across patient populations

One review of clinical trial failure found that out of 640 phase III trials of novel therapeutics, 57% failed due to inability to demonstrate treatment efficacy[7]

Failure at phase I may result due to biological unsuitability of the therapeutic in humans, evidenced by the fact that successful animal studies do not translate into comparable effects in humans largely due do inter-species biomolecular differences. For example there may be unforeseen adverse events which make the therapeutic unsafe to consume even at an active dose. Failure at a later phase due to biological unsuitability may occur when the therapeutic does demonstrate a clinical effect against the disease but this effect does not translate into increased overall survival of the disease.

Toxicity or adverse events in the diseased population could be one explanation for this scenario. As phase I trials are typically conducted on healthy volunteers and phase II trials on less ill patients than Phase II trials, these patients may be more able to tolerate the therapeutic at a particular dose than those patients weakened by disease. Alternatively, this difference in study population could mean that the determined dose may be biologically active yet sub-optimal to combat the disease.

Changes in patient population between trial phases

If a successful phase II trial is conducted and supported by strong evidence and then the patient population is changed in some way for the phase III trial this may be another avenue to failure. This is a common occurrence as typically less ill patients will be chosen for a phase II study in determining therapeutic dose than in a phase III study, both for safety reasons and due to the uncertainty of whether the treatment will be shown to be sufficiently effective.

Accelerated Approvals

Certain therapeutics may receive accelerated approval to enable early patient access before phase III and IV trials have been successfully completed. It should be noted that these agents still need to complete the full clinical trials process in order to remain on the market. Therapeutics receiving accelerated approvals by the FDA at phase II, have unfortunately been found to have increased failure rates at subsequent phases. A 2018 study by Beaver et al, showed that 72% of the clinical studies receiving accelerated approval were single arm studies and almost 90% of studies receiving accelerated approval had response rate as the primary endpoint. [14] This points to an issue with study design whereby patient outcomes are being too easily over-estimated. The fact that 66% of accelerated approvals were granted to oncology therapeutics may go some way to explain these choices. However, given these figures and the obvious methodological problems manifest with accelerated approvals it isn’t difficult to explain the reduced approval rate observed for oncological agents compared to agents for other diseases (mentioned in a previous section of this article). In fact, single arm studies are not an advisable study design unless due to recruitment or other constraints there is absolutely no other alternative. Where possible a two armed randomised design should be prioritised.

Hacking bias

Phase II trials in which the analysis has relaxed the alpha cut-of from a p value of 0.05 to 0.01, thus deeming a trial successful (statistically significant) based on essentially insufficient evidence, can also lead to failure in the subsequent phase. Hacking bias can also occur if sub-group analyses were conducted posthumously to salvage the results of an unpromising trial result. This practice can be legitimate but if similar sub-groups are not considered at later phase trials then any positive findings may not be replicated. This is a good example of where it might be safer to cut losses at an earlier phase rather than forcing a result only to invest more time and budget chasing what will ultimately be considered an ineffective treatment. [16]

Focusing on the wrong endpoints is another mishap which could come under the umbrella of hacking bias or equally be reflective of poor study design. A common example of this is where response rate (RR) is chosen as a primary endpoint for a phase II trial in situations where response rate does not translate well to overall survival (OS) or even progression free survival (PFS), which may be the more informative outcome. This choice could lead to a therapeutic being deemed effective at phase II on the basis of RR then falling short in subsequent phases.

Optimism bias and regression to the mean

Single arm studies and patient differences aside, there are scenarios where a randomised phase II trial with adequate sample size and no real changes to the patient population in phase III is still not able to reproduce positive results. This can be explained on a statistical basis by optimism bias and regression to the mean. [16]

Clinical studies only progress to the next phase of a clinical trial if the previous phase study was found to be successful. This translates to statistically significant for superiority designs or non-significant in the case of non-inferiority and equivalence designs. For example, Phase III studies only result from successful phase II studies. Phase II studies that were not statistically significant, in other words where the results were neutral or negative, are ignored in the decision to progress to the next phase. This means that false positives are potentially forming the evidence base for progression to the net phase leading to a biased expectation of success at the next phase. The result of this is overly-optimistic expectations related to variability or treatment delta and effect size that may lead to underestimation of the sample size required for the subsequent trial and generally inadequate study design to fairly evaluate the therapeutic at the next phase. It may also lead to an ineffective treatment being pursued when it shouldn’t be.

There may be many reasons for a study to fail before being repeated successfully, such as due to an adjustment in patient sample or other methodological changes. Regardless of whether these reasons seem justified it is important not to abandon the unsuccessful results in calculating expectations for the study design of the subsequent phase. If an average was taken of the failures and successes at a particular phase, a more conservative and potentially less optimistic expectation of future therapeutic performance and outcomes at the subsequent phase would result – the by-product of this being a better designed next-phase trial.

Exaggeration Bias

A 2021 study based on the results of nearly 24,000 clinical trials observed a tendency towards the observed treatment effects of the clinical trial being larger than the true treatment effects and puts forward the idea of an exaggeration ratio. As a result of regression to the mean the ratio of the observed treatment effect to the true treatment effect tends to demonstrate a higher observed treatment effect than the true effect on average. This exaggerated expectation means that when clinical studies go on to the next phase it can be harder to replicate the results. At the 0.05 alpha level the paper estimated demonstrated a 25% chance that the observed effect is >3x larger than the true effect of the treatment; 50% chance that the observed effect is >1.5x larger than the true effect; and a 25% chance that the observed effect more or less approximated the true value. While based on a large but limited subsection of all clinical trials, the authors of this paper suggest taking this exaggeration ratio into account and adjusting for it in any subsequent clinical study design such as in the next phase of a study. [15]

Study design

A poor study design can lead to trial failures, for instance selecting the wrong patients or the wrong endpoint, not to mention bad data, can lead to problems in the trial.[1] However data sources can help sponsors be sure that the right patients are then recruited as well as choosing the proper and correct sites and countries to enhance the likely hood of success.

Another common cause of failure in clinical research is based on not being able to meet criteria that have been predetermined by the MHRA or FDA. As well as this, it is important to recognise that a sponsor is necessary to move a drug or device forward in the clinical trial process. If studies are rushed into phase III after a successful phase II it could lack time for reflection on how to address safety in phase III. [2]

Data-related biases

Problems in data collection, missing data, attrition bias may that are not sufficiently accounted for may also lead to unexpected failure, as may rater-bias and unintentional un-blinding. Factors such as non-compliance to the treatment protocol, whether for site-based logistic reasons or for reasons resulting from the individual disease state, in certain patient populations can often be a factor influence results.

Financial concerns

One review found that 22% of phase III studies failed due to a lack of funding[3]. This financial burden also leads to ethical issues regarding the patients that are involved in the trial, patients are under the impression that their involvement would lead to the advancement of the trial and its successful completion.[4] Therefore underfunded trials are likely to lack the enrolment needed to demonstrate efficacy.

Financial risks occur at all stages of product development, however the costs associated with having to re-do studies or to delay studies will escalate the cost further. Taking steps to identify and address risks early on in the development process is key. Companies that do not carefully monitor for risks sometimes don’t identify problems until much further down the line when it is difficult to address them cost-effectively.[5] Sometimes this comes from the hesitation of companies to terminate a project prematurely. In a study of 842 molecules and 637 development program failures, it was evident that the companies that took time to identify problems early on and stop development on an imperilled trial, had a much higher likelihood of reaching the market with their drug.[6]

Other factors that can result in trial failure include; misspent funding, lack of a correct design study, insufficient funding designated to the trial from the outset, which implies costing may not have been accurately calculated.[7] Patient dropout rates also effect the financial stability of clinical trials and difficulties with treatment adherence such as side effects, or a lack of follow ups will also contribute to the financial impact.

Delays and unforeseen costs & challenges

With patient recruiting, there are additional expenses that might be challenging to predict and become very changeable. It is clear that marketing tactics like advertising can have a significant impact on a trial’s capacity to make a profit. [5] Additionally, healthcare professionals can have a big impact on patient recruitment. For example, recruitment and retention may suffer if staff members are absent or appear to be absent, or if there is a regular turnover of staff members and no rapport can form between them and the patients. Building this trust and communication may result in increased participation.

The experiment is impacted by all of these patient recruiting issues, some of which result in significant delays. Only 6% of clinical trials are finished before the deadline, and another 80% are at least a month behind schedule. [6] These delays increase the possibility for loss and have an impact on study costs as well as subsequent sales. Since costs are considerable, there is a risk of financial loss; consequently, increasing the rate of recruitment and retention will yield enormous benefits. [7]

Ethical issues increase the risk of trial failure, severely damaging the reputation of all parties involved, i.e. the pharmaceutical or medical-device company, the CRO and the associated physicians.[1] Many industry cases illustrate that a pursuit of short-term gains can rapidly turn into long-term losses.[2] The general problems with the ethics of clinical trials come from the fact that participants bear the risk and burden. Participation in a clinical trial has an increased level of risk; this is because of the exposure to effects of new treatment. These risks however are not “offset by a prospective clinical benefit”[3], this is because the goal of the trial is not to treat trial participants but to produce generalised medical knowledge.

Take-aways from a statistical perspective

  • Avoid single arm studies where possible or at least consider the possible trade-offs.
  • Be careful about switching to more promising endpoints, alpha levels or sub-groups as a short-term tactic to get a study over the line.
  • Be cognisant that changing the study population from one phase to the next may alter the study outcomes, and not always in the desired direction.
  • Consider “Exaggeration Bias” and adjust for it when designing the next phase of your clinical trial.

References:

[1] Saberwal, Gayatri. “Biobusiness in Brief: What Ails Clinical Trials?” Current Science, vol. 115, no. 9, Current Science Association, 2018, pp. 1648–52, https://www.jstor.org/stable/26978474.

[2] Pharmafile “clinical trials and their patient”  https://www.pharmafile.com/news/511225/clinical-trials-and-their-patients-rising-costs-and-how-stem-loss (2016)

[3] Chris Plaford “why do most clinical trials fail” https://www.clinicalleader.com/doc/why-do-most-clinical-trials-fail-0001#_ftn1 (2015)

 [4]  Hwang T.J., Carpenter D., Lauffenburger J.C., Wang B., Franklin J.M., Kesselheim A.S. Failure of investigational drugs in late-stage clinical development and publication of trial results. JAMA Intern. Med. 2016;176:1826–1833.

 [5] Tukey, John W. “Use of Many Covariates in Clinical Trials.” International Statistical Review / Revue Internationale de Statistique, vol. 59, no. 2, [Wiley, International Statistical Institute (ISI)], 1991, pp. 123–37, https://www.jstor.org/stable/1403439?origin=crossref.

[6] Worrall, John. “<em>What</Em> Evidence in Evidence‐Based Medicine?” Philosophy of Science, vol. 69, no. S3, [The University of Chicago Press, Philosophy of Science Association], 2002, pp. S316–30, https://doi.org/10.1086/341855.

[7] Altman, Douglas G. “Size Of Clinical Trials.” British Medical Journal (Clinical Research Edition), vol. 286, no. 6381, BMJ, 1983, pp. 1842–43, https://www.jstor.org/stable/29511193.

[8] Fogel DB. Factors associated with clinical trials that fail and opportunities for improving the likelihood of success: A review. Contemp Clin Trials Commun. 2018;11:156-164. Published 2018 Aug 7. doi:10.1016/j.conctc.2018.08.001

[9] Jansen, Lynn A. “The Problem with Optimism in Clinical Trials.” IRB: Ethics & Human Research, vol. 28, no. 4, Hastings Center, 2006, pp. 13–19, https://www.jstor.org/stable/30033204.

[10] Chris Plaford “why do most clinical trials fail” https://www.clinicalleader.com/doc/why-do-most-clinical-trials-fail-0001#_ftn1 (2015)

[11] Kobak Kenneth “why do clinical trials fail? Journal of Clinical Psychopharmacology: February 2007 – Volume 27 – Issue 1 – p 1-5 doi: 10.1097/JCP.0b013e31802eb4b7

[12] Liang et al. European Journal of Cancer 2019; 121:19-28

[13] Tap et al, JAMA 2020; 323(13):1266-1276

[14] Beaver et al. JAMA Oncology 2018; 4:849-856

[15] van Zwet et al, Significance, December 2021; 16

[16] Michiels & Wason, European Journal of Cancer 2019; 123:116

Emerging use-cases for AI in clinical trials

Overview

Clinical trials are becoming more expensive where, according to a Deloitte’s report1, the average cost to get a drug to market in the USA was $1.188 billion in 2010, and $1.981 billion in 2019. This increase in cost reflects the difficulties that are associated with current linear clinical trial designs. Clinical trials can take a long time due to difficulty in finding suitable/eligible patients for each study and the growing amount of data available used to plan or inform a trial. Current clinical trials are still evolving to make use of the rapidly developing technologies, scientific methods, and data availability of recent years.

One possible way to improve and transform the clinical trial process as we know it would be through the implementation of Artificial Intelligence (AI). AI incorporates all intelligence demonstrated by machines and includes important aspects such as Machine Learning and Natural Language Processing. It is already widely used in modern technology, such as in smartphones and online website searches, but has also been used more recently to innovate the drug discovery process. AI implementation could be of benefit across diverse tasks is the planning, execution and analysis stages of clinical trials to improve cost effectiveness, study time, treatment efficacy, and quality of data2.

Many emerging developments in artificial intelligence (AI) have the potential to benefit the clinical trials landscape.

AI in Adaptive clinical trial design

Many clinical trials are designed linearly, however adaptive designs are being used to allow predetermined changes during a study in response to ongoing trial data. Adaptive designs provide the flexibility to optimise resource allocation, end an unproductive trial early, and better characterise a treatment’s efficacy and safety through multiple endpoints. AI can help to inform and optimise adaptive designs through the analysis of healthcare data to select optimal endpoints, determine and monitor parameters for early stopping, and identify appropriate protocols for the trial3. These study design changes can increase the efficiency of a study, resulting in a trial which is more cost and time effective while maintaining high quality data collection and analysis.

Meta-analysis

AI can also enhance the exploratory analysis of data from previous trials. AI-enabled technology can collect, organise, and analyse increasing amounts of data, which could be applied to data from previous trials. This is normally performed manually by a biostatistician as part of a meta-analysis, but AI could help to gather and perform initial analyses before a more in-depth statistical approach is taken. This could highlight potentially important patterns in collected evidence which would then be used in informing trial design.

Synthetic control arms

Current clinical trials typically compare an experimental treatment to placebo and established treatments, assigning enrolled patients to either a treatment or control group. Synthetic control arms are an AI-driven solution for having a control group in a single-arm trial, which usually have only a treatment group. Synthetic control arms use data from previous studies to simulate the control treatment in patients. This would allow for all patients in a trial to receive active treatment to provide more evidence for the treatment efficacy and safety at each clinical trial stage. This may also have an impact on patient enrolment as patients generally show less interest in enrolling on placebo-controlled trials4,5.

As synthetic control arms are relatively novel, comparisons should be made with traditional control groups. In a typical blinded trial, patients are unaware if they are receiving the experimental active treatment, or a placebo. This is to test that the new treatment is causative of any clinically meaningful response. Synthetic control arms could create more single-arm clinical trials, and the fact that patients are aware of receiving active treatment would be a factor given clinically meaningful results are found.

Site selection

Identification of suitable sites and investigators to perform a trial is an important factor in study efficiency and feasibility. A study site should be amply equipped to carry out a study, must be of a suitable size to process the needs of study participants, and be located in an accessible area to potential participants and investigators. Identification of target locations and investigators can be optimised through AI implementation, which also enables real-time monitoring of site performance once the trial has started. Study sites can be evaluated and compared through the development of a points-based algorithm which could factor in location, site size, and equipment.

Patient enrolment/recruitment

Most patients enrol on a clinical trial if they have not responded to existing treatments in a clinically meaningful way. However, there are often strict eligibility criteria required for patients to enrol for a trial, including diagnostic tests, biomarker profiles, and demographics. Currently, patients find out about clinical trials either through manually searching online databases or, on occasion, through a clinician’s recommendation. This puts a lot of responsibility on patients to search for potential trials, just to be faced with trying to understand eligibility criteria full of medical jargon. This can lead to low patient recruitment.

AI and natural language processing offer a potential solution for this issue. Natural language processing could be used to match patients with trials based on eligibility criteria and patient electronic health records. Potentially suitable trials could then be suggested to patients or their clinicians, making it easier for patients to find suitable trials and for trials to recruit patients. While this is an improvement to the current recruitment method, natural language processing may initially have some difficulty with clinical notes due to heavy use of acronyms, medical jargon, and deciphering of hand-written clinical notes. These problems aren’t specific to AI though, as currently patients looking for suitable trials have the same difficulties.

AI-driven patient recruitment could also be used to reduce population heterogeneity and use prognostic and predictive enrichment to increase study power. While reduced heterogeneity can be beneficial (e.g. in testing the safety of drugs for patients unable to enrol on early clinical trials), caution should be taken with limiting patient diversity. Selectively enrolling patients who are more likely to respond well to treatment initially could lead to advancing a treatment that may not be as wel tolerated in the post-market patient population.

AI balancing of diversity in clinical trials could be used to balance this. It is important to test safety and efficacy in with differing demographics for a treatment to be properly characterised. This may include different ethnic groups, body types (height, weight, BMI), ages, and sexes. For example, a drug intended for female patients should specify dosage programs and any specific adverse effects in female patients before approval.

Patient diagnostics

Trial data often uses diagnostic tests to measure a patient’s response to treatment. AI can be used to improve diagnostic accuracy and objectivity during trials, reduce the potential for bias, and help with blinding a trial. AI programs have already been shown to provide more accurate diagnoses compared to clinicians6, which could be extended to use in clinical trials. AI can also be used to integrate multiple biomarkers or large data sets (e.g. bioinformatics data) to better monitor and understand a patient’s response, and to make any needed changes in dosing.

Another application of AI in clinical trials would be in patient management. Wearable devices or apps could be used to provide real time safety and effectiveness data to both clinicians and patients, leading to better data quality and higher patient retention.

Patient monitoring, retainment, & medication adherence

AI could be used to monitor patients through automatic data capture and digital clinical assessments. Automated monitoring with AI can allow for personalised adherence alerts, and wearable devices could provide real time safety and efficacy data shared with both patient and clinician to increase retainment and adherence rates. Video consultations with clinicians could improve retainment by reducing travelling required from patients, but there would still be a risk of drop due to travel as some diagnostic tests would need to be done at an appropriate site, which may warrant additional costs if tests for research purposes are not covered by a patient’s health insurance or national health service.

Currently, medication adherence is mostly dependent on each patient’s diary/record keeping or memory, which is then discussed with clinicians during routine appointments. This can make it difficult to accurately track adherence. Digitising this through use of a website/app would allow for more accurate adherence data to be obtained, in addition to providing patients with notifications, educational content, and adherence records. Other medical devices such as timed medication bottles could also be used to ensure medication is used in appropriate intervals, and smart bottles could be used to synchronise this with a smartphone app if applicable.

Data Cleaning

Data cleaning for clinical trials is typically performed by trained biostatisticians and is essential to ensure that collected data is consistently formatted and free from inputting errors. However, the data cleaning process can be time-consuming, especially with large datasets collected during clinical trials. AI could be implemented through machine learning methods to identify and correct errors found in clinical trial datasets7, leading to better quality data which is optimised for analysis. An AI approach may also reduce the amount of time spent on data cleaning.

AI implementation and clinical trial digitisation

AI implementation is relevant in several aspects of clinical trials, be it in study design, patient diagnostics, or trial management. Several tech giants (including Apple & Google) have invested in developing solutions to process electronic health records, monitor patients remotely, and integrate healthcare data into devices. By improving the cost and time effectiveness of clinical trials, both patients and pharma-tech companies benefit with more affordably priced treatment costs for patients and greater return of investment for companies.

However, for clinical trials to implement AI successfully, many aspects of clinical trials would first require digitisation. Many trials still use paper documents instead of digital alternatives, which results in lost documents and slowed trial progression. Integrating electronic health records, digital copies of clinicians’ notes, and digital patient monitoring alone would help in designing and managing a trial. There is a concern that a switch to digital may be difficult for patients unfamiliar with technology, or for those who might prefer to keep paper diaries. However, digital solutions would allow for the development and implementation of AI-based solutions which would modernise and streamline the clinical trial process.

References

  1. Taylor K, Properzi F, Cruz M, Ronte H, Haughey J. Intelligent clinical trials [Internet]. www2.deloitte.com. 2020 [cited 16 March 2022]. Available from: https://www2.deloitte.com/content/dam/insights/us/articles/22934_intelligent-clinical-trials/DI_Intelligent-clinical-trials.pdf

  2. Glass L, Shorter G, Patil R. AI IN CLINICAL DEVELOPMENT [Internet]. IQVIA. 2019 [cited 16 March 2022]. Available from: https://www.iqvia.com/-/media/iqvia/pdfs/library/white-papers/ai-in-clinical-development.pdf
  • Bhatt A. Artificial intelligence in managing clinical trial design and conduct: Man and machine still on the learning curve?. Perspectives in Clinical Research. 2021;12(1):1-3. Available from: https://doi.org/10.4103/picr.PICR_312_20