Yaobin Ling, Muhammad Bilal Tariq, Kaichen Tang, Jaroslaw Aronowski, Yang Fann, Sean I. Savitz, Xiaoqian Jiang, Yejin Kim
Abstract
Randomized Clinical trials (RCT) suffer from a high failure rate which could be caused by heterogeneous responses to treatment. Despite many models being developed to estimate heterogeneous treatment effects (HTE), there remains a lack of interpretable methods to identify responsive subgroups. This work aims to develop a framework to identify subgroups based on treatment effects that prioritize model interpretability. The proposed framework leverages an ensemble uplift tree method to generate descriptive decision rules that separate samples given estimated responses to the treatment.
Subsequently, we select a complementary set of these decision rules and rank them using a sparse linear model. To address the trial’s limited sample size problem, we proposed a data augmentation strategy by borrowing control patients from external studies and generating synthetic data. We apply the proposed framework to a failed randomized clinical trial for investigating an intracerebral hemorrhage therapy plan.
Introduction
The success rate of clinical trials was estimated to be only 13.8%, [1], and an investigation of 640 Phase III trials found that around 57% of them failed due to inadequate efficacy. [2] The success rate is much lower for some diseases without disease-modifying therapies. For example, intracerebral hemorrhage (ICH) is a devastating form of stroke, with the highest mortality rate of all stroke subtypes and severe disability affecting ICH survivors. [3] Many efforts have been devoted to identifying effective therapies to help patients recover from the disease.
Materials and Methods
Study Overview
Based on Neyman-Rubin’s potential outcome framework, we developed an interpretable causal clustering method. Our model was based on the recursive partitioning and rule selection. To overcome the limited sample size to explore heterogeneity, we proposed a data augmentation strategy based on borrowing historical data and generating synthetic data. We applied our method to an ICH clinical trial and demonstrated its ability to derive responsive subgroups with clinical implications. In the following subsections, we will first introduce our causal clustering framework, and then go through the data analysis pipeline for the real-world ICH trial data.
Results
We reported the number of treated and control samples in each cohort. 200 samples were randomly drawn from ATACH2 as the test data. To address the potential confounding bias by pooling the data from two studies, we performed a 1:1 PSM. We reported the cohort size, SMD, and the AUC for distinguishing between treated and control patients before and after matching in Table 2. The average SMD between the confounders of the treatment and control arms was 0.0605 after matching, and the AUC to distinguish the treatment and control group decreased from 0.9183 to 0.6539 (Table 2), showing adequate balance between the treatment and control groups.
Discussion
In this study, we proposed a framework for automatically identifying responsive subgroups from real-world RCT data. We generated candidate rules using an ensemble of recursive partition algorithms and employed a regularized linear model for complementary rule selection. Given the limited sample size of the RCT, we embraced a data augmentation strategy that tapped into both external observational study data and synthetic data. The proposed approach amplifies our model’s efficacy in analyzing the RCT data and augments the statistical power. Additionally, we considered the potential confounding bias introduced by the external data by employing a matching strategy during the data augmentation process.
Conclusion
The proposed framework helps identify several responsive subgroups regarding HTE in a comprehensive decision rule format. By doing data augmentation with data from different resources, we improved the model’s performance in terms of Qini-coefficient compared with the model trained on the trial data only. The model of the best evaluation metric gives rules of good quality from a clinical perspective and coincides with many other studies’ findings of the therapy plan for intracerebral hemorrhage. This work provides a foundation for mining information regarding causal effects from failed trials which helps develop new trials and treatment plans.
Citation: Ling Y, Tariq MB, Tang K, Aronowski J, Fann Y, Savitz SI, et al. (2024) An interpretable framework to identify responsive subgroups from clinical trials regarding treatment effects: Application to treatment of intracerebral hemorrhage. PLOS Digit Health 3(5): e0000493. https://doi.org/10.1371/journal.pdig.0000493
Editor: Sulaf Assi, Reader in Forensic Intelligent Data Analysis, UNITED KINGDOM
Received: February 22, 2024; Accepted: March 26, 2024; Published: May 7, 2024
Copyright: This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Data Availability: We used the clinical trial ATACH 2 data in this study. The information of the trial is registered as NCT01176565. The data that support the findings of this study are available from the National Institutes of Neurological Disorders and Strokes, but restrictions apply to the availability of these data, which were used under license for the current study and so are not publicly available. Data are, however, available from the authors upon reasonable request and with permission from the National Institutes of Neurological Disorders and Strokes. To apply, contact https://www.ninds.nih.gov/contact-us.
Funding: YK is supported in part by UTHealth startup and the National Institute of Health (NIH) under award number R01AG082721, R01AG066749, and R01AG084637. XJ is CPRIT Scholar in Cancer Research (RR180012), and he was supported in part by Christopher Sarofim Family Professorship, UT Stars award, UTHealth startup, the National Institute of Health (NIH) under award number R01AG066749, R01LM013712, R01LM014520, R01AG082721, R01AG066749, U01AG079847, and the National Science Foundation (NSF) #2124789. YCF is supported by funding from the Intramural Research Program of the National Institute of Neurological Disorders and Stroke, National Institutes of Health, USA. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
