(42) Machine Learning to Assess Stimulant Therapy Risk for Stimulant Use Disorder

Background & Introduction: Stimulant-related harms, including stimulant misuse, use disorder (StUD) and overdoses, have risen sharply over the past decade, with the most recent wave of the overdose fatalities fueled by a combination of synthetic opioids and stimulants. At the same time, the number of dispensed stimulant prescriptions has doubled. The surge in both stimulant-related harms and prescriptions has sparked speculations whether the use of could have contributed to these harms, reminiscent of early opioid prescribing patterns in the opioid epidemic. Although prescription stimulants are primarily used to treat attention deficit hyperactivity disorder in children and adolescents, more than one-half of stimulant medications are prescribed to adults, with a substantial proportion of treated adults reporting stimulant misuse. Given the potential pathway from stimulant therapy to stimulant-related harms, more research is urgently needed to understand factors contributing to the risk of harms among adults prescribed stimulants. The objective of this study was to apply machine learning (ML) modeling into real-world electronic health record (EHR) data to identify potential key risk factors for StUD and provide clinicians with evidence that can help inform decisions when considering stimulant treatment.

Methods: This retrospective study utilized EHR data from the TriNetX Research database. The study sample included adults (age≥18 years) who had their first stimulant prescription (“index date”) issued between January 2010 and December 2021 (study period), received ≥ 3 stimulant prescriptions within 90 consecutive days during the study period, and did not have StUD diagnoses prior to their index date. The outcome was a new StUD diagnosis (1=yes, 0=no) recorded after the index date. Baseline data on 36 demographic, clinical, and health utilization characteristics were analyzed as potential risk factors for StUD using grid-search-based ML methods: gradient boost, random forest, neural network, and lasso logistic regression. The sample was randomly split into training (80%) and testing (20%) datasets to evaluate the model performance using the Area Under the Receiver Operating Characteristic curve (AUROC) score. The SHapley Additive exPlanations (SHAP) values were calculated to identify key risk factors for StUD. Disparate impact and equal opportunity scores were calculated to quantify and minimize the model’s potential bias toward gender or race/ethnicity-based subgroups. Sensitivity analysis was conducted in the subgroup prescribed higher-intensity therapy (≥ 9 stimulant prescriptions issued within 12 consecutive months) to better understand the cumulative risk of prescribed stimulants on StUD.

Results: The study sample comprised 361,545 adults prescribed stimulant medications. Of those, 12,761 (3.5%) were subsequently diagnosed with StUD (StUD group), with the median time from the first stimulant prescription date to the StUD diagnosis of 12.0 months (mean=24.0 [standard deviation 28.3]). Overall, the StUD group participants tended to be female, younger, and non-Hispanic White, and were more likely to have preexisting diagnoses of medical or mental health conditions, or substance use disorders (other than StUD, which was a part of the exclusionary criteria).

A total of 1,481 models were constructed and evaluated during the grid-search ML process. The optimized random forest, lasso regression, and neural network models achieved good predictive performance (AUROC >0.80), and the gradient boost model performed best (AUROC 0.97 [95% confidence intervals, CI: 0.96-0.98]), including the most favorable F1 score (0.80, 95% CI: 0.77-0.81), which indicated a good balance in identifying higher-risk individuals while minimizing false positive and false negative cases. Therefore, the gradient boost model was selected as the final risk assessment model for the present analysis. The feature importance analysis identified the ”top 10” most influential risk factors for acquiring a new StUD diagnosis after the index date (i.e., date of the first stimulant prescription): stimulant therapy-related factors (stimulant prescription issued within 6 months prior to the StUD diagnosis - the strongest risk factor; average number of outpatient stimulant prescriptions during the study period; and younger age when the stimulant prescription was issued) and the number of outpatient visits during the study period, as well as the baseline reports of “current” cigarette smoking, the total number of comorbidities (calculated using the Elixhauser Comorbidity Index), presence of diagnoses of non-StUD drug use disorders (especially those involving cannabis and opioids), and bipolar disorder.

The disparate impact and equal opportunity analyses indicated the gradient boost model performed equitable across all gender or racial/ethnic subgroups, supporting the fairness of the model’s performance.

The sensitivity analysis within the subgroup of participants prescribed higher-intensity stimulant therapy yielded results consistent with those of the above primary analysis, suggesting the relationship between the risk factors and acquiring a new diagnosis of StUD was robust and stable, regardless of the intensity of stimulant treatment.

Conclusion & Discussion: Among adults prescribed stimulant medications, the ML-based analysis of EHR data identified several risk factors for a subsequent StUD diagnosis, such as cigarette smoking, history of other substance use disorders and selected mental health problems, and the extent of comorbidities in general, with the presence of prescriptions for stimulants within the prior 6 months before the StUD diagnosis found to be the strongest risk factor. This suggests a connection between stimulant therapy initiation and the likelihood of developing StUD, highlighting the need for close monitoring of patients treated with these medications, especially if they have other risk factors. The relationship between the new StUD diagnosis and preexisting substance use, mental health, and other comorbid disorders underscores the need for integrated, comprehensive assessment and care strategies. These results should be interpreted with caution because they may reflect associations rather than causation. This study showed that the gradient boost model can provide robust risk assessment, while minimizing potential biases, supporting the potential utility of this ML model, laying an analytical foundation for early identification of adults on stimulant therapy who are at increased risk for StUD.

References: 1. Volkow ND. Rising Stimulant Deaths Show that We Face More than Just an Opioid Crisis. November 12, 2020. Accessed May 10, 2022. https://nida.nih.gov/about-nida/noras-blog/2020/11/rising-stimulant-deaths-show-we-face-more-than-just-opioid-crisis
2. U.S. Food and Drug Administration. FDA updating warnings to improve safe use of prescription stimulants used to treat ADHD and other conditions. Published online May 11, 2023. Accessed August 6, 2024. https://www.fda.gov/drugs/drug-safety-and-availability/fda-updating-warnings-improve-safe-use-prescription-stimulants-used-treat-adhd-and-other-conditions
3. Brumbaugh S, Tuan WJ, Scott A, Latronica JR, Bone C. Trends in characteristics of the recipients of new prescription stimulants between years 2010 and 2020 in the United States: An observational cohort study. EClinicalMedicine. 2022;50:101524. doi:10.1016/j.eclinm.2022.101524
4. Friedman J, Shover CL. Charting the fourth wave: Geographic, temporal, race/ethnicity and demographic trends in polysubstance fentanyl overdose deaths in the United States, 2010-2021. Addict Abingdon Engl. 2023;118(12):2477-2485. doi:10.1111/add.16318
5. Kollins SH. ADHD, substance use disorders, and psychostimulant treatment: current literature and treatment guidelines. J Atten Disord. 2008;12(2):115-125. doi:10.1177/1087054707311654