Research in Progress (RIP): "Data-Mining Electronic Health Records for Clinical Decision Support"

"Wisdom of the Crowd or Tyranny of the Mob? OrderRex: Data-Mining Electronic Health Records for Clinical Decision Support"

Please note: All research in progress seminars are off-the-record. Any information about methodology and/or results is embargoed until publication.

Background

Uncertainty and undesirable variability is pervasive in medical decision making.  Clinical decision support like order sets help distribute expertise, but are constrained by resource intensive manual development. 

Objective

To overcome scalability limitations by automatically generating decision support content from existing practice patterns, analogous to Amazon.com’s product recommender.  To perform the first structured validation of such a system against external standards-of-care and outcome predictions.

Methods

We extracted deidentified electronic health record data from all hospitalizations at Stanford Hospital in 2011 (>5.4M structured data items from >19K patients) to build a system with association statistics for 811 clinical orders (e.g., labs, imaging, medications) and clinical outcomes.  We manually reviewed the National Guideline Clearinghouse for diagnoses of chest pain, gastrointestinal hemorrhage, and pneumonia.  We compared system generated clinical orders against guideline referenced orders by receiver operating characteristic (ROC) analysis.  Human authored order sets provided real-world benchmarks.  We compared predicted vs. actual outcomes by ROC analysis for separate validation patients.

Results

System generated orders were overall consistent with guidelines (ROC AUC c-statistics 0.89, 0.95, 0.83) and improve upon statistical prevalence (0.76, 0.74, 0.73) and pre-existing order sets (0.81, 0.77, 0.73) (P<10^-30 in all cases).  Clinical outcome prediction ROC AUC c-statistics were 0.84 for 30 day mortality , 0.84 for 1 week ICU life support, 0.80 for 1 week discharge / length of stay, and 0.68 for 30 day readmission.

Conclusions

Automatically generated order suggestions can reproduce and even optimize manual constructs like order sets while remaining largely concordant with guidelines and avoiding inappropriate recommendations.  This has even more important implications for prevalent cases where well-defined guidelines and order sets do not exist.  The same methodology is predictive of clinical outcomes comparable to state-of-the-art prognosis models (e.g., APACHE II), pointing to opportunities to link suggestions against favorable outcomes.