Dylan Phelan

Recognizing patient symptoms is fundamental to medicine, research, and public health. However, symptoms are often underreported in coded formats even though they are routinely documented in physician notes. Large language models (LLMs), noted for their generalizability, could help bridge this gap by mimicking the role of human expert chart reviewers for symptom identification. The primary objective of this multisite study was to measure the accurate identification of infectious respiratory… Show more

Recognizing patient symptoms is fundamental to medicine, research, and public health. However, symptoms are often underreported in coded formats even though they are routinely documented in physician notes. Large language models (LLMs), noted for their generalizability, could help bridge this gap by mimicking the role of human expert chart reviewers for symptom identification. The primary objective of this multisite study was to measure the accurate identification of infectious respiratory disease symptoms using LLMs instructed to follow chart review guidelines. The secondary objective was to evaluate LLM generalizability in multisite settings without the need for site-specific training, fine-tuning, or customization.

Four LLMs were evaluated: GPT-4, GPT-3.5, Llama2 70B, and Mixtral 8×7B. LLM prompts were instructed to take on the role of chart reviewers and follow symptom annotation guidelines when assessing physician notes. Generalizability of the most performant LLM was then measured in a validation corpus of 308 notes from 21 emergency departments in the Indiana Health Information Exchange. LLMs significantly outperformed an ICD-10–based method for respiratory symptom identification in emergency department electronic health records. GPT-4 demonstrated the highest accuracy and generalizability, suggesting that LLMs may augment or replace traditional approaches. LLMs can be instructed to mimic human chart reviewers with high accuracy. Future work should assess broader symptom types and health care settings. Show less

The 21st Century Cures Act Final Rule requires that certified electronic health records (EHRs) be able to export a patient's full set of electronic health information (EHI). This requirement becomes more powerful if EHI exports use interoperable application programming interfaces (APIs). We sought to advance the ecosystem, instantiating policy desiderata in a working reference implementation based on a consensus design.

We formulate a model for interoperable, patient-controlled… Show more

The 21st Century Cures Act Final Rule requires that certified electronic health records (EHRs) be able to export a patient's full set of electronic health information (EHI). This requirement becomes more powerful if EHI exports use interoperable application programming interfaces (APIs). We sought to advance the ecosystem, instantiating policy desiderata in a working reference implementation based on a consensus design.

We formulate a model for interoperable, patient-controlled, app-driven access to EHI exports in an open source reference implementation following the Argonaut FHIR Accelerator consensus implementation guide for EHI Export. The reference implementation, which asynchronously provides EHI across an API, has three central components: a web application for patients to request EHI exports, an EHI server to respond to requests, and an administrative export management web application to manage requests. It leverages mandated SMART on FHIR/Bulk FHIR APIs.

A patient-controlled app enabling full EHI export from any EHR across an API could facilitate national-scale patient-directed information exchange. We hope releasing these tools sparks engagement from the health IT community to evolve the design, implement and test in real-world settings, and develop patient-facing apps.

To advance regulatory innovation, we formulate a model that builds on existing requirements under the Cures Act Rule and takes a step toward an interoperable, scalable approach, simplifying patient access to their own health data; supporting the sharing of clinical data for both improved patient care and medical research; and encouraging the growth of an ecosystem of third-party applications. Show less