With half of the world’s population lacking access to healthcare services, and 30% of the adult population in the US having inadequate health insurance coverage to get even basic access to services, it should have been clear that a pandemic like COVID-19 would strain the global healthcare system way over its maximum capacity. In this context, many are trying to embrace and encourage the use of telehealth as a way to provide safe and convenient access to care. However, telehealth in itself can not scale to cover all our needs unless we improve scalability and efficiency through AI and automation.
In this talk, we will describe how our work on combining latest AI advances with medical experts and online access has the potential to change the landscape in healthcare access and provide 24/7 quality healthcare. Combining areas such as NLP, vision, and automatic diagnosis we can augment and scale doctors. We will describe our work on combining expert systems with deep learning to build state-of-the-art medical diagnostic models that are also able to model the unknowns. We will also show our work on using language models for medical Q&A . More importantly, we will describe how those approaches have been used to address the urgent and immediate needs of the current pandemic.
3. ● >50% world with no access to
essential health services
○ ~30% of US adults under-insured
● ~15 min. to capture information,
diagnose, recommend treatment
● 30% of the medical errors causing
~400k deaths a year are due to
misdiagnosis
Healthcare access, quality, and scalability
shortage of 120,000 physicians by 2030
4. Towards an AI powered learning health system
● Mobile-First Care, always
on, accessible, affordable
● AI + human providers in
the loop for quality care
● Always-Learning system
● AI to operate in-the-wild
(EHR)
FEEDBACK
DATA
MODEL
AI-augmented
medical
conversations
9. Research areas at Curai
● Medical Reasoning and Diagnosis
○ Learning from the Experts: combining expert systems and deep
learning
● NLP
○ Medical dialogue summarization
○ Transfer Learning for Similar Questions
○ Medical entity recognition: An active learning approach
● Multimodal healthcare AI
○ Few-shot dermatology image classification
10. General principles
1. Extensible AI
a. Data feedback loops
b. Incrementally/iteratively learn from
“physician-in-the-loop”
2. Domain Priors + Data
3. AI in the wild
a. Uncertainty in prediction
b. Enables fall-back to
“physician-in-the-loop”
12. Local to global summarization
DOCTOR: what color is the phlegm?
PATIENT: dark green dark green
DOCTOR: do you have chest pain while coughing?
PATIENT: yes and breathing . pretty much all the time
DOCTOR: just clarifying. is your chest pain constant?
or is it on & off ?
PATIENT: pretty constant it is constant pain it does
not turn off.
Summaries
dark green phlegm
chest pain while coughing and breathing all the
time
constant chest pain
● Local structures in the conversation / Local summaries to global summaries
13. Designing the Model
● Hybrid model between abstractive and extractive summarization
○ Copy more (maintain integrity of domain)
○ Focus on medical concepts
○ Get negations and affirmations right
● Extension of the Pointer Generator Network (Abigail See et al. 2017)
○ Loss Contributions: Penalized generation to focus on copying
○ Architecture Contributions: Negations and medical concepts in attention mechanism and
final probability distributions.
○ Formulated new evaluation criteria: Automated and doctor metrics
14. Results
Copied from Doctor Copied from Patient Generated
Proposed ModelFine-tuned pointer
generator network
15. Using GPT-3 to Supplement Training Data
Snippet Trained on Human Trained on GPT-3 Trained on
Human + GPT-3
DR: Have you ever been tested for any underlying health conditions such as
diabetes, hypothyroidism or polycystic ovarian syndrome?
PT: No
PT: I have been told I have prediabetes
Has not been tested for
any underlying health
conditions.
Hasn't tested for any
underlying health
conditions such as
diabetes, hypothyroidism
or polycystic ovarian
syndrome
Has not been tested for
any underlying health
conditions. Has been told
has prediabetes.
DR: Do you have pus appearing discharge from the site?
PT: Yes. If the bubbles pop it leaks out a watery substance
Has pus appearing
from the sire.
Pus appearing from the site
Pus discharge from the
site. If bubbles pop it leaks
out a substance.
22. ML + Expert systems for Dx models
female
middle aged
fever
cough
Influenza 16.9
bacterial pneumonia 16.9
acute sinusitis 10.9
asthma 10.9
common cold 10.9
influenza 0.753
bacterial pneumonia 0.205
asthma 0.017
acute sinusitis 0.008
pulmonary tuberculosis 0.007
Inputs
DDx with expert system DDx with ML model
Expert
system
Clinical case
simulator
Clinical cases
DDx
ML
model
Common cold
UTI
Acute bronchitis
Female
Middle-aged
Chronic cough
Nasal congestion
Other data
(e.g. EHR)
23. COVID-aware modeling
Expert
system
Clinical case
simulator
Clinical cases with
DDx
ML
model
Common cold
UTI
Acute bronchitis
Female
Middle-aged
Chronic cough
Nasal congestion
COVID-19
assessment data
COVID-19
COVID-19
female
middle-age
cough
headache
nose discharge
cigarette smoking
hospital personnel
26. Question similarity for COVID FAQs
● Transfer learning
● Double-finetuned BERT model
● Handle data sparsity
● Medical domain knowledge through an intermediate QA binary task
Does A answer Q?
Q A
Pretrained
BERT
BERT
Q1 similar to Q2?
Q1 Q2
BERT
27. COVID-19: Automated FAQ Matching
●
●
User Questions
Matching Questions in our FAQ
(Answer not shown here for brevity)
When do COVID symptoms start after exposure? How long is it between when a person is exposed to
the virus and when they start showing symptoms?
Currently I’m experiencing a cough and slight
chest pain. Should I just stay at home? At what
point will I know I have to go to the ER?
When should you go to the emergency room?
27
29. Conclusions
● Healthcare needs to scale quickly, and this has become obvious in a global
pandemic like the one we are facing
● The only way to scale healthcare while improving quality and accessibility is
through technology and AI
● AI cannot be simply “dropped” in the middle of old workflows and processes
○ It needs to be integrated in end-to-end medical care benefitting both patients and providers
https://curai.com/work