Panel @ Collaborative Assistants for the Society (CASY 2020) https://casy.aiisc.ai/schedule

1. Collaborative Assistant/Virtual Assistant/
Conversational AI agents at AIISC
Panel @ Collaborative Assistants for the Society (CASY 2020)
Amit Sheth
Director, Artificial Intelligence Institute of UofSC (#AIISC, aiisc.ai)

2.  Focus on two domains: Health & Education
 Broad variety: Simple chatbot (flexible interactions) to quite complex (manage
chronic disease)
 Uniqueness of complex examples: a. Context with deep domain knowledge, b.
Personalization with personalized health knowledge graph, c. active/passive
sensing, d. multimodal (text, voice, image)
Examples – at various stages of development
 Health-e Gamecock COVID-10 daily symptom checker
 Pediatric Asthma
 Nutrition
 Mental Health
 Autism, After Cancer Exercise, Diabetes, and more in planning.
http://wiki.aiisc.ai/index.php/Covid19

3. 1. Self
Monitoring
2. Self
Appraisal
3. Self
Management
4.
Intervention
5. Disease
Progression
and
Tracking
Virtual Health Assistant for Augmented
Personalized Health: example of Asthma
• Complex- multifactorial disease, personalized
• Access- challenges in working with patients
(IRB, privacy, medical/clinical knowledge and
lack of gold standards, etc.

4. 4
Sensor, Social, Clinical Datastreams: Informed &
Intelligent Questions
Weather information
(temperature, pollen,
humidity, etc)
Elasticsearch (ES)
Database Query & Rule Abstract raw values
into information
Asthma Domain Knowledge
https://bioportal.bioontology.org/ontologies/AO
http://www.childhealthservicemodels.eu
Patient Data from EMR & PGHD
(Compliance score, prescribed
medications, asthma control level)
IoTs (Foobot & Fitbit)
Conversation Rules & Scripts
(DialogFlow)
Sensor,Social,Clinical Architectural Framework for Intelligent & Informed Conversations: kBOT Asthma
Sensor (IoTs) & Cyber
Datastream
Clinical (Baseline) Datastream
Patient Consented Social Data
(Facebook, Instagram, Twitter
Activity)
Social Datastream
Knowledge Datastream
★ Smarter & engaging agent
★ Minimize active sensing
(Questions to be asked)
★ Ask only informed & intelligent questions
★ Relevant & Contextualized conversations
★ Personalized & Human-Like
Human-Like Aspect

5. Contextualization and
Personalization
kBOT initiates greeting
conversation.
Understands the patient’s health
condition (allergic reaction to high
ragweed pollen level) via the
personalized patient’s knowledge
graph generated from EMR, PGHD, and
prior interactions with the kBOT.
Generates predictions or
recommended course of actions.
Inference based on patient’s historical
records and background health
knowledge graph containing
contextualized (domain-specific)
knowledge.
Figure: Example kBOT conversation which
utilizes background health knowledge graph
and patient’s knowledge graph to infer and
generate recommendation to patients.
★ Conversing only information relevant to
the patient
5

6. “Huge need in several fields – health and education among the important areas.
• Examples from health domain: not enough available clinical expertise to meet mental
health needs; chronic disease need continuous care; massive growth in patient generate
data with demonstrated value for making decisions on health, wellness and fitness.
But,
Quite challenging: big data challenges – esp variety of data, need for context to interpret
data (and relevant deep domain knowledge), personalization for each patient, need to be
able to explain results before clinicians will take the technology seriously, ease of use/UX,
incentives in validation studies (patients, clinicians, insurance companies), etc.