The corresponding video is at https://youtu.be/ztNHKLTHBrA AIISC conducts foundational and translational research in AI. In this talk, we review part of the AIISC's research in Social Good, Social Harm, and Public Health.
This talk was given to the UofSC College on Information and Communication.
Additional project details at http://wiki.aiisc.ai
Fintech app development company with financial management
AIISC’s research in Social Good/Social Harm, Public Health and Epidemiology
1. AIISC’s research in
Social Good/Social Harm,
Public Health and Epidemiology
A review with the colleagues in the
College of Information and Communications
Amit Sheth with relevant #AIISC team
10 Mar 2021
http://aiisc.ai
Amit Sheth
Ugur
Kursuncu
Kaushik
Roy
Manas
Gaur
Usha
Lokala
Thilini
Wijesiriwardene
3. Translational Research Areas of relevance to College of I & C
Social Harm
◎ Harassment on Social Media
◎ Toxic Content
◎ Extremism & Radicalization
◎ Disinformation
◎ Gender-based Violence
Social Good
◎ Crisis Management
Public Health & Epidemiology
◎ Mental Health/
Depression/Suicide
◎ Drug Abuse, Addiction
◎ Zika, COVID-19 epedemic
Personalized Health
◎ Asthma, Mental Health,
Hypertension, Diabetes,
3
4. Funded Multidisciplinary Projects [NIH, NSF, Other- past & current]
❖ Context-Aware Harassment Detection on Social Media
❖ eDarkTrends : Monitoring Cryptomarkets to Identify Emerging Trends of Illicit Synthetic
Opioids Use
❖ Social media analysis to monitor cannabis and synthetic cannabinoid use
❖ Community-Driven Data Engineering for Substance Abuse Prevention in the Rural
Midwest
❖ Modeling Social Behavior for Healthcare Utilization in Depression
❖ Hazards SEES: Social and Physical Sensing Enabled Decision Support for Disaster
Management and Response
❖ KHealth: Semantic Multisensory Mobile Approach to Personalized Asthma Care
❖ Project Safe Neighborhood: Westwood Partnership to Prevent Juvenile Repeat Offenders
❖ SoCS: Social Media Enhanced Organizational Sensemaking in Emergency Response
❖ PREDOSE: PREscription Drug abuse Online Surveillance and Epidemiology
4
5. Sample Proposals indicating strategic direction [P=pending]
❏ Developing the Framework for a Comprehensive Multi-modal mHealth Tool
designed to Assist Patients Suffering from Multiple Chronic Conditions [P]
❏ Narrative, Moral and Social Foundations of Social Cyber-Attack in Social Media
(involving Shannon Bowen- Mass Communication, Matthew Brashears- Sociology,
Mirta Galesic- Psychology, Fil Menczer, Brendan Nyhan-Government, Amit Sheth)
❏ Study of the Discripancies in Diagnosis and Treatment of Cardiovascular Disease
Based on Sex and Gender to improve Women’s Health (with UCSF Medicine) [P]
❏ NYCConnect: A Digital Platform to Improve Access to Healthcare Services for
Older Adults in New York City [with Cornell Medicine]
❏ Developing a Socio-Cognitive-Computational Approach for Understanding
Persuasion through Misinformation and its Diffusion across Social
Communications Platforms (Amit Almor, Nitin Agarwal, Huan Liu, Amit Sheth,
Marco Valtorta, Douglas Wedell)
5
6. Research and Technology underpinning these efforts
★ Big Data (Volume, Variety- text/social media, images,
video, sensors, multimodal data, Velocity, Veracity)
★ Artificial Intelligence:
○ Knowledge Graphs (domain models, vocabularies,
taxonomies)
○ Natural Language Processing
○ Machine/Deep Learning
6
7. Other topics of possible interest
Any challenging involving social media/news/literature big
data
➔ SM platforms such as Twitter, Reddit, 4Chan
➔ multimodal (text, images,...)
➔ Large scale: 10s of million to billion+ tweets; millions of
Reddit post
➔ analysis: spatio-temporal-thematic or content-
➔ people-network-sentiment-emotion-intention
Applications: Election/public opinion prediction and analysis,
Marketing, Branding (drivers of intentions and actions)
7
8. Biased Blackboxes: Data Has Inherent Bias that DL won’t find on its
own--How to discover and mitigate?
8
Historical Bias
Image Search: “CEO”.
In 2018, 5% of Fortune 500
CEOs were women
Representation Bias
ImageNet Database
Only 1% and 2.1% of the images
come from China and India
Measurement Bias
Race Biased Crime Prediction
Proxy variable “arrest” is often
used to measure “crime”
Aggregation Bias
Predicting Health Complications
Single model catering to different
ethnicities is error prone
Evaluation Bias
Facial Recognition
Only 7.4% & 4.4% share for dark
skinned females in benchmark
datasets such as Adience and
IJB-A.
Deployment Bias
Risk Assessment Tools
Crime prediction models used for
predicting length of a sentence.
And, its error prone.
Suresh, Harini, and John V. Guttag. "A framework for understanding unintended consequences of machine learning." arXiv preprint
arXiv:1901.10002 (2019).
9. 12
Social Good and Social Harm on Social Media
A spectrum to demonstrate the variety of social good, social bad and social ugly.
Adapted from : Purohit, Hemant & Pandey, Rahul. (2019). Intent Mining for the Good, Bad, and Ugly Use of Social Web: Concepts, Methods, and
Challenges. 10.1007/978-3-319-94105-9_1.
Help
offering
Expertise
sharing
Disaster
Relief
Joking
Marketing
Radicalization
Rumouring
Deceiving
Accusing
Sensationalizing
Harassment
Manipulation
Social Good Social Ugly
Social Bad
Positive Effects Negative Effects
10. Context-Aware Harassment Detection on Social
Media
Severity of online harm can differ based on
several criteria
It can span for more than a decades in
one’s life
Or it can lead a teenager to suicide
Visit Project Page: Context-Aware Harassment Detection on Social
Media
Supported by NSF Grants CNS 1513721
11. “Language used to express hatred towards a targeted
individual or group, or is intended to be derogatory,
to humiliate, or to insult the membe
rs of the group, on the basis of attributes such as
race, religion, ethnic origin, sexual orientation,
disability, or gender is hate speech.” - Founta et al.
2018
Challenges in Online Harassment Detection
Ambiguity
According to Pew research center (2017) Researchers have defined harassment using jargon
that overlaps, causing ambiguity in annotations
“Profanity, strongly impolite, rude or vulgar language
expressed with fighting or hurtful words in order to
insult a targeted individual or group is offensive
language ” - Founta et al. 2018
Subjectivity &
Ex: @user_name nah you just a dumb #*! who doesn’t
know her place 😂 😂
This tweet belongs to hate speech and offensive
language based on above definitions
12. Challenges in Online Harassment Detection
Sparsity
Dataset # of Tweets Classes (%)
Waseem et al.
(2016)
16093
Racism (12%),
Sexism (19.6%),
Neither (68.4%)
Davidson et al.
(2017)
24,802
Hate (5%), Non-
hate (95%)
Zhang et al.
(2018)
2,435
Hate (17%),
Non-hate (83%)
Mostly binary classifications
Datasets have small percentages of
positive (harassing) instances
A Quality Type-aware Annotated
Corpus And Lexicon For Harassment
Research [Rezvan et al.]
This paper provides both a quality annotated
corpus and an offensive words lexicon
capturing different types of harassment
content:
(i) sexual
(ii) racial
(iii) appearance-related
(iv) intellectual
(v) political
13. Analyzing and learning the language for different types of
harassment
Why and how
conversations
matter?
Performance of multi-class classifier for
predicting type of harassment incident
Dataset
Linguistic Analysis
Statistical Analysis
Type-aware classifier to identify
type-specific harassment
Using LIWC
High
‘“female
references” in the
intellectual
harassing corpus
High occurrence of
word “I” in sexually
non-harassing
corpus
Analysis of unigrams
Offensive words are
commonplace in
harassing and non-
harassing corpora.
Frequent words not
necessarily offensive
14. ALONE: A Dataset for Toxic Behavior among Adolescents on Twitter
Unique characteristics of the ALONE dataset:
● Data of Adolescent population
● Data organized based on interactions
● Multimodal Data
● Further insights:
○ Positive and negative lexical items in tweets used with
contextual analysis of interactions suggest sarcasm and
sometimes exonerate the potential toxic content.
○ Toxic interactions contain significantly high number of tweets
per interaction along with a high number of multimodal
elements such as images, videos and emojis.
15. Modeling Online Extremism
using Knowledge-infused
and Context-aware
Learning
Visit Project page: http://wiki.aiisc.ai/index.php/Modeling_Radicalization_on_Social_Media_using_Knowledge-infused_and_Context-
aware_Learning
Amit Sheth, Ugur Kursuncu, Vedant Khandelwal, Manas Gaur, Valerie L. Shalin (WSU), Dilshod Achilov (UMass-D),
Krishnaprasad Thirunarayan (WSU), I. Budak Arpinar (UGA),
16. 20
“Reportedly, a number of
apostates were killed in
the process. Just because
they like it I guess..
#SpringJihad
#CountrysideCleanup”
“Kindness is a language
which the blind can see
and the deaf can hear
#MyJihad be kind always”
“By the Lord of Muhammad (blessings and peace be upon him)
The nation of Jihad and martyrdom can never be defeated”
“Jihad” can appear in tweets with different meanings in different dimensions of
the context.
H
I
R
Example Tweets with “Jihad”
18. 22
1. Kursuncu, U., Gaur, M., Castillo, C., Alambo, A., Thirunarayan, K., Shalin, V., ... & Sheth, A. (2019). Modeling Islamist Extremist Communications on Social Media using Contextual Dimensions: Religion,
Ideology, and Hate. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), 1-22.
2. Kursuncu, U., Purohit, H., Agarwal, N., & Sheth, A. (2020). When the Bad is Good and the Good is Bad: Understanding Cyber Social Health through Online Behavioral Change. Expert Systems with
Applications, 2019, 1.
Radicalization Process over time
Modeling
Modeling
Modeling
Dimension
1
Dimension
2
Dimension
3
Dimension
Dimension
Dimension
Dimension
Modeling Process
Dimension
based
Knowledge
enhanced
Representation
Non-extremist
ordinary
individual
Radicalized
extremist
individual
0 1 2 4
Severe
High
Low
None Elevated
3
● Challenges vs. Opportunities
○ Context, false alarms, ambiguity, bias, transparency,
multimodality
○ Incorporate knowledge in ML
● Dimensions:
○ Religion: Qur’an, Hadith, Qur’an Ontology
○ Ideology: Books, lectures of ideologues
○ Hate: Hate Speech Corpus (Davidson et al. 2017)
19. ● A group of extremist users, form a cluster farther
from other users for Religion and Hate.
● Suggesting there might be outliers in the
dataset.
23
Non-Extremist Context for
Religion
Extremist Context for Religion
User Visualization for Dimensions
Kursuncu, U., Gaur, M., Castillo, C., Alambo, A., Thirunarayan, K., Shalin, V., ... & Sheth, A. (2019). Modeling Islamist Extremist Communications on Social Media using
Contextual Dimensions: Religion, Ideology, and Hate. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), 1-22.
20. Public Health, Epidemiology,
Addictions x Social Media & AI
Visit Project page: http://wiki.aiisc.ai/index.php/
Amit Sheth, Raminta Daniulaityte (now ASU), Robert Carlson (WSU),
Francois Lamy, Usha Lokala, Ugur Kursuncu
21. Addiction work - Acknowledgements
PREDOSE - (NIDA) Grant No. R21 DA030571-01A1
eDrug Trends - (NIDA) Grant No. 5R01DA039454-02
eDark Trends - (NIDA) Grant No 1R21DA044518
BD Spoke - (NSF) Award No 1761969
25
22. PREDOSE: Prescription Drug abuse
Online Surveillance and
Epidemiology
Visit Project page: http://wiki.aiisc.ai/index.php/PREDOSE
Amit Sheth, Raminta Daniulaityte (now ASU), Robert Carlson (WSU),
Delroy Cameron (now Apple), Pavan Kapanipathi (now IBM), Sujan Perera (now Amazon)
23. ● Facilitate Techniques for the illicit
use of pharmaceutical opioids
● Capture the knowledge, attitudes,
and behaviors of prescription drug
abusers
● Detection of non-medical use of
pharmaceutical opioids
(buprenorphine)
● Determine spatio-temporal-
thematic patterns and trends in
pharmaceutical opioid abuse
PREDOSE Pipeline
25. Our “loperamide discovery” discovery:
"I Just Wanted to Tell You That Loperamide WILL WORK": A Web-Based Study of
Extra-Medical Use of Loperamide. Journal of Drug and Alcohol Dependence. 2013.
The opioid addictions treatment drugs Buprenorphine and Methadone are
commonly prescribed for treatment of withdrawal symptoms. Our analysis of Web
forums found that Loperamide we widely used for a similar purpose by taking it in
10x-20x prescribed OTC dosage. Three toxicology studies following this work led
to FDA warning in 2016.
Outcome of PREDOSE
Loperamide Withdrawal Discovery
26. eDrug Trends: Trending Social
Media Analysis to Monitor
Cannabis and Synthetic
Cannabinoid use
Visit Project page: http://wiki.aiisc.ai/index.php/EDrugTrends
Amit Sheth, Raminta Daniulaityte (now ASU), Francois R. Lamy, Robert Carlson (WSU),
Krishnaprasad Thirunarayan (WSU), Ramzi Nahhas (WSU), Usha Lokala, Ugur Kursuncu,
27. What is eDrug Trends?
Semi-automated platform to identify emerging trends in cannabis and synthetic
cannabinoid use in the U.S.
To analyze characteristics of marijuana concentrate users, describe patterns and
reasons of use.
To identify factors associated with daily use of concentrates among U.S.-based
cannabis users recruited via a Twitter-based online survey
Identify and compare trends in knowledge, attitudes, and behaviors related to
cannabis and synthetic cannabinoid use across U.S. regions with different cannabis
legalization policies using Twitter and Web forum data.
Analyze social network characteristics and identify key influencers (opinions leaders) in
cannabis and synthetic cannabinoid-related discussions on Twitter
31
29. Outcome of eDrug Trends
" When they say weed causes depression, but it's your fav antidepressant": Knowledge-
aware Attention Framework for Relationship Extraction between Cannabis and Depression
" Is depression related to cannabis?": A knowledge-infused model for Entity and Relation
Extraction with Limited Supervision
'Time for dabs': Analyzing Twitter data on butane hash oil use
"Time for dabs": Analyzing Twitter data on marijuana concentrates across the U.S.
“When ‘Bad’ is ‘Good”: Identifying Personal Communication and Sentiment in Drug-
Related Tweets.
“Those edibles hit hard”: exploration of Twitter data on cannabis edibles in the U.S.
What's your Type?: Contextualized Classification of User Types in Marijuana-Related
Communications Using Compositional Multiview Embedding
eDrug Trends Wiki : http://wiki.aiisc.ai/index.php/EDrugTrends
33
30. eDark Trends: Monitoring
Cryptomarkets to Identify Emerging
Trends of Illicit Synthetic Opioids
Use
Visit Project page: http://wiki.aiisc.ai/index.php/Public_Health_Addictions_Research_at_AIISC
Amit Sheth, Francois R. Lamy, Raminta Daniulaityte (now ASU), Ramzi Nahhas (WSU), Ugur Kursuncu,
31. What is eDark Trends
To monitor Cryptomarkets to Identify Emerging Trends of Illicit Synthetic Opioids Use
Semi automated platform to monitor illicit online transactions of several illicit
synthetic opioids in dark web.
To design effective and responsive prevention and policies for public health
professionals
Epidemiological surveillance by providing timely data regarding emerging substances
and product form
To monitor Darknet supply and marketing trends over time.
Enhancing the capacities of early warning systems like NDEWS
35
33. Outcome of eDark Trends
Global trends, local harms: availability of fentanyl-type drugs on the dark web and
accidental overdoses in Ohio
eDarkFind: Unsupervised Multi-view Learning for Sybil Account Detection
Listed for sale: analyzing data on fentanyl, fentanyl analogs and other novel
synthetic opioids on one cryptomarket
DAO: An Ontology for Substance Use Epidemiology on Social Media and Dark Web
Public Health Addictions Wiki Page:
http://wiki.aiisc.ai/index.php/Public_Health_Addictions_Research_at_AIISC
39
34. BD Spokes: Community-Driven
Data Engineering for Substance
Abuse Prevention in the Rural
Midwest
Visit Project page: http://wiki.aiisc.ai/index.php/Community-
Driven_Data_Engineering_for_Substance_Abuse_Prevention_in_the_Rural_Midwest
Amit Sheth, Ugur Kursuncu, Usha Lokala, Goonmeet Bajaj (OSU), Ayaz Hyder (OSU)
35. Motivation
● The opioid epidemic entrenched in
Ohio and the Midwest of the US.
● The prevalence of opioid and its
impact on the well-being of
individuals and the society in Ohio.
○ Mental Health & Suicide Risk
Questions
1. How can we use social media to measure
mental health impact of opioid prevalence?
1. Are there association between opioid and
mental health/suicide risk based on social
media data?
Approach
Monitoring the prevalence of opioid and its impact on mental health and suicide in Ohio, utilizing
a scalable knowledge and data driven BIGDATA (BD) approach via social media.
BD Spoke: Opioid and Substance Use in Ohio
36. Score
Calculation
Opioid
Mental Health
Depression
Addiction
Suicide Risk
Ideation, Behavior
Attempt
Correlations
● Sheth, Amit, and Pavan Kapanipathi. "Semantic filtering for social data." IEEE Internet Computing 20, no. 4 (2016): 74-78.
● Hussein S. Al-Olimat, Krishnaprasad Thirunarayan, Valerie Shalin, and Amit Sheth. 2018. Location Name Extraction from Targeted Text Streams using Gazetteer-based Statistical
Language Models. In Proceedings of the 27th International Conference on Computational Linguistics (COLING2018), pages 1986-1997. Association for Computational Linguistics
● Gaur, M., Kursuncu, U., Alambo, A., Sheth, A., Daniulaityte, R., Thirunarayan, K., & Pathak, J. (2018, October). " Let Me Tell You About Your Mental Health!" Contextualized Classification
of Reddit Posts to DSM-5 for Web-based Intervention. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management (pp. 753-762).
● Gaur, M., Alambo, A., Sain, J. P., Kursuncu, U., Thirunarayan, K., Kavuluru, R., ... & Pathak, J. (2019, May). Knowledge-aware assessment of severity of suicide risk for early intervention.
In The World Wide Web Conference (pp. 514-525).
● Yazdavar, A. H., Al-Olimat, H. S., Ebrahimi, M., Bajaj, G., Banerjee, T., Thirunarayan, K., ... & Sheth, A. (2017, July). Semi-supervised approach to monitoring clinical depressive symptoms
in social media. In Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017 (pp. 1191-1198).
● Daniulaityte, R., Nahhas, R. W., Wijeratne, S., Carlson, R. G., Lamy, F. R., Martins, S. S., ... & Sheth, A. (2015). “Time for dabs”: Analyzing Twitter data on marijuana concentrates across
the US. Drug and alcohol dependence, 155, 307-311.
News
Articles
Twitter
Data
Domain
Knowledg
e
Content
Enrichment
DAO
DSM-5
Location Extraction
Keyphrase Extraction
Age-based
Clustering
Semantic Filtering
Entity
Extraction
NLM Training
f(.)
Knowledge Infused
Natural Language
Processing (Ki-NLP)
Semantic
Mapping
Semantic
Proximity
Topic Model
Language Model
DAO
DSM-5
Dashboard
Visualizations
(Online)
Offline
Analysis
&
Visualizations
BD Spoke: Opioid and Substance Use in Ohio
37. ● Substance use addictive disorder linked to
opioid with higher correlation.
● Gender dysphoria, Dissociative and OCD
disorders are correlating moderately.
Opioid Prevalence in Ohio vs. Mental Health & Suicide
● Suicide ideation (initial stage) with highest correlation.
● Mild severity level of suicide risk linked to higher
correlation.
● Weak correlation for suicide indication (before initial)
p
N counties
p
N counties
38. Psychidemic:
Measuring the Spatio-Temporal
Psychological Impact of Novel
Coronavirus
Visit Project page: http://wiki.aiisc.ai/index.php/Covid19
Amit Sheth, Valerie L. Shalin (WSU), Ugur Kursuncu, Manas Gaur, Vedant Khandelwal, Vishal Pallagani
40. A calculated Social Quality Index (SQI)
aggregates mental health components
(Depression, Anxiety), Addiction and
Substance Use Disorders.
vecteezy.com
● Change in SQI informs comparisons between
states.
● Raw transformed SQI into relative state
rankings changing over time.
Social Quality Index (SQI)
41. Psychidemic: Mental health & Addiction during CO
SQI Declining..
Frequency
Depression: 88491
Addiction: 24373
Anxiety: 37725
Total: 146589
Frequency
Depression: 123244
Addiction: 84879
Anxiety: 94999
Total: 303122
States show different
patterns on mental
health and addiction.
For the states; OH,
OR, IN, WY, NH, WA,
KS, social well-being
is going worse.
in OH, OR, IN, WY, NH, WA, KS
For information: wiki.aiisc.ai/index.php/Covid19
Psychidemic: Mental health & Addiction during COVID-19
42. SQI bad SQI better SQI better
SQI better
Frequency
Depression: 125037
Addiction: 92897
Anxiety: 81891
Total: 299825
Frequency
Depression: 113830
Addiction: 81810
Anxiety: 74080
Total: 269720
Frequency
Depression: 81463
Addiction: 60166
Anxiety: 45998
Total: 187627
Frequency
Depression: 59088
Addiction: 49086
Anxiety: 46887
Total: 155061
IL, NY, MD,
AZ, NM, MA.
March 14-20 March 21-27 March 28-April 3
April 4-10
Reactions of States, --Improving SQI Ranking
● States have different circumstances, social, health, political.
● Mental health implications are also different.
43. SQI worse
Cluster 4:
CT, LA, NJ,
NV, OK, RI,
WI.
SchoolClosures: CT, LA, NJ, NV,RI,
WV, WI
Business Closures: CT, LA,NJ, RI,
WV, WI
Social Distancing Reg: LA, NJ, RI,
WV, WI
Business Relief: WI
Unemploymentincrease:
CT 2.5K %, LA 2.5K %, NJ 1.2K %,
NV 1.2K %, OK 1.2K %, RI 2.5K %,
WI 1.2K %.
Stay at home: CT, LA,NJ, OK, RI, WI,
WV
Extension School: CT, WV
Major Disaster: NJ
Business Relief: NJ
Unemploymentincrease:
CT 180%,LA 0 %, NJ 64 %,
NV 0 %, OK 99 %, RI -23%, WI 99 %.
Major Disaster: CT, WV
Strict Social Dist: CT, RI
Extensions deadlines: CT
Medical shortage: NJ
Extension Stay home: OK
Extension School: RI
Extension Business Closure: RI
Business Relief: NJ, RI
IndividualRelief: RI
Unemploymentincrease:
CT 0%, LA 5 %, NJ 3 %,
NV 11 %, OK 7 %, RI 0%, WI -5 %.
Extension School: CT
Extension Stay home: LA
Strict Social Dist: NJ
Business Relief: WI
Cluster 5:
FL, GA, MI,
NE, TN, VA,
WV.
SchoolClosures: FL, GA, MI, TN, VA,
WV,
Business Closures: WV, MI
Social Distancing Reg: FL, MI, NE,
TN, VA,WV,
Business Relief: FL, GA, MI, NE, TN,
VA
IndividualRelief: TN, VA
Unemploymentincrease:
FL 600%,GA 650%,MI 180%,
NE 70%, TN 180%,VA 180%,
WV 600%
Stay at home: MI, WV
Shelterin Place: GA
Business Closure: GA, TN
Extension School: GA, WV
Major Disaster: FL
Business Relief: TN
IndividualRelief: TN
Unemploymentincrease:
FL 3.1K%, GA 3K%, MI 1.8K%,
NE 200%,TN 700%,VA 1.6K%,
WV 1.7K%
Stay at home: FL, VA
Shelterin Place: TN
Major Disaster: GA, MI, TN, VA,WV
Strict Social Dist: GA
Extension School: GA, MI
Unemploymentincrease:
FL -25%, GA 190%,MI 27%,
NE 8%, TN 26%, VA 33%,
WV 0%
Extension School: GA
Extension Stay home: MI
SQI worse
SQI worse
SQI worse
SQI better SQI better
SQI better SQI better
March 14-20 March 21-27 March 28-April 3
Influence of External Events
44. Modeling Social Behavior for
Healthcare Utilization in
Depression (NIMH R01)
http://wiki.aiisc.ai/index.php/Modeling_Social_Behavior_for_Healthcare_Utilization_in_Depression
52
45. Overview on Mental Health Research
◎ Understanding mental health from social media (Twitter,
Reddit - at a massive scale; multimodal- text, images) and
clinical (EMR) documents; also developing virtual agent
(chatbot)
◎ Collaborations with domain experts/practitioners, Use of
clinical knowledge (extracted from DSM-5) to enhance
state-of-the-art deep learning and NLP/NLU
◎ Target users/audience/beneficiaries - patients, mental
health experts, clinical researchers, policy makers
53
46. 54
Really struggling with my bisexuality which is causing chaos in my relationship with a girl.
Being a fan of LGBTQ community, I am equal to worthless for her. I’m now starting to get
drunk because I can’t cope with the obsessive, intrusive thoughts, and need to get it out of
my head.
Is mental health related ? Yes: 0.71 , No: 0.29
Which Mental Health condition?
Predicted: Depression (False)
True: Obsessive Compulsive Disorder
Reasoning over Model:
Why model predicted
Depression?
Unknown
Manas Gaur, Ugur Kursuncu, Amanuel Alambo, Amit Sheth,
Raminta Daniulaityte, Krishnaprasad Thirunarayan, and
Jyotishman Pathak. "" Let Me Tell You About Your Mental
Health!" Contextualized Classification of Reddit Posts to DSM-5
for Web-based Intervention." CIKM 2018. [CIKM 2018]
47. 55
Really struggling with my bisexuality which
is causing chaos in my relationship with a
girl. Being a fan of XXX community, I am
equal to worthless for her. I’m now starting
to get drunk because I can’t cope with the
obsessive, intrusive thoughts, and need to
get out of my head.
288291000119102: High risk bisexual behavior
365949003: Health-related behavior finding 365949003: Health-related behavior finding
307077003: Feeling hopeless
365107007: level of mood
225445003: Intrusive thoughts
55956009: Disturbance in content of thought
26628009: Disturbance in thinking
1376001: Obsessive compulsive personality disorder
Multi-hop traversal on
Medical knowledge
graphs
<is symptom>
Achieving Explainability through Medical Entity Normalization :
Replacing Entities in the post with Concepts in the Medical Knowledge Graph through Semantic Annotation
48. 56
K-IL: Semi-Deep Infusion : Matching Reddit Conversation to DSM-5
Scenario
Really struggling with my bisexuality which is causing chaos in my relationship with a girl. Being a
fan of LGBTQ community, I am equal to worthless for her. I’m now starting to get drunk because I
can’t cope with the obsessive, intrusive thoughts, and need to get out of my head.
BPD
DICD PND SAD SBI OCD
Don’t want to live anymore. Sexually assault, ignorant family members and my never
ending loneliness brights up my path to death.
SCW
PND SBI SAD DPR DICD
DPR
I do have a potential to live a decent life but not with people who abandon me.
Hopelessness and feelings of betrayal have turned my nights to days. I am developing
insomnia because of my restlessness.
SBI DPR DICD
BPD I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been
diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.
SBI PND
Reddit DSM-5 [Gaur 2018]
49. 58
Really struggling with my [health-related behavior] which is causing [health-related
behavior] with a girl. Being a fan of [XXX] community, I am equal to [level of mood] for
her. I’m now starting to [drinking] because I can’t cope with the [obsessive compulsive
personality disorder] [disturbance in thinking], and [disturbance in thinking].
Is mental health related ? Yes: 0.96 , No: 0.04
Which Mental Health condition?
Predicted: Obsessive Compulsive Disorder(True)
True: Obsessive Compulsive Disorder
DSM-5 Knowledge
Graph
DSM-5 and Post
Correlation Matrix
Reasoning over Model:
Why model predicted
Obsessive Compulsive
Disorder ? known
Interpretable and
Explainable Learning
D
εRN
P εRN
W f(W)
50. 59
K-IL: Semi-Deep Infusion : Matching Reddit Conversation to DSM-5
Domain-specific
Knowledge lowers
False Alarm Rates.
2005-2016
550K Users
8 Million
Conversations
15 Mental Health
Subreddits
[Gkotsis 2017]
[Saravia 2016]
[Park 2018]
Performance Gains in the outcomes
[CIKM 2018]
51. Other Works: Not Covered
60
Knowledge-aware assessment of severity of suicide risk for early intervention.
In The Web Conference 2019
1. Estimating the severity of suicide risk of an individual without the use of Clinician-
authorized Questionnaire would hurt the explainability of the model.
2. Challenge in Social Media: (a) Dynamic user Roles and Behavior, (b) Sparsity in
Clinician-specific information, and (c) How to adapt Questionnaire to Social Media
3. The study addresses these questions in following ways:
a. Clinical knowledge to study user roles (supportive/non-supportive) and
behaviors (transitioning between communities on Reddit)
b. Columbia-Suicide Severity Rating Scale was modified and adapted to social
Media -- Two new classes were added {Supportive, Suicide Indication}
c. Contextualization and Abstraction procedures were developed for explainable
classification.
52. Other Works: Not Covered
61
Knowledge-infused Abstractive Summarization of Clinical Diagnostic
Interviews In JMIR Mental Health 2021
1. How to pick the relevant question from the recorded script of interview?
2. How to associate the meaningful response to the question being asked?
3. While addressing the 1 and 2, how to incorporate domain knowledge?
4. Addressing 1,2, and 3, how to effectively summarize the long open-ended Clinical
Diagnostic Interviews.
The strategy for summarizing is heavily grounded in the use of mental health knowledge
crafted in prior literature: PHQ-9 Lexicon, DSM-5 Lexicon, Hierarchical knowledge in
SNOMED-CT, and Anxiety Lexicon.
In natural language processing, a domain-specific language model was designed
specific for CDIs and used to identify terms in CDIs which are important for summaries.
53. Hazards SEES: Social and Physical
Sensing Enabled Decision Support for
Disaster Management and Response
Visit Project page:
http://wiki.aiisc.ai/index.php/Social_and_Physical_Sensing_Enabled_Decision_Support
Amit Sheth, Srinivasan Parthasarathy (OSU), Valerie Shalin (WSU), T.K. Prasad (WSU), Manas Gaur
Slide 3: Inner circle : talks about our research areas and strength
Knowledge bias is a bias you can read.
Consider combining data-centric/bottom up/statistical learning with knowledge-based/top down techniques
To improve understanding of simpler content
To understand complex content and concepts
To understand heterogeneous/multimodal content
and a lot more
==============
Move this slide to earlier.
The dataset, and the necessary resources for a specific domain (i.e., white supremacy). The datasets and resources for one problem may not work for another.
V: Consider a graphic at the outset for data driven and knowledge driven approaches, and your infused approach.
Same for ALONE dataset and online toxicity as well.
You need to provide a very strong argument as why they are not alone enough to capture context and knowledge.
Earlier, we have to state that why sole ML solutions and KG solutions have limitations. And why we need knowledge.
Any evaluation of current state-of-the-art models and their shortcomings, figure or a table.
If not solved, big social media platforms make big harms to the society.
Online harassment example (prolonged): https://www.theguardian.com/society/2018/aug/03/harassed-online-for-13-years-the-victim-who-feels-free-at-last
Online harassment(teenagers death) - https://www.cnn.com/2018/01/23/us/florida-cyberstalking-charges-girl-suicide/index.html
Online harassment subjectivity article : https://www.newscientist.com/article/2140342-online-harassment-on-the-rise-but-no-one-can-agree-what-it-is/
Sparsity:
[1] Zeerak Waseem. Are you a racist or am i seeing things? annotator influence on hate speech detection on twitter. In Proc. of the Workshop on NLP and Computational Social Science, pages 138–142. Association for Computational Linguistics, 2016.
[2] Thoams Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. Automated hate speech detection and the problem of offensive language. In Proceedings of the 11th Conference on Web and Social Media. AAAI, 2017.
[3] Zhang, Ziqi & Luo, Lei. (2018). Hate Speech Detection: A Solved Problem? The Challenging Case of Long Tail on Twitter. Semantic Web. Accepted. 10.3233/SW-180338.
A Quality Type-aware Annotated Corpus and Lexicon for Harassment Research. Web Science, WebSci 2018, Amsterdam, The Netherlands, May 27-30, 2018
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0227330
“In general, a single tweet identified as ‘harassing’ may not provoke the same intense negative feeling that we associate with that word in the real-world scenario. However, in practice, ‘conversational’ exchanges containing a sequence of such tweets can rise to the level of harassment causing mental and psychological anguish, and fear of physical harm.”
Ambiguity: Different meanings of diagnostic terms
Sparsity: Low prevalence of relevant content
Subjectivity: Different perceptions of same concepts.
Multi Dimensionality: Nature of content with more than one context.
Multimodal Content: Different modalities of data
========================
Move this slide to earlier and back it up with examples.
Challenges:
Context in social media conversations is fluid and have shades of gray.
Deeper Understanding of Content through Contextual Dimensions
False alarms in the models developed and deployed.
Ambiguity/multi-dimensionality, Sparsity, Subjectivity, Multimodality...
Bias and lack of transparency : Impacts masses.
Interpretability + Traceability → Explainability
Opportunity: The use of knowledge to improve the model
What is the right knowledge resource to use for quality insights?
Knowledge plays an indispensable role in deeper understanding of content
V: HIGHLIGHT JIHAD in Slide
With the focus on the role knowledge plays, often complementing/enhancing
ML and NLP techniques, in contextual “understanding” of data to help solve the problem for which the data is potentially relevant.
This encompasses topics of information extraction and semantic annotation.
Challenges:
Context in social media conversations is fluid and have shades of gray.
Deeper Understanding of Content through Contextual Dimensions
False alarms in the models developed and deployed.
Ambiguity/multi-dimensionality, Sparsity, Subjectivity, Multimodality...
Bias and lack of transparency : Impacts masses.
Interpretability + Traceability → Explainability
Opportunity: The use of knowledge to improve the model
What is the right knowledge resource to use for quality insights?
R. Daniulaityte, R. Carlson, R. Falck, D. Cameron, S. Perera, L. Chen, A. P. Sheth. 130(1-3): 241-244, 2013. ScienceDirect, [PMID 23201175]</ref> <ref>R. Daniulaityte, R. Carlson, R. Falck, D. Cameron, S. Perera, L. Chen, A. P. Sheth. A Web-Based Study of Self-Treatment of Opioid Withdrawal Symptoms with Loperamide. The College on Problems of Drug Dependence CPDD 2012, Palm Springs, CA USA, June 9-14, 2012.
To design effective and responsive prevention and policies for public health professionals
Epidemiological surveillance by providing timely data regarding emerging substances and product form
Pairwise correlation between opioid and mental health, suicide.
Raw SQI does not take into account preceding state conditions.
Change in SQI is also potentially informative, particularly for comparisons between states.
We transformed raw state SQI into a relative state ranking, to capture drifts between worsening and improvement.
Used to examine the effect of events, e.g., school closure, business closure, unemployment, and lockdown on worsening mental health.
Raw SQI does not take into account preceding state conditions.
Change in SQI is also potentially informative, particularly for comparisons between states.
We transformed raw state SQI into a relative state ranking, to capture drifts between worsening and improvement.
Used to examine the effect of events, e.g., school closure, business closure, unemployment, and lockdown on worsening mental health.
Disambiguating and Contextualizing the tweets using medical knowledge graphs, we observed patterns of improvement in conditions as the decline in the number of tweets on Depression, Addiction, and anxiety
Much of these is due to meditation, yoga, indoor games, increase use of streaming video platforms
Among many external factors, financial events and the specific government interventions have substantial effect in the social quality of people. Specifically, business and individual relief announcements, business closures, increase in unemployment, and stay at home orders. Whenever the unemployment increase is much more significant than the previous week, the social quality is worse; and whereas whenever the individuals and businesses are given financial reliefs, the social quality is better.