Paper: IAMHAPPY: Towards An IoT Knowledge-Based Cross-Domain Well-Being Recommendation System for Everyday Happiness IEEE/ACM Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE) Conference

1. IAMHAPPY: Towards an IoT Knowledge-Based
Cross-Domain Well-Being Recommendation System
for Happiness
Ohio Center of Excellence in Knowledge-Enabled Computing
Dr. Amelie Gyrard
Kno.e.sis Research Center
Department of Computer Science and Engineering,
Wright State University, Dayton, Ohio (USA)
Research website
LinkedIn, Twitter, Facebook, Google Group
Dr. Amit Sheth
AI Institute
University of South Carolina

2. Agenda
• Introduction to Happiness and Well-Being
• Towards a Recommender System for Happiness
and Well-Being
• Reusing the Knowledge Expertise:
̶ Demo: LOV4IoT Knowledge Repository
̶ Demo: SLOR Reasoning Engine and
Recommender System
• Conclusion & Future Work
2

3. The Importance of Positive Thinking:
Happiness and Well-Being
3
● Have you already ask yourself how to feel happy everyday?
University of Pennsylvania
Positive Psychology Center

4. Happiness Manager New Trend
● Is there a “Chief Happiness Officer” or “Happiness Manager” at your
institute?

5. Dr. Masaru Emoto's Rice and
Water Experiments
Dr. Emoto was President Emeritus of the International Water For Life Foundation,
a non-profit organization based in Oklahoma City in the United States.
In 1992, Emoto became a Doctor of Alternative Medicine at
the Open International University for Alternative Medicine in India.
http://www.truthin7minutes.com/negative-rice-experiment/, https://www.masaru-emoto.net/en/science-of-messages-from-water/
Dr. Masaru Emoto's Rice Experiment

6. Affective Computing
6
How to find the right keywords/key phrases
that are synonyms?
• Affective Sciences
• Affective Computing
• Emotions and specific emotions name
(e.g., happy)
• Wearable devices for emotion
• Positive thinking
• Positive psychology
• Etc.

7. Limitations of Current Emotion-related Applications
• Emotion applications are focused on facial recognitions
• Most of the current IoT applications are just visualizing the data.

8. Literature Survey: Limitations
8
Covering one specific
emotion per application
Need to support new
multimodal sensors
Need to unify the
reasoning mechanism to
interpret (IoT) data
Need to support
complementary domains

9. Research Challenges (RC)
• (RC1) Which physiological data generated by sensors are
needed to deduce users' emotions?
(e.g., a stress sensor, does the serotonin level help
understanding depression level?).
• (RC2) How to integrate knowledge from complementary
domains?
(e.g., naturopathy and mindful activities).
• (RC3) How to deduce meaningful knowledge from sensor data?
How to design the well-being recommendation system?
9
RC
RC 3
RC 2
RC 1

10. Agenda
• Introduction to Happiness and Well-Being
• Towards a Recommender System for Happiness
and Well-Being
• Reusing the Knowledge Expertise:
̶ Demo: LOV4IoT Knowledge Repository
̶ Demo: SLOR Reasoning Engine and
Recommender System
• Conclusion & Future Work
10

11. Our Solution: IAMHAPPY
• An innovative IoT-based well-being recommendation system to
encourage every day happiness.
• A (Semantic) Web-based knowledge repository helps analyze
data produced by IoT devices to understand users' emotions
and health.
• The knowledge repository is integrated with a rule-based engine
to suggest recommendations.
11

12. IAMHAPPY Architecture
12

13. Reusing Reliable Scientific Knowledge
13
Knowledge Repository based on
Reliable Scientific Research
Taichi health and well-being benefits:
● Heart, bones, nerves, muscles, immune system, and the mind.
● Enhance work productivity, creativity, and sports performance.
● Enhancing mood

14. Cross-Domain Knowledge Repository
14
Qigong
Art
Therapy
Music
Therapy
Acupuncture
Visualization
Techniques
Taichi
Etc.
Naturopathy/
Food
Fitness
Obesity
Emotion
Sleep
Depression
Stress/Anxiety
Yoga
Ontology-based
projects found
for those topics
Meditation
Topics published as
reliable scientific
publication
Legend:

15. Research Studies on Meditation
• Effects of meditation on the brain
Paul Ekman,
pioneer in the study of emotions
Matthieu Ricard,
PhD in cellular genetics and monk.

16. NCCIH complementary and alternative medicine:
● Whole medical systems such as homeopathy, naturopathy, traditional Chinese medicine,
and ayurveda.
● Mind-body medicine such as meditation, mental healing, art therapy, music therapy, and
dance therapy.
● Biologically based practices such as dietary supplements, herbal supplements.
● Manipulative and body-based practices such as spinal manipulation (both chiropractic and
osteopathic) and massage.
● Energy therapies such as qigong, reiki, therapeutic touch, and electromagnetic therapy.
US NSF NIH NCCIH

17. Demo & Reminder: Ontology, Schema, and
Knowledge Graph
Structured information
automatically built by machines =>

18. Demo & Background: Schema.org
Structured information
automatically built by machines =>

19. Demo & Background:
Google Knowledge Graph
Video (2 mins 44): https://youtu.be/mmQl6VGvX-c
Blog: https://googleblog.blogspot.com/2012/05/introducing-knowledge-graph-things-not.html
Google Knowledge Graph 2012

20. Paper: “Semantic Web of Things: an analysis of the application semantics for the IoT
moving towards the IoT convergence” [Jara et al. 2014]
Why Do We Use Semantic Web Technologies Within Internet of Things?
⇒ How to provide a common
description of sensor data
to later reason on it?
• Machine-understandable data
• Describe data with common
vocabularies
• Reuse domain knowledge
• Link to other data
• Ease the reasoning
20

21. Workflow: From Data to Recommendations
21
Paper: Cross-Domain Internet of Things Application Development: M3 Framework and Evaluation
[Gyrard et al. 2015]. International Conference on Future Internet of Things and Cloud (FiCloud) (Cited x60 in Sept. 2019).
White Paper: Semantic Interoperability for the Web of Things [Murdock et al. 2016] (35 authors)

22. Agenda
• Introduction to Happiness and Well-Being
• Towards a Recommender System for Happiness
and Well-Being
• Reusing the Knowledge Expertise:
̶ Demo: LOV4IoT Knowledge Repository
̶ Demo: SLOR Reasoning Engine and
Recommender System
• Conclusion & Future Work
22

23. Reusing domain knowledge from
Ontology Catalogs
● BioPortal for biomedical ontologies [1]
● Linked Open Vocabularies (LOV) [2]
● Linked Open Vocabularies for Internet of Things (LOV4IoT) [3]
○ Exploring knowledge from ontologies and scientific
publications
[1] https://bioportal.bioontology.org/, [2] https://lov.linkeddata.es/dataset/lov/,
[3] http://lov4iot.appspot.com/
RC 1, 2, 4

24. Cannot Find Ontologies on BioPortal for
Our Needs
Knowledge repositories for the
topics that we are interested in are
needed

25. LOV4IoT-Extensions
Emotion
(15)
Fitness
(5)
Food
(40)
Obesity
(4)
Sleep (1)
Depression
(6)
Stress (1) Anxiety
• Knowledge repository extended to support
Happiness and Well-Being Systems
Topic
(# ontology-based
projects)
Acupuncture
(3)
Legend:

26. Demo: Reusing Domain Knowledge
already Designed within Applications
LOV4IoT Project: http://lov4iot.appspot.com/?p=ontologies 26
Classify
Exchange
Annotate
Annotate
Collect

27. Demo: Ontology Catalog for Food
27
http://lov4iot.appspot.com/?p=lov4iot-food
<= New projects
can be added
<= Ontology
code availability

28. KE4WoT: Domain Knowledge
Extraction Research Methodology
28
Outstanding Paper Award - Concept Extraction from Web of Things Knowledge
Bases. International Conference WWW/Internet. 21-23 October 2018, Budapest,
Hungary. Mahda Noura, Amelie Gyrard, Sebastian Heil, and Martin Gaedke

29. KE4WoT Knowledge Extraction of Popular Terms
E.g., Smart Home and Ambient Assisted Living
29
Paper: Automatic Knowledge Extraction to build Semantic Web of Things Applications. Mahda Noura, Amelie Gyrard,
Sebastian Heil, Martin Gaedke. IEEE Internet of Things (IoT) Journal 2019. Impact factor: 9.515 in Sept. 2019.
3rd top journal in Computing Systems according to Google Scholar, h5 index=70 29
Also statistics: Extent of
re-use among ontologies
Planned extensions for
numerous domains
introduced here.

30. Agenda
• Introduction to Happiness and Well-Being
• Towards a Recommender System for Happiness
and Well-Being
• Reusing the Knowledge Expertise:
̶ Demo: LOV4IoT Knowledge Repository
̶ Demo: SLOR Reasoning Engine and
Recommender System
• Conclusion & Future Work
30

31. Demo: Selecting Emotion Sensors
and Existing Projects
31http://linkedopenreasoning.appspot.com/?p=slorv2
1 - Choose the “Emotion” domain 2 - Choose a sensor from our dictionary
(automatically retrieved)
3 - Projects using this sensor are
automatically retrieved from the
LOV4IoT knowledge repository

32. Demo: Selecting Health Sensors
and Existing Rules to Interpret data
32
http://linkedopenreasoning.appspot.com/?p=slorv2
1 - Generic solution for another domain:
“Healthcare”
3 - Rules automatically retrieved for a
specific sensor
2 - Choose a sensor from our dictionary
(automatically retrieved)

33. Demo: Using Reliable Knowledge
for Recommendations
• Current suggestions provide the source of knowledge
Most of the time scientific publications who designed ontologies
33
http://sensormeasurement.appspot.com/?p=naturopathy
Scientific publications
are provided to prove the
veracity of the
recommendations

34. Planned Evaluation
● Planned evaluation
○ User feedback (need more robust software)
○ Physiological datasets
○ Partnership with clinicians to access depressed-diagnosed patients
● Guidelines to evaluate patient's happiness and reduce depression
○ Satisfaction With Life Scale (SWLS)
○ Oxford Happiness Questionnaire (OHQ)
○ Gratitude Questionnaire (GQ-6)
○ Circumplex mood model
○ Oldenburg burnout inventory
34

35. Agenda
• Introduction to Happiness and Well-Being
• Towards a Recommender System for Happiness
and Well-Being
• Reusing the Knowledge Expertise:
̶ Demo: LOV4IoT Knowledge Repository
̶ Demo: SLOR Reasoning Engine and
Recommender System
• Conclusion & Future Work
35

36. Conclusion & Future Work
● IAMHAPPY: Towards An IoT Knowledge-Based Cross-Domain Well-Being
Recommendation System for Everyday Happiness
○ Societal impact
● Future work:
○ Integration with real sensors
○ Evaluation with real patients
36

37. Acknowledgments & Questions
37
● Thanks to the Kno.e.sis team for fruitful discussions.
● The opinions expressed are those of the authors and do not reflect those
of the sponsors.

38. IAMHAPPY: Towards an IoT Knowledge-Based
Cross-Domain Well-Being Recommendation System
for Happiness
Ohio Center of Excellence in Knowledge-Enabled Computing
Dr. Amelie Gyrard
Kno.e.sis Research Center
Department of Computer Science and Engineering,
Wright State University, Dayton, Ohio (USA)
Research website
LinkedIn, Twitter, Facebook, Google Group
Dr. Amit Sheth
AI Institute
University of South Carolina