SlideShare ist ein Scribd-Unternehmen logo
1 von 216
Downloaden Sie, um offline zu lesen
Data Processing and Semantics for Advanced
Internet of Things (IoT) Applications:
modeling, annotation, integration, and perception
Pramod Anantharam1
, Payam Barnaghi2
, Amit Sheth1
1
Kno.e.sis Center, Wright State University
2
Centre for Communication Systems Research, University of Surrey
Madrid, Spain, June 12-14, 2013
http://aida.ii.uam.es/wims13/keynotes.php#tutorials
Special Thanks to:
Cory Henson, Kno.e.sis Research Center, Wright State University
Wei Wang, Centre for Communication Systems Research, University of Surrey
2
Part 1: Introduction
to Internet of “Things”
Image source: CISCO
Internet of Things
 “sensors and actuators embedded in physical
objects — from containers to pacemakers —
are linked through both wired and wireless
networks to the Internet.”
 “When objects in the IoT can sense the
environment, interpret the data, and
communicate with each other, they become
tools for understanding complexity and for
responding to events and irregularities swiftly”
source: http://www.iot2012.org/
“Thing” connected to the internet
“In 2013 cumulative shipments of Bluetooth-enabled devices will surpass 10 billion
and Wi-Fi enabled devices will surpass 10 billion cumulative shipments in 2015,”
- Peter Cooney, wireless analyst with ABI Research
“73% of Countries with 4G Services Have Dropped Their 4G Tariffs by an
Average of 30% in the Past 6 Months”1
- ABI Research (07 Feb 2013)
1http://www.abiresearch.com/
5
Network connected Things and
Devices
Image courtesy: CISCO
6
Sensor devices are becoming widely
available
- Programmable devices
- Off-the-shelf gadgets/tools
7
More “Things” are being connected
Home/daily-life devices
Business and
Public infrastructure
Health-care
…
8
People Connecting to Things
Motion sensor
Motion sensor
Motion sensor
ECG sensor
Internet
9
Things Connecting to Things
- Complex and heterogeneous
resources and networks
10
Wireless Sensor Networks (WSN)
Sink
node Gateway
Core network
e.g. InternetGateway
End-user
Computer services
- The networks typically run Low Power Devices
- Consist of one or more sensors, could be different type of sensors (or actuators)
11
Key characteristics of IoT devices
 Often inexpensive sensors (actuators) equipped with a radio
transceiver for various applications, typically low data rate ~ 10-250
kbps.
 Deployed in large numbers
 The sensors should coordinate to perform the desired task.
 The acquired information (periodic or event-based) is reported back
to the information processing centre (or sometimes in-network
processing is required)
 Solutions are application-dependent.
11
12
Beyond conventional sensors
 Human as a sensor (citizen sensors)
 e.g. tweeting real world data and/or events
 Virtual (software) sensors
 e.g. Software agents/services
generating/representing data
Road block, A3
Road block, A3
Suggest a different route
13
Cyber, Physical and Social Data
14
Citizen Sensors
Source: How Crisis Mapping Saved Lives in Haiti, Ushahidi Haiti Project (UHP).
15
Cosm- Air Quality Egg
16
Cosm- data readings
Tags
Data formats
Location
17
Making Sense of Data
In the next few years, sensor networks will produce
10-20 time the amount of data generated by social
media. (source: GigaOmni Media)
18
Things, Data, and lots of it
image courtesy: Smarter Data - I.03_C by Gwen Vanhee
19
Big Data and IoT
 "Big data" is a term applied to data sets whose size is beyond the ability of
commonly used software tools to capture, manage, and process the data
within a tolerable elapsed time. Big data sizes are a constantly moving
target, as of 2012 ranging from a few dozen terabytes to many petabytes of
data in a single data set.” (wikipedia)
 Every day, we create 2.5 quintillion bytes of data — so much that 90% of
the data in the world today has been created in the last two years alone.
(source IBM)
20
The seduction of data
 Turn 12 terabytes of Tweets created each day into sentiment analysis
related to different events/occurrences or relate them to products and
services.
 Convert (billions of) smart meter readings to better predict and balance
power consumption.
 Analyze thousands of traffic, pollution, weather, congestion, public transport
and event sensory data to provide better traffic management.
 Monitor patients, elderly care and much more…
Adapted from: What is Bog Data?, IBM
21
Do we need all these data?
22
“Raw data is both an oxymoron and bad
data”
Geoff Bowker, 2005
ource: Kate Crawford, "Algorithmic Illusions: Hidden Biases of Big Data", Strata 2013.
23
IoT Data in the Cloud
Image courtesy: http://images.mathrubhumi.com
http://www.anacostiaws.org/userfiles/image/Blog-Photos/river2.jpg
24
Perceptions and Intelligence
Data
Information
Knowledge
Wisdom
Raw sensory data
Structured data (with
semantics)
Abstraction and perceptions
Actionable intelligence
25
Change in communication paradigm
Sink
node Gateway
Core network
e.g. Internet End-user
Data
Sender
Data
Receiver
A sample data communication in conventional networks
A sample data communication in WSN
Fire! Some bits
01100011100
26
 Collaboration and in-network processing
 In some applications a single sensor node is not able to handle the
given task or provide the requested information.
 Instead of sending the information form various source to an external
network/node, the information can be processed in the network itself.
 e.g. data aggregation, summarisation and then propagating the processed
data with reduced size (hence improving energy efficiency by reducing the
amount of data to be transmitted).
 Data-centric
 Conventional networks often focus on sending data between two
specific nodes each equipped with an address.
 Here what is important is data and the observations and measurements
not the node that provides it.
Required mechanisms
“People want answers, not numbers”
(Steven Glaser, UC Berkley)
Sink
node Gateway
Core network
e.g. Internet
What is the temperature at home?Freezing!
28
IoT Data alone is not enough
 Domain knowledge
 Machine interpretable meta data
 Delivery, sharing and representation services
 Query, discovery, aggregation services
 Publish, subscribe, notification, and access interfaces/services
29
Storing, Handling and Processing
the Data
Image courtesy: IEEE Spectrum
30
IoT Data Challenges
 Discovery: finding appropriate device and data sources
 Access: Availability and (open) access to IoT resources and data
 Search: querying for data
 Integration: dealing with heterogeneous device, networks and data
 Interpretation: translating data to knowledge usable by people and
applications
 Scalability: dealing with large number of devices and myriad of data
and computational complexity of interpreting the data.
Energy consumption of the nodes
 Batteries have small capacity and recharging
could be complex (if not impossible) in some
cases.
 The main consumers of the energy are: the
controller, radio, to some extent memory and
depending on the type, the sensor(s).
 A controller can go to:
 “active”, “idle” and “sleep”
 A radio modem could turn transmitter,
receiver, or both on or off,
 sensors and memory can be also turned on
and off.
Actuators
Stepper Motor [1]
Image credits:
[1] http://directory.ac/telco-motion.html
[2] http://bruce.pennypacker.org/category/theater/
[3] http://www.busytrade.com/products/1195641/TG-100-Linear-Actuator.html
[4] http://www.arbworx.com/services/fencing-garden-fencing/
[2]
[3][4]
Wireless Sensor Networks (WSN)-
gateway connection
Distributed WSN
What are the main issues?
 Heterogeneity
 Interoperability
 Mobility
 Energy efficiency
 Scalability
 Security
What is important?
 Robustness
 Quality of Service
 Scalability
 Seamless integration
 Security, privacy, Trust
In-network processing
 Mobile Ad-hoc Networks are supposed to deliver
bits from one end to the other
 WSNs, on the other end, are expected to provide
information, not necessarily original bits
 Gives addition options
 E.g., manipulate or process the data in the network
 Main example: aggregation
 Applying aggregation functions to a obtain an average
value of measurement data
 Typical functions: minimum, maximum, average, sum, …
 Not amenable functions: median
source: Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor Networks
Holger Karl, Andreas Willig, chapter 3, Wiley, 2005 .
In-network processing- example
Applying Symbolic Aggregate Approximation (SAX)
SAX Pattern (blue) with word length of 20 and a vocabulary of 10 symbols
over the original sensor time-series data (green)
Data-centric networking
 In typical networks (including ad hoc networks),
network transactions are addressed to the identities
of specific nodes
 A “node-centric” or “address-centric” networking paradigm
 In a redundantly deployed sensor networks, specific
source of an event, alarm, etc. might not be important
 Redundancy: e.g., several nodes can observe the same
area
 Thus: focus networking transactions on the data
directly instead of their senders and transmitters !
data-centric networking
 Principal design change
source: Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor Networks
Holger Karl, Andreas Willig, chapter 3, Wiley, 2005 .
Implementation options for
data-centric networking
 Overlay networks & distributed hash tables (DHT)
 Hash table: content-addressable memory
 Retrieve data from an unknown source, like in peer-to-peer networking – with
efficient implementation
 Some disparities remain
 Static key in DHT, dynamic changes in WSN
 DHTs typically ignore issues like hop count or distance between nodes when
performing a lookup operation
 Publish/subscribe
 Different interaction paradigm
 Nodes can publish data, can subscribe to any particular kind of data
 Once data of a certain type has been published, it is delivered to all subscribes
 Subscription and publication are decoupled in time; subscriber and published are
agnostic of each other (decoupled in identity);
 There is concepts of Semantic Sensor Networks- to annotate sensor resources
and observation and measurement data!
Adapted from: Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor Networks
Holger Karl, Andreas Willig, chapter 3, Wiley, 2005 .
IoT and Semantic technologies
 The sensors (and in general “Things”) are increasingly
being integrated into the Internet/Web.
 This can be supported by embedded devices that
directly support IP and web-based connection (e.g.
6LowPAN and CoAp) or devices that are connected
via gateway components.
 Broadening the IoT to the concept of “Web of Things”
 There are already Sensor Web Enablement (SWE)
standards developed by the Open Geospatial
Consortium that are widely being adopted in industry,
government and academia.
 While such frameworks provide some interoperability,
semantic technologies are increasingly seen as key
enabler for integration of IoT data and broader Web
information systems.
Semantics and IoT resources and
data
 Semantics are machine-interpretable metadata (for mark-up),
logical inference mechanisms, query mechanism, linked data
solutions
 For IoT this means:
 ontologies for: resource (e.g. sensors), observation and
measurement data (e.g. sensor readings), domain concepts (e.g.
unit of measurement, location), services (e.g. IoT services) and
other data sources (e.g. those available on linked open data)
 Semantic annotation should also supports data represented
using existing forms
 Reasoning /processing to infer relationships and hierarchies
between different resources, data
 Semantics (/ontologies) as meta-data (to describe the IoT
resources/data) / knowledge bases (domain knowledge).
43
A Few Words
on
Semantic Web
SSW Introduction
lives
in
has
pet
is ahas pet
Perso
n
Perso
n
Anima
l
Anima
l
Concrete Facts
Resource Description Framework
Concrete Facts
Resource Description Framework
Semantic Web
(according to Farside)
General Knowledge
Web Ontology Language
General Knowledge
Web Ontology Language
“Now! – That should clear up a few things around
is a
Semantic Web Stack
Linked Open Data
Linked Open Data
~ 50 Billion Statements~ 50 Billion Statements
SW is moving from academia
to industry
In the last few years, we have
seen many successes …
Knowledge Graph
Watson
Apple
Siri
Google Knowledge Graph
51
Sensors and the Web
Sensors are ubiquitous
Enabling the Internet of Things
Situational awareness
enables:
 Devices/things to function
and adapt within their
environment
 Devices/things to work
together
Sensor systems are
too often stovepiped.
Closed centralized
management of sensing
resources
Closed inaccessible
data and sensors
We want to set this data free
With freedom comes
responsibility
Discovery, access, and search
Integration and interpretation
Scalability
Drowning in Data
A cross-country flight from New York to Los Angeles on a
Boeing 737 plane generates a massive 240 terabytes of
data
- GigaOmni Media
Drowning in Data
Challenges
To fulfill this vision, there are difficult challenges to overcome such
as the discovery, access, search, integration, and interpretation of
sensors and sensor data at scale
Discovery finding appropriate sensing resources and data sources
Access sensing resources and data are open and available
Search querying for sensor data
Integration dealing with heterogeneous sensors and sensor data
Interpretation translating sensor data to knowledge usable by people
and
applications
Scalability dealing with data overload and computational complexity
of interpreting the data
Solution
Semantic Sensor Web
Internet Computing, July/Aug.
2008
Uses the Web as platform for
managing sensor resources and
data
Uses semantic technologies for
representing data and knowledge,
integration, and interpretation
Solution
Discovery, access, and search
 Using standard Web services
 OGC Sensor Web Enablement
Solution
Integration
 Using shared domain models / data
representation
 OGC Sensor Web Enablement
 W3C Semantic Sensor Networks
Solution
Interpretation
 Abstraction – converting low-level data to high-level knowledge
 Machine Perception – w/ prior knowledge and abductive
reasoning
 IntellegO – Ontology of Perception
Solution
Scalability
 Data overload – sensors produce too much data
 Computational complexity of semantic interpretation
 “Intelligence at the edge” – local and distributed integration and
interpretation of sensor data
SSW Adoption and Applications
65
Part 2: Data and knowledge
modelling requirements,
semantic annotation
Image source: CISCO
Recall of the Internet of Things
 A primary goal of interconnecting devices and
collecting/processing data from them is to
create situation awareness and enable
applications, machines, and human users to
better understand their surrounding
environments.
 The understanding of a situation, or context,
potentially enables services and applications
to make intelligent decisions and to respond to
the dynamics of their environments.Barnaghi et al 2012, “Semantics for the Internet of Things: early progress and back to the future”
IoT challenges
 Numbers of devices and different users and interactions required.
 Challenge: Scalability
 Heterogeneity of enabling devices and platforms
 Challenge: Interoperability
 Low power sensors, wireless transceivers, communication, and networking
for M2M
 Challenge: Efficiency in communications
 Huge volumes of data emerging from the physical world, M2M and new
communications
 Challenge: Processing and mining the data, Providing secure access and
preserving and controlling privacy.
 Timeliness of data
 Challenge: Freshness of the data and supporting temporal requirements in
accessing the data
 Ubiquity
 Challenge: addressing mobility, ad-hoc access and service continuity
 Global access and discovery
 Challenge: Naming, Resolution and discovery
IoT: one paradigm, many visions
Diagram adapted from L. Atzori et al., 2010, “the Internet of Things: a Survey”
Semantic oriented vision
 “The object unique addressing and the representation
and storing of the exchanged information become the
most challenging issues, bringing directly to a ‘‘Semantic
oriented”, perspective of IoT”, [Atzori et al., 2010]
 Data collected by different sensors and devices is
usually multi-modal (temperature, light, sound, video,
etc.) and diverse in nature (quality of data can vary with
different devices through time and it is mostly location
and time dependent [Barnaghi et al, 2012]
 some of challenging issues: representation, storage, and
search/discovery/query/addressing, and processing IoT
resources and data.
What is expected?
 Unified access to data: unified descriptions
 Deriving additional knowledge (data mining)
 Reasoning support and association to other entities and
resources
 Self-descriptive data an re-usable knowledge
 In general: Large-scale platforms to support discovery
and access to the resources, to enable autonomous
interactions with the resources, to provide self-
descriptive data and association mechanisms to reason
the emerging data and to integrate it into the existing
applications and services.
Semantic technologies and IoT
 There are already Sensor Web Enablement
(SWE) standards developed by the Open
Geospatial Consortium that are widely adopted.
 While such frameworks provide certain levels of
interoperability, semantic technologies are seen
as key enabler for integration of IoT data and
and existing business information systems.
 Semantic technologies provide potential support
for:
 Interoperability and machine automation
 IoT resource and data annotation, logical inference, query
and discovery, linked IoT data
Identify IoT domain concepts
 Users
 Physical entities
 Virtual entities
 Devices
 Resource
 Services
 …
Diagram adapted from IoT-A project D2.1
IoT domain concepts - Entity
 Physical entities (or entity of
interests): objects in the physical world,
features of interest that are of interests to
users (human users or any digital artifacts).
 Virtual entities: virtual representation of the
physical entities.
 Entities are the main focus of interactions
between humans and/or software agents.
 This interaction is made possible by a hardware
component called Device.
Definition adapted from De et al, 2012, “Service modeling for the Internet of Things”
IoT domain concepts –
Device, Resource and Service
 A Device mediates the interactions between users
and entities.
 The software component that provides information
on the entity or enables controlling of the device, is
called a Resource.
 A Service provides well-defined and standardised
interfaces, offering all necessary functionalities for
interacting with entities and related processes.
Definition adapted from De et al, 2012, “Service modeling for the Internet of Things”
Other concepts need to considered
 Gateways
 Directories
 Platforms
 Systems
 Subsystems
 …
 Relationships among them
 And links to existing knowledge base and linked data
Don’t forget the IoT data
 Sensors and devices provide observation and
measurement data about the physical world objects
which also need to be semantically described and can
be related to an event, situation in the physical world.
 The processing of data into knowledge/ perception and
using it for decision making, automated control, etc.
 Huge amount of data from our physical world that need
to be
 Annotated
 Published
 Stored (temporary or for longer term)
 Discovered
 Accessed
 Proceeded
 Utilised in different applications
Semantics for IoT resources and data
 Semantics are machine-interpretable metadata, logical inference
mechanisms, query and search mechanism, linked data…
 For IoT this means:
 ontologies for: resource (e.g. sensors), observation and
measurement data (e.g. sensor readings), services (e.g. IoT
services), domain concepts (e.g. unit of measurement, location)
and other data sources (e.g. those available on linked open
data)
 Semantic annotation should also supports data represented using
existing forms
 Reasoning/processing to infer relationships between different
resources and services, detecting patterns from IoT data
Characteristics of IoT resources
 Extraordinarily large number
 Limited computing capabilities
 Limited memory
 Resource constrained environments (e.g.,
battery life, signal coverage)
 Location is important
 Dynamism in the physical environments
 Unexpected disruption of services
 …
Characteristics of IoT data
 Stream data (depends on time)
 Transient nature
 Almost always related to a phenomenon or
quality in our physical environments
 Large amount
 Quality in many situations cannot be assured
(e.g., accuracy and precision)
 Abstraction levels (e.g., raw, inferred or
derived)
 …
Utilise semantics
 Find all available resources (which can provide
data) and data related to “Room A” (which is an
object in the linked data)?
 What is “Room A”? What is its location? returns “location” data
 What type of data is available for “Room A” or that “location”?
(sensor category types)
 Predefined Rules can be applied based on
available data
 (TempRoom_A > 80°C) AND (SmokeDetectedRoom_A position==TRUE) 
FireEventRoom_A
 Learning these rules needs data mining or pattern recognition
techniques
81
Part 3: Semantics and data modelling
for IoT
Image source: CISCO
Semantic modelling
 Lightweight: experiences show that a lightweight
ontology model that well balances expressiveness and
inference complexity is more likely to be widely adopted
and reused; also large number of IoT resources and
huge amount of data need efficient processing
 Compatibility: an ontology needs to be consistent with
those well designed, existing ontologies to ensure
compatibility wherever possible.
 Modularity: modular approach to facilitate ontology
evolution, extension and integration with external
ontologies.
Existing models for resources and data
 W3C Semantic Sensor Network Incubator
Group’s SSN ontology (mainly for sensors
and sensor networks, observation and
measurement, and platforms and systems)
 Quantity Kinds and Units
 Used together with the SSN ontology
 based on QUDV model OMG SysML(TM)
 Working group of the SysML 1.2 Revision Task
Force (RTF) and W3C Semantic Sensor Network
Incubator Group
Existing models for services
 OWL-S and WSMO are heavy weight models: practical
use?
 Minimal service model
 Deprecated
 Procedure-Oriented Service Model (POSM) and Resource-
Oriented Service Model (ROSM): two different models for
different service technologies
 Defines Operations and Messages
 No profile, no grounding
 SAWSDL: mixture of XML, XML schema, RDF and OWL
 hRESTS and SA-REST: mixture of HTML and reference
to a semantic model; sensor services are not anticipated
to have HTML
W3C’S SSN ontology
Diagram adapted from SSN report
Some existing IoT models and
ontologies
 FP7 IoT-A project’s Entity-Resource-Service
ontology
 A set of ontologies for entities, resources, devices
and services
 Based on the SSN and OWL-S ontology
 FP7 IoT.est project’s service description
framework
 A modular approach for designing a description
framework
 A set of ontologies for IoT services, testing and
QoS/QoI
IoT-A resource model
Diagram adapted from IoT-A project D2.1
IoT-A resource description
Diagram adapted from IoT-A project D2.1
IoT-A service model
Diagram adapted from IoT-A project D2.1
IoT-A service description
Diagram adapted from IoT-A project D2.1
Service modelling in IoT.est
Diagrams adapted from Iot.est D3.1
IoT.est service profile highlight
 ServiceType class represents the service
technologies: RESTful and SOAP/WSDL services.
 serviceQos and serviceQoI are defined as
subproperty of serviceParameter; they link to concepts
in the QoS/QoI ontology.
 serviceArea: the area where the service is provided;
different from the sensor observation area
 Links to the IoT resources through “exposedBy”
property
 Future extension:
 serviceNetwork, servicePlatform and
serviceDeployment
 Service lifecycle, SLA…
93
Part 4: Linked-data and semantic
enabled systems
Image source: CISCO
Linked data principles
 using URI’s as names for things: Everything is
addressed using unique URI’s.
 using HTTP URI’s to enable people to look up
those names: All the URI’s are accessible via
HTTP interfaces.
 provide useful RDF information related to
URI’s that are looked up by machine or
people;
 including RDF statements that link to other
URI’s to enable discovery of other related
concepts of the Web of Data: The URI’s are
linked to other URI’s.
Linked data in IoT
 Using URI’s as names for things;
- URI’s for naming IoT resources and data (and also streaming
data);
 Using HTTP URI’s to enable people to look up those
names;
- Web-level access to low level sensor data and real world
resource descriptions (gateway and middleware solutions);
 Providing useful RDF information related to URI’s that are looked
up by machine or people;
- publishing semantically enriched resource and data descriptions
in the form of linked RDF data;
 Including RDF statements that link to other URI’s to enable
discovery of other related things of the web of data;
- linking and associating the real world data to the existing data on
the Web;
Linked data layer for not only IoT…
Images from Stefan Decker, http://fi-ghent.fi-week.eu/files/2010/10/Linked-Data-scheme1.png; linked data diagram: http://richard.cyganiak.de/2007/10/lod/
Creating and using linked sensor data
http://ccsriottb3.ee.surrey.ac.uk:8080/IOTA/
Sensor discovery using linked sensor
data
Semantics in IoT - reality
 If we create an Ontology our data is interoperable
 Reality: there are/could be a number of ontologies for a domain
 Ontology mapping
 Reference ontologies
 Standardisation efforts
 Semantic data will make my data machine-understandable and my system
will be intelligent.
 Reality: it is still meta-data, machines don’t understand it but can interpret it. It still
does need intelligent processing, reasoning mechanism to process and interpret the
data.
 It’s a Hype! Ontologies and semantic data are too much overhead; we deal
with tiny devices in IoT.
 Reality: Ontologies are a way to share and agree on a common vocabulary and
knowledge; at the same time there are machine-interpretable and represented in
interoperable and re-usable forms;
 You don’t necessarily need to add semantic metadata in the source- it could be added
to the data at a later stage (e.g. in a gateway);
100
Part 5: Semantic Sensor Web
and
Perception
Image source: semanticweb.com; CISCO
What is the Sensor Web?
 Sensor Web is an additional layer connecting sensor
networks to the World Wide Web.
 Enables an interoperable usage of sensor resources by
enabling web based discovery, access, tasking, and
alerting.
 Enables the advancement of
cyber-physical applications through
improved situation awareness.
Why is the Sensor Web important?
 In general
 Enable tight coupling of the cyber and
physical world
 In relation to IoT
 Enable shared situation awareness (or
context) between devices/things
Bridging the Cyber-Physical Divide
Psyleron’s Mind-Lamp (Princeton U),
connections between the mind and the
physical world.
Neuro Sky's mind-controlled headset to
play a video game.
MIT’s Fluid Interface Group: wearable
device with a projector for deep
interactions with the environment
Bridging the Cyber-Physical Divide
Foursquare is an online application which
integrates a persons physical location and
social network.
Community of enthusiasts that share experiences of
self-tracking and measurement.
FitBit Community allows the
automated collection and
sharing of health-related data,
goals, and achievements
Bridging the Cyber-Physical Divide
Tweeting Sensors
sensors are becoming social
How do we design the Sensor Web?
 Integration through shared semantics
 OGC Sensor Web Enablement
 W3C SSN ontology and Semantic Annotation
 Interpretation through integration of
heterogeneous data and reasoning with prior
knowledge
 Semantic Perception/Abstraction
 Linked Open Data as prior knowledge
 Scale through distributed local interpretation
 “intelligence at the edge”
OGC Sensor Web Enablement
Role of OGC SWE
Vision of Sensor Web
 Quickly discover sensors (secure or public) that can meet
my needs – location, observables, quality, ability to task
 Obtain sensor information in a standard encoding that is
understandable by me and my software
 Readily access sensor observations in a common manner,
and in a form specific to my needs
 Task sensors, when possible, to meet my specific needs
 Subscribe to and receive alerts when a sensor measures a
particular phenomenon
Principles of Sensor Web
 Sensors will be web accessible
 Sensors and sensor data will be discoverable
 Sensors will be self-describing to humans and software
(using a standard encoding)
 Most sensor observations will be easily accessible in real
time over the web
OGC SWE Services
 Sensor Observation Service (SOS)
 access sensor information (SensorML) and sensor
observations (O&M
 Sensor Planning Service (SPS)
 task sensors or sensor systems
 Sensor Alert Service (SAS)
 asynchronous notification of sensor events (tasks,
observation of phenomena)
 Sensor Registries
 discovery of sensors and sensor data
OGC SWE Services
OGC SWE Languages
 Sensor Model Language (SensorML)
 Models and schema for describing sensor
characteristics
 Observation & Measurement (O&M)
 Models and schema for encoding sensor
observations
OCG SWE Observation
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception
Semantic Sensor Web
RDF OWL
OGC Sensor Web
Enablement
Sensor Web + Semantic Web
Semantic Web
The web of data where web content is processed
by machines, with human actors at the end of the
chain.
The web as a huge, dynamic, evolving database
of facts, rather than pages, that can be interpreted
and presented in many ways (mashups).
Fundamental importance of ontologies to describe
the fact that represents the data. RDF(S)
emphasises labelled links as the source of meaning:
essentially a graph model . A label (URI) uniquely
identifies a concept.
OWL emphasises inference as the source of
meaning: a label also refers to a package of logical
axioms with a proof theory.
Usually, the two notions of meaning fit.
Goal to combine information and services
for targeted purpose and new knowledge
Sensor Web
The internet of things made up of Wireless Sensor
Networks, RFID, stream gauges, orbiting satellites,
weather stations, GPS, traffic sensors, ocean buoys,
animal and fish tags, cameras, habitat monitors,
recording data from the physical world.
Today there are 4 billion mobile sensing devices
plus even more fixed sensors. The US National
Research Council predicts that this may grow to
trillions by 2020, and they are increasingly connected
by internet and Web protocols.
Record observations of a wide variety of
modalities: but a big part is time-series‟ of numeric
measurements.
The Open Geospatial Consortium has some web-
service standards for shared data access (Sensor
Web Enablement).
Goal is to open up access to real-time and
archival data, and to combine in applications.
So, what is a Semantic Sensor Web?
 Reduce the difficulty and open up sensor networks by:
 Allowing high-level specification of the data collection process;
 Across separately deployed sensor networks;
 Across heterogeneous sensor types; and
 Across heterogeneous sensor network platforms;
 Using high-level descriptions of sensor network capability; and
 Interfacing to data integration methods using similar query and
capability descriptions.
 To create a Web of Real Time Meaning!
W3C SSN Incubator Group
 SSN-XG commenced: 1 March 2009
 Chairs:
 Amit Sheth, Kno.e.sis Center, Wright State University
 Kerry Taylor, CSIRO
 Amit Parashar  Holger Neuhaus  Laurent Lefort, CSIRO
 Participants: 39 people from 20 organizations, including:
 Universities in: US, Germany, Finland, Spain, Britain, Ireland
 Multinationals: Boeing, Ericsson
 Small companies in semantics, communications, software
 Research institutes: DERI (Ireland), Fraunhofer (Germany),
ETRI (Korea), MBARI (US), SRI International (US), MITRE
(US), US Defense, CTIC (Spain), CSIRO (Australia), CESI
(China)
W3C SSN Incubator Group
Two main objectives:
The development of an ontology for describing
sensing resources and data, and
The extension of the SWE languages to support
semantic annotations.
Sensor Standards Landscape
SSN Ontology
 OWL 2 DL ontology
 Authored by the XG
participants
 Edited by Michael
Compton
 Driven by Use Cases
 Terminology carefully
tracked to sources through
annotation properties
 Metrics
 Classes: 117
 Properties: 148
 DL Expressivity:
SIQ(D)
SSN Ontology –
SSN Use Cases
SSN Use Cases
SSN Ontology
Stimulus-Sensor-Observation
 The SSO Ontology Design Pattern is developed following the principle of
minimal ontological commitments to make it reusable for a variety of application
areas.
 Introduces a minimal set of classes and relations centered around the notions
of stimuli, sensor, and observations. Defines stimuli as the (only) link to the
physical environment.
 Empirical science observes these stimuli using sensors to infer information
about environmental properties and construct features of interest.
SSN Ontology Modules
SSN Ontology Modules
SSN Sensor
 A sensor can do (implements) sensing: that is, a sensor is any entity that can
follow a sensing method and thus observe some Property of a
FeatureOfInterest.
 Sensors may be physical devices, computational methods, a laboratory setup
with a person following a method, or any other thing that can follow a Sensing
SSN Measurement Capability
 Collects together measurement properties (accuracy, range, precision, etc) and
the environmental conditions in which those properties hold, representing a
specification of a sensor's capability in those conditions.
SSN Observation
 An Observation is a Situation in which a Sensing method has been used to estimate or
calculate a value of a Property.
 Links to Sensing and Sensor describe what made the Observation and how; links to
Property and Feature detail what was sensed; the result is the output of a Sensor; other
metadata gives the time(s) and the quality.
 Different from OGC’s O&M, in which an “observation” is an act or event, although it also
provides the record of the event.
Alignment with DOLCE
What SSN does not model
 Sensor types and models
 Networks: communication, topology
 Representation of data and units of measurement
 Location, mobility or other dynamic behaviours
 Animate sensors
 Control and actuation
 ….
Semantic Annotation of SWE
Recommended
technique via Xlink
attributes requires no
change to SWE
xlink:href - link to
ontology individual
xlink:role - link to
ontology class
xlink:arcrole - link to
ontology object property
How do we design the Sensor Web?
 Integration through shared semantics
 OGC Sensor Web Enablement
 W3C SSN ontology and Semantic Annotation
 Interpretation through integration of
heterogeneous data and reasoning with prior
knowledge
 Semantic Perception/Abstraction
 Linked Open Data as prior knowledge
 Scale through distributed local interpretation
 “intelligence at the edge”
136
Interpretation of data
 A primary goal of interconnecting devices and
collecting/processing data from them is to
create situation awareness and enable
applications, machines, and human users to
better understand their surrounding
environments.
 The understanding of a situation, or context,
potentially enables services and applications
to make intelligent decisions and to respond to
the dynamics of their environments.
137
Observation and measurement data
Source: W3C Semantic Sensor Networks, SSN Ontology presentation, Laurent Lefort et al.
138
How to say what a sensor is and
what it measures?
Sink
node
Gateway
139
Data/Service description frameworks
 There are standards such as Sensor Web Enablement (SWE) set
developed by the Open Geospatial Consortium that are widely being
adopted in industry, government and academia.
 While such frameworks provide some interoperability, semantic
technologies are increasingly seen as key enabler for integration of
IoT data and broader Web information systems.
140
Sensor Markup Language (SensorML)
Source: http://www.mitre.org/
141
W3C SSN Ontology
makes observations
of this type
Where it is
What it
measures
units
SSN-XG ontologies
SSN-XG annotations
SSN-XG Ontology Scope
142
Semantics and IoT data
 Creating ontologies and defining data models is not enough
 tools to create and annotate data
 data handling components
 Complex models and ontologies look good, but
 design lightweight versions for constrained environments
 think of practical issues
 make it as compatible as possible and/or link it to the other existing
ontologies
 Domain knowledge and instances
 Common terms and vocabularies
 Location, unit of measurement, type, theme, …
 Link it to other resources
 Linked-data
 URIs and naming
143
Semantics and sensor data
Source: W. Wang, P. Barnaghi, "Semantic Annotation and Reasoning for Sensor Data", In proceedings of the 4th European Conference on Smart
Sensing and Context (EuroSSC2009), 2009.
144
Semantics and Linked-data
 The principles in designing the linked data are
defined as:
 using URI’s as names for things;
 using HTTP URI’s to enable people to look up
those names;
 provide useful RDF information related to URI’s
that are looked up by machine or people;
 including RDF statements that link to other URI’s
to enable discovery of other related concepts of
the Web of Data;
145
Linked Sensor data
146
Myth and reality
 #1: If we create an Ontology our data is interoperable
 Reality: there are/could be a number of ontologies for a domain
 Ontology mapping
 Reference ontologies
 Standardisation efforts
 #2: Semantic data will make my data machine-understandable and my
system will be intelligent.
 Reality: it is still meta-data, machines don’t understand it but can interpret it. It still does
need intelligent processing, reasoning mechanism to process and interpret the data.
 #3: It’s a Hype! Ontologies and semantic data are too much overhead; we
deal with tiny devices in IoT.
 Reality: Ontologies are a way to share and agree on a common vocabulary and knowledge;
at the same time there are machine-interpretable and represented in interoperable and re-
usable forms;
 You don’t necessarily need to add semantic metadata in the source- it could be added to the
data at a later stage (e.g. in a gateway);
 Legacy applications can ignore it or to be extended to work with it.
147
Processing Streaming Sensor Data
148
148
Symbolic Aggregate Approximation
(SAX)
Variable String Length and Vocabulary size.
Length: 10, VocSize: 10 Length: 10, VocSize: 4
“gijigdbabd” “cdddcbaaab”
Green Curve: consists of 100 Samples, Blue Curve: SAX
149
SAX representation
SAX Pattern (blue) with word length of 20 and a vocabulary of 10 symbols
over the original sensor time-series data (green)
P. Barnaghi, F. Ganz, C. Henson, A. Sheth, "Computing Perception from Sensor Data",
in Proc. of the IEEE Sensors 2012, Oct. 2012.
fggfffhfffffgjhghfff
jfhiggfffhfffffgjhgi
fggfffhfffffgjhghfff
150
Data Processing Framework
fggfffhfffffgjhghfff dddfffffffffffddd cccddddccccdddccc aaaacccaaaaaaaaccccdddcdcdcdcddasddd
PIR Sensor Light Sensor
Temperature
Sensor
Raw sensor data
stream
Raw sensor data
stream
Raw sensor data
stream
Attendance Phone
Hot
Temperature
Cold
Temperature
Bright
Day-time
Night-time
Office room
BA0121
On going
meeting
Window has
been left open
….
Temporal data
(extracted from
SSN descriptions)
Spatial data
(extracted from
SSN descriptions)
Thematic
data
(low level
abstractions)
Parsimonious
Covering Theory
Observations
Perceptions
Domain knowledge
SAX Patterns
Raw Sensor Data
(Annotated with SSN
Ontology)
…
….
Perception
Computation
High-level
Perceptions
151
SensorSAX
F. Ganz, P. Barnaghi, F. Carrez, “Information Abstraction for Heterogeneous Real World Internet Data”, Feb. 2013.
152
Evaluation results of abstraction
creation
F. Ganz, P. Barnaghi, F. Carrez, “Information Abstraction for Heterogeneous Real World Internet Data”, Feb. 2013.
153
Data size reduction
F. Ganz, P. Barnaghi, F. Carrez, “Information Abstraction for Heterogeneous Real World Internet Data”, Feb. 2013.
154
Enabling the Internet of Things
- Diversity range of applications
- Interacting with large number of
devices with various types
-Multiple heterogeneous networks
-Deluge of data
-Processing and interpretation of the
IoT data
155
Challenges and opportunities

Providing infrastructure
 Publishing, sharing, and access solutions on a global scale
 Indexing and discovery (data and resources)
 Aggregation and fusion
 Trust, privacy and security
 Data mining and creating actionable knowledge
 Integration into services and applications in e-health, the public sector,
retail, manufacturing and personalized apps.
 Mobile apps, location-based services, monitoring control etc.
 New business models
156
Part 6: Cognitive aspects of
knowledge representation, reasoning
and perception
Image source: CISCO
Abstraction provides the ability to interpret and synthesize information
in a way that affords effective understanding and communication of
ideas, feelings, perceptions, etc. between machines and people.
Abstraction
 People are excellent at abstraction;
of sensing and interpreting stimuli
to understand and interact with the
world.
 The process of interpreting stimuli
is called perception; and studying
this extraordinary human capability
can lead to insights for developing
effective machine perception.
Abstraction
observe perceive
conceptualization
of “real-world”
“real-world”
Abstraction
Semantic Perception/Abstraction
Fundamental Questions
What is perception, and how can
we design machines to perceive?
What can we learn from cognitive
models of perception?
Is the Semantic Web up to the task
of modeling perception?
What is Perception?
Perception is the act of
 Abstracting
 Explaining
 Discriminating
 Choosing
What can we learn from
Cognitive Models of
Perception?
 A-priori background knowledge is a key
enabler
 Perception is a cyclical, active process
Ulric Neisser (1976)Ulric Neisser (1976) Richard Gregory (1997)Richard Gregory (1997)
Is Semantic Web up to the task
of modeling perception?
Representation
Heterogeneous sensors, sensing, and observation
records
Background knowledge (observable properties,
objects/events, etc.)
Inference
Explain observations (hypothesis building)
Focus attention by seeking additional stimuli (that
discriminate between explanations)
Difficult Issues to Overcome
Perception is an inference to the best explanation
Handle streaming data
Real-time processing (or nearly)
Both people and machines are capable of observing
qualities, such as redness.
* Formally described in a sensor/ontology (SSN ontology)
observes
Observer Quality
The ability to perceive is afforded through the use of
background knowledge, relating observable qualities to
entities in the world.
* Formally described in
domain ontologies
(and knowledge bases)
inheres in
Quality
Entity
With the help of sophisticated inference, both people and
machines are also capable of perceiving entities, such as
apples.
 the ability to degrade gracefully with incomplete information
 the ability to minimize explanations based on new
information
 the ability to reason over data on the Web
 fast (tractable)
perceives
EntityPerceiver
Perceptual Inference
minimize
explanations
degrade gracefully
tractabl
e
Abductive Logic (e.g.,
PCT)
high complexity
Deductive Logic (e.g.,
OWL)
(relatively) low complexity
Web
reasoning
Perceptual Inference
(i.e., abstraction)
The ability to perceive efficiently is afforded through the
cyclical exchange of information between observers and
perceivers.
Traditionally called the
Perceptual Cycle
(or Active Perception)
sends
focus
sends
observation
Observer
Perceiver
Neisser’s Perceptual Cycle
 1970’s – Perception is an active, cyclical process of
exploration and interpretation.
- Nessier’s Perception Cycle
 1980’s – The perception cycle is driven by
background knowledge in order to generate and test
hypotheses.
- Richard Gregory (optical illusions)
 1990’s – In order to effectively test hypotheses,
some observations are more informative than others.
- Norwich’s Entropy Theory of Perception
Cognitive Theories of Perception
Key Insights
Background knowledge plays a crucial role in perception; what we
know (or think we know/believe) influences our perception of the
world.
Semantics will allow us to realize computational models of
perception based on background knowledge.
Contemporary Issues
Internet/Web expands our background knowledge to a global
scope; thus our perception is global in scope
Social networks influence our knowledge and beliefs, thus
influencing our perception
observes
inheres in
Integrated together, we have an general model – capable of
abstraction – relating observers, perceivers, and background
knowledge.
perceives
sends
focus
sends
observation
Observer Quality
EntityPerceiver
 Ontology of Perception – as an extension of SSN
 Provides abstraction of sensor data through perceptual
inference of semantically annotated data
Prior Knowledge
W3C SSN Ontology Bi-partite Graph
 Prior knowledge conformant to SSN ontology (left),
structured as a bipartite graph (right)
Explanation is the act of accounting for sensory observations (i.e.,
abstraction); often referred to as hypothesis building.
Observed Property: A property that has been observed.  
ObservedProperty ≡ ∃ssn:observedProperty—
.{o1} ⊔ … ⊔
∃ssn:observedProperty—
.{on}
 
Explanatory Feature: A feature that explains the set of observed
properties.
 
ExplanatoryFeature ≡ ∃ssn:isPropertyOf—
.{p1} ⊓ … ⊓
∃ssn:isPropertyOf—
.{pn}
 
Semantics of Explanation
Example
Assume the properties elevated blood pressure and
palpitations have been observed, and encoded in RDF
(conformant with SSN):
 
ssn:Observation(o1), ssn:observedProperty(o1, elevated blood
pressure)
ssn:Observation(o2), ssn:observedProperty(o2, palpitations)
 
Given these observations, the following
ExplanatoryFeature class is constructed:
ExplanatoryFeature ≡ ∃ssn:isPropertyOf—
.{elevated blood
pressure} ⊓ ∃ssn:isPropertyOf—
.{palpitations}
Given the KB, executing the query
ExplanatoryFeature(?y) can infer the features,
Hypertension and Hyperthyroidism, as explanations:
ExplanatoryFeature(Hypertension)
ExplanatoryFeature(Hyperthyroidism)
Semantics of Explanation
Discrimination is the act of deciding how to narrow down the multitude of
explanatory features through further observation.
Expected Property: A property is expected with respect to (w.r.t.) a set
of features if it is a property of every feature in the set.
ExpectedProperty ≡ ∃ssn:isPropertyOf.{f1} ⊓ … ⊓ ∃ssn:isPropertyOf.{fn}  
NotApplicable Property: A property is not-applicable w.r.t. a set of
features if it is not a property of any feature in the set.
NotApplicableProperty ≡ ¬∃ssn:isPropertyOf.{f1} ⊓ … ⊓
¬∃ssn:isPropertyOf.{fn}
Discriminating Property: A property is discriminating w.r.t. a set of
features if it is neither expected nor not-applicable.
DiscriminatingProperty ≡ ¬ExpectedProperty ⊓ ¬NotApplicableProperty
Semantics of Discrimination
Example
Given the explanatory features from the previous
example, Hypertension and Hyperthyroidism, the
following classes are constructed:
ExpectedProperty ≡ ∃ssn:isPropertyOf.{Hypertension} ⊓
∃ssn:isPropertyOf.{Hyperthyroidism}
 
NotApplicableProperty ≡ ¬∃ssn:isPropertyOf.{Hypertension} ⊓
¬∃ssn:isPropertyOf.{Hyperthyroidism}
 
Given the KB, executing the query
DiscriminatingProperty(?x) can infer the property clammy
skin as discriminating:
 
DiscriminatingProperty(clammy skin)
Semantics of Discrimination
How do we design the Sensor Web?
 Integration through shared semantics
 OGC Sensor Web Enablement
 W3C SSN ontology and Semantic Annotation
 Interpretation through integration of
heterogeneous data and reasoning with prior
knowledge
 Semantic Perception/Abstraction
 Linked Open Data as prior knowledge
 Scale through distributed local interpretation
 “intelligence at the edge”
Efficient Algorithms for IntellegO
 Use of OWL-DL reasoner too resource-intensive for use in
resource constrained devices (such as sensor nodes, mobile
phones, IoT devices)
 Runs out of resources for problem size (prior knowledge) > 20
concepts
 Asymptotic complexity: O(n3
) [Experimentally determined]
 To enable their use on resource-constrained devices, we now
describe algorithms for efficient inference of explanation and
discrimination.
 These algorithms use bit vector encodings and operations,
leveraging a-priori knowledge of the environment.
Efficient Algorithms for IntellegO
Semantic (RDF)
Encoding
Bit Vector Encoding
Lower
Lift
 First, developed lifting and
lowering algorithms to translate
between RDF and bit vector
encodings of observations.
Efficient Algorithms for IntellegO
Explanation Algorithm
Discrimination Algorithm
Utilize bit vector operators to efficiently
compute explanation and
discrimination
Explanation: Use of the bit vector AND
operation to discover and dismiss those
features that cannot explain the set of
observed properties
Discrimination: Use of the bit vector AND
operation to discover and indirectly assemble
those properties that discriminate between a
set of explanatory features. The discriminating
properties are those that are determined to be
neither expected nor not-applicable
Efficient Algorithms for IntellegO
Evaluation: The bit vector encodings and algorithms yield significant and
necessary computational enhancements – including asymptotic order of
magnitude improvement, with running times reduced from minutes to
milliseconds, and problem size increased from 10’s to 1000’s.
Adoption of SSN
185
Part 7: Physical-Cyber-Social
Computing
Image source: CISCO
186
J. McCarthyJ. McCarthy M. WeiserM. Weiser
D.
Engelbart
D.
Engelbart J. C. R. LickliderJ. C. R. Licklider
Similar Visions of Computing
187
If people do not believe that mathematics is simple, it is only
because they do not realize how complicated life is.
- John von Neumann
If people do not believe that mathematics is simple, it is only
because they do not realize how complicated life is.
- John von Neumann
Computational paradigms have always dealt with a
simplified representations of the real-world…
Computational paradigms have always dealt with a
simplified representations of the real-world…
Algorithms work on these simplified
representations
Algorithms work on these simplified
representations
Solutions from these algorithms are transcended back
to the real-world by humans as actions
Solutions from these algorithms are transcended back
to the real-world by humans as actions
http://ngs.ics.uci.edu/blog/?p=1501
Grand challenges in the real-world are
complex
What has changed now?
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS
2013, Madrid, Spain
There are more devices connected to the internet than
the entire human population.
There are more devices connected to the internet than
the entire human population.
Today we have around 10 billion devices connected
to the internet making it an era of IoT (Internet of Things)
Today we have around 10 billion devices connected
to the internet making it an era of IoT (Internet of Things)
http://www.cisco.com/web/about/ac79/docs/innov/IoT_IBSG_0411FINAL.pdf
189
What has not changed?What has not changed?
We need computational paradigms to tap
into the rich pulse of the human populace
We need computational paradigms to tap
into the rich pulse of the human populace
We are still working on the simpler representations of the real-
world!
We are still working on the simpler representations of the real-
world!
Represent, capture, and compute with richer and
fine-grained representations of real-world
problems
Represent, capture, and compute with richer and
fine-grained representations of real-world
problems
What should change?What should change?
PCS Computing: Asthma Scenario
190
Sensordrone – for monitoring
environmental air quality
Wheezometer – for monitoring
wheezing sounds
Can I reduce my asthma attacks at night?
What are the triggers?
What is the wheezing level?
What is the propensity toward asthma?
What is the exposure level over a day?
What is the air quality indoors?
Commute to Work
Personal
Public Health
Population Level
Closing the window at home
in the morning and taking an
alternate route to office may
lead to reduced asthma attacks
Actionable
Information
Actionable
Information
Personal, Public Health, and
Population Level Signals for Monitoring
Asthma
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS
2013, Madrid, Spain
ICS= inhaled corticosteroid, LABA = inhaled long-acting beta2-agonist, SABA= inhaled short-acting beta2-agonist ;
*consider referral to specialist
Asthma Control
and Actionable Information
Sensors and their observations
for understanding asthma
Asthma Early Warning Model
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS
2013, Madrid, Spain
192
Personal
Level
Signals
Personal
Level
Signals
Societal Level
Signals
Societal Level
Signals
(Personal Level Signals)
(Personalized
Societal Level Signal)
(Societal Level Signals)
Societal Level
Signals Relevant to
the Personal Level
Societal Level
Signals Relevant to
the Personal Level
Personal Level Sensors
(mHealth**)
QualifyQualify QuantifyQuantify
Action
Recommendation
Action
Recommendation
What are the features influencing my asthma?
What is the contribution of each of these features?
How controlled is my asthma? (risk score)
What will be my action plan to manage asthma?
StorageStorage
Societal Level Sensors
Asthma Early Warning Model (AEWM)
Query AEWM
Verify & augment
domain knowledge
Recommended
Action
Action
Justification
*http://www.slideshare.net/jain49/eventshop-120721, ** http://www.youtube.com/watch?v=btnRi64hJp4
(EventShop*)
Health Signal Extraction to
Understanding
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS
2013, Madrid, Spain
Population Level
Personal
Wheeze – Yes
Do you have tightness of chest? –Yes
ObservationsPhysical-Cyber-Social System Health Signal Extraction Health Signal Understanding
<Wheezing=Yes, time, location>
<ChectTightness=Yes, time, location>
<PollenLevel=Medium, time, location>
<Pollution=Yes, time, location>
<Activity=High, time, location>
Wheezing
ChectTightness
PollenLevel
Pollution
Activity
Wheezing
ChectTightness
PollenLevel
Pollution
Activity
RiskCategory
<PollenLevel, ChectTightness, Pollution,
Activity, Wheezing, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
.
.
.
Expert
Knowledge
Background
Knowledge
tweet reporting pollution level
and asthma attacks
Acceleration readings from
on-phone sensors
Sensor and personal
observations
Signals from personal, personal
spaces, and community spaces
Risk Category assigned by doctors
Qualify
Quantify
Enrich
Outdoor pollen and pollution
Public Health
Well Controlled - continue
Not Well Controlled – contact
nurse
Poor Controlled – contact doctor
194
J. McCarthyJ. McCarthy M. WeiserM. Weiser
D.
Engelbart
D.
Engelbart J. C. R. LickliderJ. C. R. Licklider
195
Part 8: IoT/PCS Systems:
Applications
Image source: CISCO
SSN Applications
Applications of
SSN HealthcareWeather Rescue
Traffic Fire Fighting Logistics
SSN Application: Weather
50% savings in sensing
resource requirements during
the detection of a blizzard
Order of magnitude resource
savings between storing observations
vs. relevant abstractions
Linked Sensor Data
Linked Sensor Data
(~2 Billion Statements)
Sensor Discovery Application
Query w/ location name to find nearby sensors
Real-Time Feature Streams
Demo: http://www.youtube.com/watch?
v=_ews4w_eCpg
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS
2013, Madrid, Spain
SSN Application: Fire Detection
Weather Application
SECURE: Semantics-empowered Rescue
Environment
(detect different types of fires)
DEMO: http://www.youtube.com/watch?v=in2KMkD_uqg
SSN Application: Health Care
MOBILEMD: Mobile app to help reduce re-
admission of patients with Chronic Heart Failure
SSN Application: Health Care
Passive Monitoring PhasePassive Monitoring Phase
• Abnormal heart rate
• Clammy skin
• Panic Disorder
• Hypoglycemia
• Hyperthyroidism
• Heart Attack
• Septic Shock
Observed Symptoms Possible Explanations
Passive Sensors – heart rate, galvanic skin
response
SSN Application: Health Care
Active Monitoring PhaseActive Monitoring Phase
Are you feeling lightheaded?Are you feeling lightheaded?
Are you have trouble taking deep
breaths?
Are you have trouble taking deep
breaths?
yesyes
yesyes
Have you taken your Methimazole
medication?
Have you taken your Methimazole
medication?
Do you have low blood pressure?Do you have low blood pressure?
yesyes
• Abnormal heart rate
• Clammy skin
• Lightheaded
• Trouble breathing
• Low blood pressure
• Panic Disorder
• Hypoglycemia
• Hyperthyroidism
• Heart Attack
• Septic Shock
Observed Symptoms Possible Explanations
nono
Active Sensors – blood pressure, weight scale, pulse
oxymeterDemo: http://www.youtube.com/watch?v=btnRi64hJp4
Domain ExpertsDomain Experts
ColdWeatherColdWeather
PoorVisibilityPoorVisibility
SlowTrafficSlowTraffic
IcyRoadIcyRoad
Declarative domain knowledge
Causal
knowledge
Linked Open Data
ColdWeather(YES/NO)IcyRoad (ON/OFF) PoorVisibility (YES/NO)SlowTraffic (YES/NO)
1 0 1 1
1 1 1 0
1 1 1 1
1 0 1 0
Domain ObservationsDomain Observations
Domain
Knowledge
Domain
Knowledge
Structure and parameters
205
WinterSeasonWinterSeason Otherknowledge
Correlations to causations using
Declarative knowledge on the
Semantic Web
kTraffic
Traffic jamTraffic jam
Link
Description
Link
Description
Scheduled
Event
Scheduled
Event
traffic jamtraffic jambaseball gamebaseball game
Add missing random variables
Time of dayTime of day
bad weather CapableOf slow traffic
bad weatherbad weather
Traffic data from sensors deployed on road
network in San Francisco Bay Area
time of daytime of day
traffic jamtraffic jambaseball gamebaseball game
time of daytime of day
slow trafficslow traffic
Three Operations: Complementing graphical model structure extraction
Add missing links bad weatherbad weather
traffic jamtraffic jambaseball gamebaseball game
time of daytime of day
slow trafficslow traffic
Add link direction
bad weatherbad weather
traffic jamtraffic jambaseball gamebaseball game
time of daytime of day
slow trafficslow traffic
go to baseball game Causes traffic jam
Knowledge from ConceptNet5
traffic jam CapableOfoccur twice each day
traffic jam CapableOf slow traffic
206
207
Scheduled
Event
Scheduled
Event
Active EventActive Event
Day of weekDay of week
Time of dayTime of day
delaydelay
Travel
time
Travel
time
speedspeed
volumevolume
Structure extracted form
traffic observations
(sensors + textual) using
statistical techniques
Scheduled
Event
Scheduled
Event
Active EventActive Event
Day of weekDay of week
Time of dayTime of day
delaydelay
Travel
time
Travel
time
speedspeed
volumevolume
Bad WeatherBad Weather
Enriched structure which has
link directions and new
nodes such as “Bad Weather”
potentially leading to better
delay predictions
kTraffic
SemMOB
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS
2013, Madrid, Spain
First responders have limited time to analyze
sensor (on and around them) observations.
First responders have limited time to analyze
sensor (on and around them) observations.
Fire fighters need to combine prior knowledge of
fires and its behavior, extinguishers, and floor
plan of the building for rescue strategy and operations.
Fire fighters need to combine prior knowledge of
fires and its behavior, extinguishers, and floor
plan of the building for rescue strategy and operations.
There are a variety of sensors used to monitor
vitals of firefighters, location, and poisonous gases.
There are a variety of sensors used to monitor
vitals of firefighters, location, and poisonous gases.
O2
O2
Heart rateHeart rate
COCO
CO2
CO2
GPSGPS
AccelerometerAccelerometer CompassCompass
FootstepsFootsteps
There is a team of them making
it further difficult for analysis
There is a team of them making
it further difficult for analysis
Semantic Web allow us to describe the domain, sensors, and
first responders -- apply reasoning techniques to derive actionable
insights
Semantic Web allow us to describe the domain, sensors, and
first responders -- apply reasoning techniques to derive actionable
insights
Sensors on each responder provides
different view of the event and they need
to register dynamically as the firefighters
arrive.
Sensors on each responder provides
different view of the event and they need
to register dynamically as the firefighters
arrive.
SemMOB
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS
2013, Madrid, Spain
PlatformPlatform SensingDeviceSensingDevice
System_AGT
C1
System_AGT
C1
Sensor_LSM1
1
Sensor_LSM1
1
SensorOutputSensorOutput
O2Concentraction_outputO2Concentraction_output
Measuremen
tCapability
Measuremen
tCapability
MeasurementCa
pability_LSM11
MeasurementCa
pability_LSM11
PropertyProperty
concentratio
n
concentratio
n
onPlatform
hasOutput
hasMeasurementCapability
Observes
ObservationObservation
O2ConcentrationO2Concentration
ObservedProperty
O2Obs_1505201
2
O2Obs_1505201
2
observationResultTimehttp://geonames.org/629864
0
http://geonames.org/629864
0
hasLocation
name
xsd:floatxsd:float
latlong
Dayton, Dayton-Wright Brothers Airport^^xsd:string
FeatureFeature
Demo: http://www.youtube.com/watch?v=nX11OqgtQlw
LOKILAS
(LOst Key Identification, Localization and Alert System)
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS
2013, Madrid, Spain
LOKILAS
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS
2013, Madrid, Spain
LOKILAS
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS
2013, Madrid, Spain
Future work
 Creating ontologies and defining data models are not
enough
 tools to create and annotate data
 Tools for publishing linked IoT data
 Designing lightweight versions for constrained
environments
 think of practical issues
 make it as much as possible compatible and/or link it to the other
existing ontologies
 Linking to domain knowledge and other resources
 Location, unit of measurement, type, theme, …
 Linked-data
 URIs and naming
Some of the open issues
 Efficient real-time IoT resource/service
query/discovery
 Directory
 Indexing
 Abstraction of IoT data
 Pattern extraction
 Perception creation
 IoT service composition and compensation
 Integration with existing Web services
 Service adaptation
Selected references
 Payam Barnaghi, Wei Wang, Cory Henson, Kerry Taylor, "Semantics for the Internet of Things: early progress and back to the future",
(to appear) International Journal on Semantic Web and Information Systems (special issue on sensor networks, Internet of Things and
smart devices), 2012.
 Atzori, L., Iera, A. & Morabito, G. , “The Internet of Things: A survey”, Computer Networks, Volume 54, Issue 15, 28 October 2010, 2787-
2805.
 Suparna De, Tarek Elsaleh, Payam Barnaghi , Stefan Meissner, "An Internet of Things Platform for Real-World and Digital Objects",
Journal of Scalable Computing: Practice and Experience, vol 13, no.1, 2012.
 Suparna De, Payam Barnaghi, Martin Bauer, Stefan Meissner, "Service modelling for the Internet of Things", in Proceedings of the
Conference on Computer Science and Information Systems (FedCSIS), pp.949-955, Sept. 2011.
 Cory Henson, Amit Sheth, and Krishnaprasad Thirunarayan, “Semantic Perception: Converting Sensory Observations to Abstractions”,
IEEE Internet Computing, Special Issue on Context-Aware Computing, March/April 2012.
 Payam Barnaghi, Frieder Ganz, Cory Henson, Amit Sheth, “Computing Perception from Sensor Data”, In proceedings of the 2012 IEEE
Sensors Conference, Taipei, Taiwan, October 28-31, 2012.
 Michael Compton et al, “The SSN Ontology of the W3C Semantic Sensor Network Incubator Group”, Journal of Web Semantics, 2012.
 Harshal Patni, Cory Henson, and Amit Sheth , “Linked Sensor Data”, in Proceedings of 2010 International Symposium on Collaborative
Technologies and Systems (CTS 2010), Chicago, IL, May 17-21, 2010.
 Amit Sheth, Cory Henson, and Satya Sahoo , “Semantic Sensor Web IEEE Internet Computing”, vol. 12, no. 4, July/August 2008, pp. 78-
83.
 Wei Wang, Payam Barnaghi, Gilbert Cassar, Frieder Ganz, Pirabakaran Navaratnam, "Semantic Sensor Service Networks", (to appear)
in Proceedings of the IEEE Sensors 2012 Conference, Taipei, Taiwan, October 2012.
 Wang W, De S, Toenjes R, Reetz E, Moessner K, "A Comprehensive Ontology for Knowledge Representation in the Internet of Things",
International Workshop on Knowledge Acquisition and Management in the Internet of Things (KAMIoT 2012) in conjunction with IEE
IUCC-2012, Liverpool, UK. Liverpool. 25-27 June, 2012.
Some useful links related to IoT
 Internet of Things, ITU
 http://www.itu.int/osg/spu/publications/internetofthings/InternetofThings_summary.pdf
 IoT Comic Book
 http://www.theinternetofthings.eu/content/mirko-presser-iot-comic-book
 Internet of Things Europe, http://www.internet-of-things.eu/
 Internet of Things Architecture (IOT-A)
 http://www.iot-a.eu/public/public-documents
 W3C Semantic Sensor Networks
 http://www.w3.org/2005/Incubator/ssn/XGR-ssn-20110628/

Weitere ähnliche Inhalte

Was ist angesagt?

Physical Cyber Social Computing
Physical Cyber Social ComputingPhysical Cyber Social Computing
Physical Cyber Social ComputingAmit Sheth
 
Smart Data for you and me: Personalized and Actionable Physical Cyber Social ...
Smart Data for you and me: Personalized and Actionable Physical Cyber Social ...Smart Data for you and me: Personalized and Actionable Physical Cyber Social ...
Smart Data for you and me: Personalized and Actionable Physical Cyber Social ...Amit Sheth
 
TRANSFORMING BIG DATA INTO SMART DATA: Deriving Value via Harnessing Volume, ...
TRANSFORMING BIG DATA INTO SMART DATA: Deriving Value via Harnessing Volume, ...TRANSFORMING BIG DATA INTO SMART DATA: Deriving Value via Harnessing Volume, ...
TRANSFORMING BIG DATA INTO SMART DATA: Deriving Value via Harnessing Volume, ...Amit Sheth
 
Semantics-empowered Approaches to Big Data Processing for Physical-Cyber-Soci...
Semantics-empowered Approaches to Big Data Processing for Physical-Cyber-Soci...Semantics-empowered Approaches to Big Data Processing for Physical-Cyber-Soci...
Semantics-empowered Approaches to Big Data Processing for Physical-Cyber-Soci...Artificial Intelligence Institute at UofSC
 
A Semantics-based Approach to Machine Perception
A Semantics-based Approach to Machine PerceptionA Semantics-based Approach to Machine Perception
A Semantics-based Approach to Machine PerceptionCory Andrew Henson
 
Computing for Human Experience [v4]: Keynote @ OnTheMove Federated Conferences
Computing for Human Experience [v4]: Keynote @ OnTheMove Federated ConferencesComputing for Human Experience [v4]: Keynote @ OnTheMove Federated Conferences
Computing for Human Experience [v4]: Keynote @ OnTheMove Federated ConferencesArtificial Intelligence Institute at UofSC
 
Knowledge-empowered Probabilistic Graphical Models for Physical-Cyber-Social ...
Knowledge-empowered Probabilistic Graphical Models for Physical-Cyber-Social ...Knowledge-empowered Probabilistic Graphical Models for Physical-Cyber-Social ...
Knowledge-empowered Probabilistic Graphical Models for Physical-Cyber-Social ...Artificial Intelligence Institute at UofSC
 
Transforming Big Data into Smart Data for Smart Energy: Deriving Value via ha...
Transforming Big Data into Smart Data for Smart Energy: Deriving Value via ha...Transforming Big Data into Smart Data for Smart Energy: Deriving Value via ha...
Transforming Big Data into Smart Data for Smart Energy: Deriving Value via ha...Amit Sheth
 
Physical Cyber Social Computing: An early 21st century approach to Computing ...
Physical Cyber Social Computing: An early 21st century approach to Computing ...Physical Cyber Social Computing: An early 21st century approach to Computing ...
Physical Cyber Social Computing: An early 21st century approach to Computing ...Amit Sheth
 
Semantics for Bioinformatics: What, Why and How of Search, Integration and An...
Semantics for Bioinformatics: What, Why and How of Search, Integration and An...Semantics for Bioinformatics: What, Why and How of Search, Integration and An...
Semantics for Bioinformatics: What, Why and How of Search, Integration and An...Amit Sheth
 
Hemant Purohit PhD Defense: Mining Citizen Sensor Communities for Cooperation...
Hemant Purohit PhD Defense: Mining Citizen Sensor Communities for Cooperation...Hemant Purohit PhD Defense: Mining Citizen Sensor Communities for Cooperation...
Hemant Purohit PhD Defense: Mining Citizen Sensor Communities for Cooperation...Artificial Intelligence Institute at UofSC
 
Semantic Web for 360-degree Health: State-of-the-Art & Vision for Better Inte...
Semantic Web for 360-degree Health: State-of-the-Art & Vision for Better Inte...Semantic Web for 360-degree Health: State-of-the-Art & Vision for Better Inte...
Semantic Web for 360-degree Health: State-of-the-Art & Vision for Better Inte...Amit Sheth
 
The UVA School of Data Science
The UVA School of Data ScienceThe UVA School of Data Science
The UVA School of Data SciencePhilip Bourne
 
Extracting City Traffic Events from Social Streams
 Extracting City Traffic Events from Social Streams Extracting City Traffic Events from Social Streams
Extracting City Traffic Events from Social StreamsPramod Anantharam
 
The NEEDS vs. the WANTS in IoT
The NEEDS vs. the WANTS in IoTThe NEEDS vs. the WANTS in IoT
The NEEDS vs. the WANTS in IoTPrasant Misra
 
Challenges in Analytics for BIG Data
Challenges in Analytics for BIG DataChallenges in Analytics for BIG Data
Challenges in Analytics for BIG DataPrasant Misra
 
Sensors, Signals and Sense-making in Human-Energy Relationships
Sensors, Signals and Sense-making in Human-Energy RelationshipsSensors, Signals and Sense-making in Human-Energy Relationships
Sensors, Signals and Sense-making in Human-Energy RelationshipsMartha Russell
 

Was ist angesagt? (20)

Physical Cyber Social Computing
Physical Cyber Social ComputingPhysical Cyber Social Computing
Physical Cyber Social Computing
 
Smart Data for you and me: Personalized and Actionable Physical Cyber Social ...
Smart Data for you and me: Personalized and Actionable Physical Cyber Social ...Smart Data for you and me: Personalized and Actionable Physical Cyber Social ...
Smart Data for you and me: Personalized and Actionable Physical Cyber Social ...
 
TRANSFORMING BIG DATA INTO SMART DATA: Deriving Value via Harnessing Volume, ...
TRANSFORMING BIG DATA INTO SMART DATA: Deriving Value via Harnessing Volume, ...TRANSFORMING BIG DATA INTO SMART DATA: Deriving Value via Harnessing Volume, ...
TRANSFORMING BIG DATA INTO SMART DATA: Deriving Value via Harnessing Volume, ...
 
Semantics-empowered Approaches to Big Data Processing for Physical-Cyber-Soci...
Semantics-empowered Approaches to Big Data Processing for Physical-Cyber-Soci...Semantics-empowered Approaches to Big Data Processing for Physical-Cyber-Soci...
Semantics-empowered Approaches to Big Data Processing for Physical-Cyber-Soci...
 
A Semantics-based Approach to Machine Perception
A Semantics-based Approach to Machine PerceptionA Semantics-based Approach to Machine Perception
A Semantics-based Approach to Machine Perception
 
Knoesis Student Achievement
Knoesis Student AchievementKnoesis Student Achievement
Knoesis Student Achievement
 
Computing for Human Experience [v4]: Keynote @ OnTheMove Federated Conferences
Computing for Human Experience [v4]: Keynote @ OnTheMove Federated ConferencesComputing for Human Experience [v4]: Keynote @ OnTheMove Federated Conferences
Computing for Human Experience [v4]: Keynote @ OnTheMove Federated Conferences
 
Knowledge-empowered Probabilistic Graphical Models for Physical-Cyber-Social ...
Knowledge-empowered Probabilistic Graphical Models for Physical-Cyber-Social ...Knowledge-empowered Probabilistic Graphical Models for Physical-Cyber-Social ...
Knowledge-empowered Probabilistic Graphical Models for Physical-Cyber-Social ...
 
Transforming Big Data into Smart Data for Smart Energy: Deriving Value via ha...
Transforming Big Data into Smart Data for Smart Energy: Deriving Value via ha...Transforming Big Data into Smart Data for Smart Energy: Deriving Value via ha...
Transforming Big Data into Smart Data for Smart Energy: Deriving Value via ha...
 
Physical Cyber Social Computing: An early 21st century approach to Computing ...
Physical Cyber Social Computing: An early 21st century approach to Computing ...Physical Cyber Social Computing: An early 21st century approach to Computing ...
Physical Cyber Social Computing: An early 21st century approach to Computing ...
 
Understanding City Traffic Dynamics Utilizing Sensor and Textual Observations
Understanding City Traffic Dynamics Utilizing Sensor and Textual ObservationsUnderstanding City Traffic Dynamics Utilizing Sensor and Textual Observations
Understanding City Traffic Dynamics Utilizing Sensor and Textual Observations
 
Semantics for Bioinformatics: What, Why and How of Search, Integration and An...
Semantics for Bioinformatics: What, Why and How of Search, Integration and An...Semantics for Bioinformatics: What, Why and How of Search, Integration and An...
Semantics for Bioinformatics: What, Why and How of Search, Integration and An...
 
Hemant Purohit PhD Defense: Mining Citizen Sensor Communities for Cooperation...
Hemant Purohit PhD Defense: Mining Citizen Sensor Communities for Cooperation...Hemant Purohit PhD Defense: Mining Citizen Sensor Communities for Cooperation...
Hemant Purohit PhD Defense: Mining Citizen Sensor Communities for Cooperation...
 
Semantic Web for 360-degree Health: State-of-the-Art & Vision for Better Inte...
Semantic Web for 360-degree Health: State-of-the-Art & Vision for Better Inte...Semantic Web for 360-degree Health: State-of-the-Art & Vision for Better Inte...
Semantic Web for 360-degree Health: State-of-the-Art & Vision for Better Inte...
 
The UVA School of Data Science
The UVA School of Data ScienceThe UVA School of Data Science
The UVA School of Data Science
 
Trust Management: A Tutorial
Trust Management: A TutorialTrust Management: A Tutorial
Trust Management: A Tutorial
 
Extracting City Traffic Events from Social Streams
 Extracting City Traffic Events from Social Streams Extracting City Traffic Events from Social Streams
Extracting City Traffic Events from Social Streams
 
The NEEDS vs. the WANTS in IoT
The NEEDS vs. the WANTS in IoTThe NEEDS vs. the WANTS in IoT
The NEEDS vs. the WANTS in IoT
 
Challenges in Analytics for BIG Data
Challenges in Analytics for BIG DataChallenges in Analytics for BIG Data
Challenges in Analytics for BIG Data
 
Sensors, Signals and Sense-making in Human-Energy Relationships
Sensors, Signals and Sense-making in Human-Energy RelationshipsSensors, Signals and Sense-making in Human-Energy Relationships
Sensors, Signals and Sense-making in Human-Energy Relationships
 

Ähnlich wie Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception

Data Modeling and Knowledge Engineering for the Internet of Things
Data Modeling and Knowledge Engineering for the Internet of ThingsData Modeling and Knowledge Engineering for the Internet of Things
Data Modeling and Knowledge Engineering for the Internet of ThingsPayamBarnaghi
 
Data Modelling and Knowledge Engineering for the Internet of Things
Data Modelling and Knowledge Engineering for the Internet of ThingsData Modelling and Knowledge Engineering for the Internet of Things
Data Modelling and Knowledge Engineering for the Internet of ThingsCory Andrew Henson
 
How to make data more usable on the Internet of Things
How to make data more usable on the Internet of ThingsHow to make data more usable on the Internet of Things
How to make data more usable on the Internet of ThingsPayamBarnaghi
 
Intelligent Data Processing for the Internet of Things
Intelligent Data Processing for the Internet of Things Intelligent Data Processing for the Internet of Things
Intelligent Data Processing for the Internet of Things PayamBarnaghi
 
The Internet of Things: What's next?
The Internet of Things: What's next? The Internet of Things: What's next?
The Internet of Things: What's next? PayamBarnaghi
 
Internet of Things and Large-scale Data Analytics
Internet of Things and Large-scale Data Analytics Internet of Things and Large-scale Data Analytics
Internet of Things and Large-scale Data Analytics PayamBarnaghi
 
Opportunities and Challenges of Large-scale IoT Data Analytics
Opportunities and Challenges of Large-scale IoT Data AnalyticsOpportunities and Challenges of Large-scale IoT Data Analytics
Opportunities and Challenges of Large-scale IoT Data AnalyticsPayamBarnaghi
 
The Internet of Things (IoT) and its evolution
The Internet of Things (IoT) and its evolutionThe Internet of Things (IoT) and its evolution
The Internet of Things (IoT) and its evolutionSathvik N Prasad
 
Intelligent Data Processing for the Internet of Things
Intelligent Data Processing for the Internet of Things Intelligent Data Processing for the Internet of Things
Intelligent Data Processing for the Internet of Things PayamBarnaghi
 
Information Engineering in the Age of the Internet of Things
Information Engineering in the Age of the Internet of Things Information Engineering in the Age of the Internet of Things
Information Engineering in the Age of the Internet of Things PayamBarnaghi
 
Semantic Technologies for the Internet of Things: Challenges and Opportunities
Semantic Technologies for the Internet of Things: Challenges and Opportunities Semantic Technologies for the Internet of Things: Challenges and Opportunities
Semantic Technologies for the Internet of Things: Challenges and Opportunities PayamBarnaghi
 
Dynamic Semantics for the Internet of Things
Dynamic Semantics for the Internet of Things Dynamic Semantics for the Internet of Things
Dynamic Semantics for the Internet of Things PayamBarnaghi
 
WIRELESS SENSORS INTEGRATION INTO INTERNET OF THINGS AND THE SECURITY PRIMITIVES
WIRELESS SENSORS INTEGRATION INTO INTERNET OF THINGS AND THE SECURITY PRIMITIVESWIRELESS SENSORS INTEGRATION INTO INTERNET OF THINGS AND THE SECURITY PRIMITIVES
WIRELESS SENSORS INTEGRATION INTO INTERNET OF THINGS AND THE SECURITY PRIMITIVEScsandit
 
RioInfo 2010: Seminário de Tecnologia - Mesa 1 - Integração e Convergência Ma...
RioInfo 2010: Seminário de Tecnologia - Mesa 1 - Integração e Convergência Ma...RioInfo 2010: Seminário de Tecnologia - Mesa 1 - Integração e Convergência Ma...
RioInfo 2010: Seminário de Tecnologia - Mesa 1 - Integração e Convergência Ma...Rio Info
 
Smart Cities and Data Analytics: Challenges and Opportunities
Smart Cities and Data Analytics: Challenges and Opportunities Smart Cities and Data Analytics: Challenges and Opportunities
Smart Cities and Data Analytics: Challenges and Opportunities PayamBarnaghi
 
Energy Efficiency in Internet of Things
Energy Efficiency in Internet of ThingsEnergy Efficiency in Internet of Things
Energy Efficiency in Internet of ThingsNiclas Maier
 

Ähnlich wie Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception (20)

Data Modeling and Knowledge Engineering for the Internet of Things
Data Modeling and Knowledge Engineering for the Internet of ThingsData Modeling and Knowledge Engineering for the Internet of Things
Data Modeling and Knowledge Engineering for the Internet of Things
 
Data Modelling and Knowledge Engineering for the Internet of Things
Data Modelling and Knowledge Engineering for the Internet of ThingsData Modelling and Knowledge Engineering for the Internet of Things
Data Modelling and Knowledge Engineering for the Internet of Things
 
How to make data more usable on the Internet of Things
How to make data more usable on the Internet of ThingsHow to make data more usable on the Internet of Things
How to make data more usable on the Internet of Things
 
Intelligent Data Processing for the Internet of Things
Intelligent Data Processing for the Internet of Things Intelligent Data Processing for the Internet of Things
Intelligent Data Processing for the Internet of Things
 
The Internet of Things: What's next?
The Internet of Things: What's next? The Internet of Things: What's next?
The Internet of Things: What's next?
 
Internet of Things and Large-scale Data Analytics
Internet of Things and Large-scale Data Analytics Internet of Things and Large-scale Data Analytics
Internet of Things and Large-scale Data Analytics
 
Opportunities and Challenges of Large-scale IoT Data Analytics
Opportunities and Challenges of Large-scale IoT Data AnalyticsOpportunities and Challenges of Large-scale IoT Data Analytics
Opportunities and Challenges of Large-scale IoT Data Analytics
 
The Internet of Things (IoT) and its evolution
The Internet of Things (IoT) and its evolutionThe Internet of Things (IoT) and its evolution
The Internet of Things (IoT) and its evolution
 
Intelligent Data Processing for the Internet of Things
Intelligent Data Processing for the Internet of Things Intelligent Data Processing for the Internet of Things
Intelligent Data Processing for the Internet of Things
 
Information Engineering in the Age of the Internet of Things
Information Engineering in the Age of the Internet of Things Information Engineering in the Age of the Internet of Things
Information Engineering in the Age of the Internet of Things
 
Semantic Technologies for the Internet of Things: Challenges and Opportunities
Semantic Technologies for the Internet of Things: Challenges and Opportunities Semantic Technologies for the Internet of Things: Challenges and Opportunities
Semantic Technologies for the Internet of Things: Challenges and Opportunities
 
Iot Report
Iot ReportIot Report
Iot Report
 
Internet Of Things (IOT)
Internet Of Things (IOT)Internet Of Things (IOT)
Internet Of Things (IOT)
 
Dynamic Semantics for the Internet of Things
Dynamic Semantics for the Internet of Things Dynamic Semantics for the Internet of Things
Dynamic Semantics for the Internet of Things
 
WIRELESS SENSORS INTEGRATION INTO INTERNET OF THINGS AND THE SECURITY PRIMITIVES
WIRELESS SENSORS INTEGRATION INTO INTERNET OF THINGS AND THE SECURITY PRIMITIVESWIRELESS SENSORS INTEGRATION INTO INTERNET OF THINGS AND THE SECURITY PRIMITIVES
WIRELESS SENSORS INTEGRATION INTO INTERNET OF THINGS AND THE SECURITY PRIMITIVES
 
Grid computing
Grid computingGrid computing
Grid computing
 
RioInfo 2010: Seminário de Tecnologia - Mesa 1 - Integração e Convergência Ma...
RioInfo 2010: Seminário de Tecnologia - Mesa 1 - Integração e Convergência Ma...RioInfo 2010: Seminário de Tecnologia - Mesa 1 - Integração e Convergência Ma...
RioInfo 2010: Seminário de Tecnologia - Mesa 1 - Integração e Convergência Ma...
 
Smart Cities and Data Analytics: Challenges and Opportunities
Smart Cities and Data Analytics: Challenges and Opportunities Smart Cities and Data Analytics: Challenges and Opportunities
Smart Cities and Data Analytics: Challenges and Opportunities
 
IOT_PPT1.pdf
IOT_PPT1.pdfIOT_PPT1.pdf
IOT_PPT1.pdf
 
Energy Efficiency in Internet of Things
Energy Efficiency in Internet of ThingsEnergy Efficiency in Internet of Things
Energy Efficiency in Internet of Things
 

Kürzlich hochgeladen

VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 

Kürzlich hochgeladen (20)

VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 

Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception

  • 1. Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception Pramod Anantharam1 , Payam Barnaghi2 , Amit Sheth1 1 Kno.e.sis Center, Wright State University 2 Centre for Communication Systems Research, University of Surrey Madrid, Spain, June 12-14, 2013 http://aida.ii.uam.es/wims13/keynotes.php#tutorials Special Thanks to: Cory Henson, Kno.e.sis Research Center, Wright State University Wei Wang, Centre for Communication Systems Research, University of Surrey
  • 2. 2 Part 1: Introduction to Internet of “Things” Image source: CISCO
  • 3. Internet of Things  “sensors and actuators embedded in physical objects — from containers to pacemakers — are linked through both wired and wireless networks to the Internet.”  “When objects in the IoT can sense the environment, interpret the data, and communicate with each other, they become tools for understanding complexity and for responding to events and irregularities swiftly” source: http://www.iot2012.org/
  • 4. “Thing” connected to the internet “In 2013 cumulative shipments of Bluetooth-enabled devices will surpass 10 billion and Wi-Fi enabled devices will surpass 10 billion cumulative shipments in 2015,” - Peter Cooney, wireless analyst with ABI Research “73% of Countries with 4G Services Have Dropped Their 4G Tariffs by an Average of 30% in the Past 6 Months”1 - ABI Research (07 Feb 2013) 1http://www.abiresearch.com/
  • 5. 5 Network connected Things and Devices Image courtesy: CISCO
  • 6. 6 Sensor devices are becoming widely available - Programmable devices - Off-the-shelf gadgets/tools
  • 7. 7 More “Things” are being connected Home/daily-life devices Business and Public infrastructure Health-care …
  • 8. 8 People Connecting to Things Motion sensor Motion sensor Motion sensor ECG sensor Internet
  • 9. 9 Things Connecting to Things - Complex and heterogeneous resources and networks
  • 10. 10 Wireless Sensor Networks (WSN) Sink node Gateway Core network e.g. InternetGateway End-user Computer services - The networks typically run Low Power Devices - Consist of one or more sensors, could be different type of sensors (or actuators)
  • 11. 11 Key characteristics of IoT devices  Often inexpensive sensors (actuators) equipped with a radio transceiver for various applications, typically low data rate ~ 10-250 kbps.  Deployed in large numbers  The sensors should coordinate to perform the desired task.  The acquired information (periodic or event-based) is reported back to the information processing centre (or sometimes in-network processing is required)  Solutions are application-dependent. 11
  • 12. 12 Beyond conventional sensors  Human as a sensor (citizen sensors)  e.g. tweeting real world data and/or events  Virtual (software) sensors  e.g. Software agents/services generating/representing data Road block, A3 Road block, A3 Suggest a different route
  • 13. 13 Cyber, Physical and Social Data
  • 14. 14 Citizen Sensors Source: How Crisis Mapping Saved Lives in Haiti, Ushahidi Haiti Project (UHP).
  • 17. 17 Making Sense of Data In the next few years, sensor networks will produce 10-20 time the amount of data generated by social media. (source: GigaOmni Media)
  • 18. 18 Things, Data, and lots of it image courtesy: Smarter Data - I.03_C by Gwen Vanhee
  • 19. 19 Big Data and IoT  "Big data" is a term applied to data sets whose size is beyond the ability of commonly used software tools to capture, manage, and process the data within a tolerable elapsed time. Big data sizes are a constantly moving target, as of 2012 ranging from a few dozen terabytes to many petabytes of data in a single data set.” (wikipedia)  Every day, we create 2.5 quintillion bytes of data — so much that 90% of the data in the world today has been created in the last two years alone. (source IBM)
  • 20. 20 The seduction of data  Turn 12 terabytes of Tweets created each day into sentiment analysis related to different events/occurrences or relate them to products and services.  Convert (billions of) smart meter readings to better predict and balance power consumption.  Analyze thousands of traffic, pollution, weather, congestion, public transport and event sensory data to provide better traffic management.  Monitor patients, elderly care and much more… Adapted from: What is Bog Data?, IBM
  • 21. 21 Do we need all these data?
  • 22. 22 “Raw data is both an oxymoron and bad data” Geoff Bowker, 2005 ource: Kate Crawford, "Algorithmic Illusions: Hidden Biases of Big Data", Strata 2013.
  • 23. 23 IoT Data in the Cloud Image courtesy: http://images.mathrubhumi.com http://www.anacostiaws.org/userfiles/image/Blog-Photos/river2.jpg
  • 24. 24 Perceptions and Intelligence Data Information Knowledge Wisdom Raw sensory data Structured data (with semantics) Abstraction and perceptions Actionable intelligence
  • 25. 25 Change in communication paradigm Sink node Gateway Core network e.g. Internet End-user Data Sender Data Receiver A sample data communication in conventional networks A sample data communication in WSN Fire! Some bits 01100011100
  • 26. 26  Collaboration and in-network processing  In some applications a single sensor node is not able to handle the given task or provide the requested information.  Instead of sending the information form various source to an external network/node, the information can be processed in the network itself.  e.g. data aggregation, summarisation and then propagating the processed data with reduced size (hence improving energy efficiency by reducing the amount of data to be transmitted).  Data-centric  Conventional networks often focus on sending data between two specific nodes each equipped with an address.  Here what is important is data and the observations and measurements not the node that provides it. Required mechanisms
  • 27. “People want answers, not numbers” (Steven Glaser, UC Berkley) Sink node Gateway Core network e.g. Internet What is the temperature at home?Freezing!
  • 28. 28 IoT Data alone is not enough  Domain knowledge  Machine interpretable meta data  Delivery, sharing and representation services  Query, discovery, aggregation services  Publish, subscribe, notification, and access interfaces/services
  • 29. 29 Storing, Handling and Processing the Data Image courtesy: IEEE Spectrum
  • 30. 30 IoT Data Challenges  Discovery: finding appropriate device and data sources  Access: Availability and (open) access to IoT resources and data  Search: querying for data  Integration: dealing with heterogeneous device, networks and data  Interpretation: translating data to knowledge usable by people and applications  Scalability: dealing with large number of devices and myriad of data and computational complexity of interpreting the data.
  • 31. Energy consumption of the nodes  Batteries have small capacity and recharging could be complex (if not impossible) in some cases.  The main consumers of the energy are: the controller, radio, to some extent memory and depending on the type, the sensor(s).  A controller can go to:  “active”, “idle” and “sleep”  A radio modem could turn transmitter, receiver, or both on or off,  sensors and memory can be also turned on and off.
  • 32. Actuators Stepper Motor [1] Image credits: [1] http://directory.ac/telco-motion.html [2] http://bruce.pennypacker.org/category/theater/ [3] http://www.busytrade.com/products/1195641/TG-100-Linear-Actuator.html [4] http://www.arbworx.com/services/fencing-garden-fencing/ [2] [3][4]
  • 33. Wireless Sensor Networks (WSN)- gateway connection
  • 35. What are the main issues?  Heterogeneity  Interoperability  Mobility  Energy efficiency  Scalability  Security
  • 36. What is important?  Robustness  Quality of Service  Scalability  Seamless integration  Security, privacy, Trust
  • 37. In-network processing  Mobile Ad-hoc Networks are supposed to deliver bits from one end to the other  WSNs, on the other end, are expected to provide information, not necessarily original bits  Gives addition options  E.g., manipulate or process the data in the network  Main example: aggregation  Applying aggregation functions to a obtain an average value of measurement data  Typical functions: minimum, maximum, average, sum, …  Not amenable functions: median source: Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor Networks Holger Karl, Andreas Willig, chapter 3, Wiley, 2005 .
  • 38. In-network processing- example Applying Symbolic Aggregate Approximation (SAX) SAX Pattern (blue) with word length of 20 and a vocabulary of 10 symbols over the original sensor time-series data (green)
  • 39. Data-centric networking  In typical networks (including ad hoc networks), network transactions are addressed to the identities of specific nodes  A “node-centric” or “address-centric” networking paradigm  In a redundantly deployed sensor networks, specific source of an event, alarm, etc. might not be important  Redundancy: e.g., several nodes can observe the same area  Thus: focus networking transactions on the data directly instead of their senders and transmitters ! data-centric networking  Principal design change source: Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor Networks Holger Karl, Andreas Willig, chapter 3, Wiley, 2005 .
  • 40. Implementation options for data-centric networking  Overlay networks & distributed hash tables (DHT)  Hash table: content-addressable memory  Retrieve data from an unknown source, like in peer-to-peer networking – with efficient implementation  Some disparities remain  Static key in DHT, dynamic changes in WSN  DHTs typically ignore issues like hop count or distance between nodes when performing a lookup operation  Publish/subscribe  Different interaction paradigm  Nodes can publish data, can subscribe to any particular kind of data  Once data of a certain type has been published, it is delivered to all subscribes  Subscription and publication are decoupled in time; subscriber and published are agnostic of each other (decoupled in identity);  There is concepts of Semantic Sensor Networks- to annotate sensor resources and observation and measurement data! Adapted from: Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor Networks Holger Karl, Andreas Willig, chapter 3, Wiley, 2005 .
  • 41. IoT and Semantic technologies  The sensors (and in general “Things”) are increasingly being integrated into the Internet/Web.  This can be supported by embedded devices that directly support IP and web-based connection (e.g. 6LowPAN and CoAp) or devices that are connected via gateway components.  Broadening the IoT to the concept of “Web of Things”  There are already Sensor Web Enablement (SWE) standards developed by the Open Geospatial Consortium that are widely being adopted in industry, government and academia.  While such frameworks provide some interoperability, semantic technologies are increasingly seen as key enabler for integration of IoT data and broader Web information systems.
  • 42. Semantics and IoT resources and data  Semantics are machine-interpretable metadata (for mark-up), logical inference mechanisms, query mechanism, linked data solutions  For IoT this means:  ontologies for: resource (e.g. sensors), observation and measurement data (e.g. sensor readings), domain concepts (e.g. unit of measurement, location), services (e.g. IoT services) and other data sources (e.g. those available on linked open data)  Semantic annotation should also supports data represented using existing forms  Reasoning /processing to infer relationships and hierarchies between different resources, data  Semantics (/ontologies) as meta-data (to describe the IoT resources/data) / knowledge bases (domain knowledge).
  • 44. SSW Introduction lives in has pet is ahas pet Perso n Perso n Anima l Anima l Concrete Facts Resource Description Framework Concrete Facts Resource Description Framework Semantic Web (according to Farside) General Knowledge Web Ontology Language General Knowledge Web Ontology Language “Now! – That should clear up a few things around is a
  • 47. Linked Open Data ~ 50 Billion Statements~ 50 Billion Statements
  • 48. SW is moving from academia to industry
  • 49. In the last few years, we have seen many successes … Knowledge Graph Watson Apple Siri
  • 53. Enabling the Internet of Things Situational awareness enables:  Devices/things to function and adapt within their environment  Devices/things to work together
  • 54. Sensor systems are too often stovepiped. Closed centralized management of sensing resources Closed inaccessible data and sensors
  • 55. We want to set this data free With freedom comes responsibility Discovery, access, and search Integration and interpretation Scalability
  • 56. Drowning in Data A cross-country flight from New York to Los Angeles on a Boeing 737 plane generates a massive 240 terabytes of data - GigaOmni Media
  • 58. Challenges To fulfill this vision, there are difficult challenges to overcome such as the discovery, access, search, integration, and interpretation of sensors and sensor data at scale Discovery finding appropriate sensing resources and data sources Access sensing resources and data are open and available Search querying for sensor data Integration dealing with heterogeneous sensors and sensor data Interpretation translating sensor data to knowledge usable by people and applications Scalability dealing with data overload and computational complexity of interpreting the data
  • 59. Solution Semantic Sensor Web Internet Computing, July/Aug. 2008 Uses the Web as platform for managing sensor resources and data Uses semantic technologies for representing data and knowledge, integration, and interpretation
  • 60. Solution Discovery, access, and search  Using standard Web services  OGC Sensor Web Enablement
  • 61. Solution Integration  Using shared domain models / data representation  OGC Sensor Web Enablement  W3C Semantic Sensor Networks
  • 62. Solution Interpretation  Abstraction – converting low-level data to high-level knowledge  Machine Perception – w/ prior knowledge and abductive reasoning  IntellegO – Ontology of Perception
  • 63. Solution Scalability  Data overload – sensors produce too much data  Computational complexity of semantic interpretation  “Intelligence at the edge” – local and distributed integration and interpretation of sensor data
  • 64. SSW Adoption and Applications
  • 65. 65 Part 2: Data and knowledge modelling requirements, semantic annotation Image source: CISCO
  • 66. Recall of the Internet of Things  A primary goal of interconnecting devices and collecting/processing data from them is to create situation awareness and enable applications, machines, and human users to better understand their surrounding environments.  The understanding of a situation, or context, potentially enables services and applications to make intelligent decisions and to respond to the dynamics of their environments.Barnaghi et al 2012, “Semantics for the Internet of Things: early progress and back to the future”
  • 67. IoT challenges  Numbers of devices and different users and interactions required.  Challenge: Scalability  Heterogeneity of enabling devices and platforms  Challenge: Interoperability  Low power sensors, wireless transceivers, communication, and networking for M2M  Challenge: Efficiency in communications  Huge volumes of data emerging from the physical world, M2M and new communications  Challenge: Processing and mining the data, Providing secure access and preserving and controlling privacy.  Timeliness of data  Challenge: Freshness of the data and supporting temporal requirements in accessing the data  Ubiquity  Challenge: addressing mobility, ad-hoc access and service continuity  Global access and discovery  Challenge: Naming, Resolution and discovery
  • 68. IoT: one paradigm, many visions Diagram adapted from L. Atzori et al., 2010, “the Internet of Things: a Survey”
  • 69. Semantic oriented vision  “The object unique addressing and the representation and storing of the exchanged information become the most challenging issues, bringing directly to a ‘‘Semantic oriented”, perspective of IoT”, [Atzori et al., 2010]  Data collected by different sensors and devices is usually multi-modal (temperature, light, sound, video, etc.) and diverse in nature (quality of data can vary with different devices through time and it is mostly location and time dependent [Barnaghi et al, 2012]  some of challenging issues: representation, storage, and search/discovery/query/addressing, and processing IoT resources and data.
  • 70. What is expected?  Unified access to data: unified descriptions  Deriving additional knowledge (data mining)  Reasoning support and association to other entities and resources  Self-descriptive data an re-usable knowledge  In general: Large-scale platforms to support discovery and access to the resources, to enable autonomous interactions with the resources, to provide self- descriptive data and association mechanisms to reason the emerging data and to integrate it into the existing applications and services.
  • 71. Semantic technologies and IoT  There are already Sensor Web Enablement (SWE) standards developed by the Open Geospatial Consortium that are widely adopted.  While such frameworks provide certain levels of interoperability, semantic technologies are seen as key enabler for integration of IoT data and and existing business information systems.  Semantic technologies provide potential support for:  Interoperability and machine automation  IoT resource and data annotation, logical inference, query and discovery, linked IoT data
  • 72. Identify IoT domain concepts  Users  Physical entities  Virtual entities  Devices  Resource  Services  … Diagram adapted from IoT-A project D2.1
  • 73. IoT domain concepts - Entity  Physical entities (or entity of interests): objects in the physical world, features of interest that are of interests to users (human users or any digital artifacts).  Virtual entities: virtual representation of the physical entities.  Entities are the main focus of interactions between humans and/or software agents.  This interaction is made possible by a hardware component called Device. Definition adapted from De et al, 2012, “Service modeling for the Internet of Things”
  • 74. IoT domain concepts – Device, Resource and Service  A Device mediates the interactions between users and entities.  The software component that provides information on the entity or enables controlling of the device, is called a Resource.  A Service provides well-defined and standardised interfaces, offering all necessary functionalities for interacting with entities and related processes. Definition adapted from De et al, 2012, “Service modeling for the Internet of Things”
  • 75. Other concepts need to considered  Gateways  Directories  Platforms  Systems  Subsystems  …  Relationships among them  And links to existing knowledge base and linked data
  • 76. Don’t forget the IoT data  Sensors and devices provide observation and measurement data about the physical world objects which also need to be semantically described and can be related to an event, situation in the physical world.  The processing of data into knowledge/ perception and using it for decision making, automated control, etc.  Huge amount of data from our physical world that need to be  Annotated  Published  Stored (temporary or for longer term)  Discovered  Accessed  Proceeded  Utilised in different applications
  • 77. Semantics for IoT resources and data  Semantics are machine-interpretable metadata, logical inference mechanisms, query and search mechanism, linked data…  For IoT this means:  ontologies for: resource (e.g. sensors), observation and measurement data (e.g. sensor readings), services (e.g. IoT services), domain concepts (e.g. unit of measurement, location) and other data sources (e.g. those available on linked open data)  Semantic annotation should also supports data represented using existing forms  Reasoning/processing to infer relationships between different resources and services, detecting patterns from IoT data
  • 78. Characteristics of IoT resources  Extraordinarily large number  Limited computing capabilities  Limited memory  Resource constrained environments (e.g., battery life, signal coverage)  Location is important  Dynamism in the physical environments  Unexpected disruption of services  …
  • 79. Characteristics of IoT data  Stream data (depends on time)  Transient nature  Almost always related to a phenomenon or quality in our physical environments  Large amount  Quality in many situations cannot be assured (e.g., accuracy and precision)  Abstraction levels (e.g., raw, inferred or derived)  …
  • 80. Utilise semantics  Find all available resources (which can provide data) and data related to “Room A” (which is an object in the linked data)?  What is “Room A”? What is its location? returns “location” data  What type of data is available for “Room A” or that “location”? (sensor category types)  Predefined Rules can be applied based on available data  (TempRoom_A > 80°C) AND (SmokeDetectedRoom_A position==TRUE)  FireEventRoom_A  Learning these rules needs data mining or pattern recognition techniques
  • 81. 81 Part 3: Semantics and data modelling for IoT Image source: CISCO
  • 82. Semantic modelling  Lightweight: experiences show that a lightweight ontology model that well balances expressiveness and inference complexity is more likely to be widely adopted and reused; also large number of IoT resources and huge amount of data need efficient processing  Compatibility: an ontology needs to be consistent with those well designed, existing ontologies to ensure compatibility wherever possible.  Modularity: modular approach to facilitate ontology evolution, extension and integration with external ontologies.
  • 83. Existing models for resources and data  W3C Semantic Sensor Network Incubator Group’s SSN ontology (mainly for sensors and sensor networks, observation and measurement, and platforms and systems)  Quantity Kinds and Units  Used together with the SSN ontology  based on QUDV model OMG SysML(TM)  Working group of the SysML 1.2 Revision Task Force (RTF) and W3C Semantic Sensor Network Incubator Group
  • 84. Existing models for services  OWL-S and WSMO are heavy weight models: practical use?  Minimal service model  Deprecated  Procedure-Oriented Service Model (POSM) and Resource- Oriented Service Model (ROSM): two different models for different service technologies  Defines Operations and Messages  No profile, no grounding  SAWSDL: mixture of XML, XML schema, RDF and OWL  hRESTS and SA-REST: mixture of HTML and reference to a semantic model; sensor services are not anticipated to have HTML
  • 85. W3C’S SSN ontology Diagram adapted from SSN report
  • 86. Some existing IoT models and ontologies  FP7 IoT-A project’s Entity-Resource-Service ontology  A set of ontologies for entities, resources, devices and services  Based on the SSN and OWL-S ontology  FP7 IoT.est project’s service description framework  A modular approach for designing a description framework  A set of ontologies for IoT services, testing and QoS/QoI
  • 87. IoT-A resource model Diagram adapted from IoT-A project D2.1
  • 88. IoT-A resource description Diagram adapted from IoT-A project D2.1
  • 89. IoT-A service model Diagram adapted from IoT-A project D2.1
  • 90. IoT-A service description Diagram adapted from IoT-A project D2.1
  • 91. Service modelling in IoT.est Diagrams adapted from Iot.est D3.1
  • 92. IoT.est service profile highlight  ServiceType class represents the service technologies: RESTful and SOAP/WSDL services.  serviceQos and serviceQoI are defined as subproperty of serviceParameter; they link to concepts in the QoS/QoI ontology.  serviceArea: the area where the service is provided; different from the sensor observation area  Links to the IoT resources through “exposedBy” property  Future extension:  serviceNetwork, servicePlatform and serviceDeployment  Service lifecycle, SLA…
  • 93. 93 Part 4: Linked-data and semantic enabled systems Image source: CISCO
  • 94. Linked data principles  using URI’s as names for things: Everything is addressed using unique URI’s.  using HTTP URI’s to enable people to look up those names: All the URI’s are accessible via HTTP interfaces.  provide useful RDF information related to URI’s that are looked up by machine or people;  including RDF statements that link to other URI’s to enable discovery of other related concepts of the Web of Data: The URI’s are linked to other URI’s.
  • 95. Linked data in IoT  Using URI’s as names for things; - URI’s for naming IoT resources and data (and also streaming data);  Using HTTP URI’s to enable people to look up those names; - Web-level access to low level sensor data and real world resource descriptions (gateway and middleware solutions);  Providing useful RDF information related to URI’s that are looked up by machine or people; - publishing semantically enriched resource and data descriptions in the form of linked RDF data;  Including RDF statements that link to other URI’s to enable discovery of other related things of the web of data; - linking and associating the real world data to the existing data on the Web;
  • 96. Linked data layer for not only IoT… Images from Stefan Decker, http://fi-ghent.fi-week.eu/files/2010/10/Linked-Data-scheme1.png; linked data diagram: http://richard.cyganiak.de/2007/10/lod/
  • 97. Creating and using linked sensor data http://ccsriottb3.ee.surrey.ac.uk:8080/IOTA/
  • 98. Sensor discovery using linked sensor data
  • 99. Semantics in IoT - reality  If we create an Ontology our data is interoperable  Reality: there are/could be a number of ontologies for a domain  Ontology mapping  Reference ontologies  Standardisation efforts  Semantic data will make my data machine-understandable and my system will be intelligent.  Reality: it is still meta-data, machines don’t understand it but can interpret it. It still does need intelligent processing, reasoning mechanism to process and interpret the data.  It’s a Hype! Ontologies and semantic data are too much overhead; we deal with tiny devices in IoT.  Reality: Ontologies are a way to share and agree on a common vocabulary and knowledge; at the same time there are machine-interpretable and represented in interoperable and re-usable forms;  You don’t necessarily need to add semantic metadata in the source- it could be added to the data at a later stage (e.g. in a gateway);
  • 100. 100 Part 5: Semantic Sensor Web and Perception Image source: semanticweb.com; CISCO
  • 101. What is the Sensor Web?  Sensor Web is an additional layer connecting sensor networks to the World Wide Web.  Enables an interoperable usage of sensor resources by enabling web based discovery, access, tasking, and alerting.  Enables the advancement of cyber-physical applications through improved situation awareness.
  • 102. Why is the Sensor Web important?  In general  Enable tight coupling of the cyber and physical world  In relation to IoT  Enable shared situation awareness (or context) between devices/things
  • 103. Bridging the Cyber-Physical Divide Psyleron’s Mind-Lamp (Princeton U), connections between the mind and the physical world. Neuro Sky's mind-controlled headset to play a video game. MIT’s Fluid Interface Group: wearable device with a projector for deep interactions with the environment
  • 104. Bridging the Cyber-Physical Divide Foursquare is an online application which integrates a persons physical location and social network. Community of enthusiasts that share experiences of self-tracking and measurement. FitBit Community allows the automated collection and sharing of health-related data, goals, and achievements
  • 105. Bridging the Cyber-Physical Divide Tweeting Sensors sensors are becoming social
  • 106. How do we design the Sensor Web?  Integration through shared semantics  OGC Sensor Web Enablement  W3C SSN ontology and Semantic Annotation  Interpretation through integration of heterogeneous data and reasoning with prior knowledge  Semantic Perception/Abstraction  Linked Open Data as prior knowledge  Scale through distributed local interpretation  “intelligence at the edge”
  • 107. OGC Sensor Web Enablement
  • 108. Role of OGC SWE
  • 109. Vision of Sensor Web  Quickly discover sensors (secure or public) that can meet my needs – location, observables, quality, ability to task  Obtain sensor information in a standard encoding that is understandable by me and my software  Readily access sensor observations in a common manner, and in a form specific to my needs  Task sensors, when possible, to meet my specific needs  Subscribe to and receive alerts when a sensor measures a particular phenomenon
  • 110. Principles of Sensor Web  Sensors will be web accessible  Sensors and sensor data will be discoverable  Sensors will be self-describing to humans and software (using a standard encoding)  Most sensor observations will be easily accessible in real time over the web
  • 111. OGC SWE Services  Sensor Observation Service (SOS)  access sensor information (SensorML) and sensor observations (O&M  Sensor Planning Service (SPS)  task sensors or sensor systems  Sensor Alert Service (SAS)  asynchronous notification of sensor events (tasks, observation of phenomena)  Sensor Registries  discovery of sensors and sensor data
  • 113. OGC SWE Languages  Sensor Model Language (SensorML)  Models and schema for describing sensor characteristics  Observation & Measurement (O&M)  Models and schema for encoding sensor observations
  • 116. Semantic Sensor Web RDF OWL OGC Sensor Web Enablement
  • 117. Sensor Web + Semantic Web Semantic Web The web of data where web content is processed by machines, with human actors at the end of the chain. The web as a huge, dynamic, evolving database of facts, rather than pages, that can be interpreted and presented in many ways (mashups). Fundamental importance of ontologies to describe the fact that represents the data. RDF(S) emphasises labelled links as the source of meaning: essentially a graph model . A label (URI) uniquely identifies a concept. OWL emphasises inference as the source of meaning: a label also refers to a package of logical axioms with a proof theory. Usually, the two notions of meaning fit. Goal to combine information and services for targeted purpose and new knowledge Sensor Web The internet of things made up of Wireless Sensor Networks, RFID, stream gauges, orbiting satellites, weather stations, GPS, traffic sensors, ocean buoys, animal and fish tags, cameras, habitat monitors, recording data from the physical world. Today there are 4 billion mobile sensing devices plus even more fixed sensors. The US National Research Council predicts that this may grow to trillions by 2020, and they are increasingly connected by internet and Web protocols. Record observations of a wide variety of modalities: but a big part is time-series‟ of numeric measurements. The Open Geospatial Consortium has some web- service standards for shared data access (Sensor Web Enablement). Goal is to open up access to real-time and archival data, and to combine in applications.
  • 118. So, what is a Semantic Sensor Web?  Reduce the difficulty and open up sensor networks by:  Allowing high-level specification of the data collection process;  Across separately deployed sensor networks;  Across heterogeneous sensor types; and  Across heterogeneous sensor network platforms;  Using high-level descriptions of sensor network capability; and  Interfacing to data integration methods using similar query and capability descriptions.  To create a Web of Real Time Meaning!
  • 119. W3C SSN Incubator Group  SSN-XG commenced: 1 March 2009  Chairs:  Amit Sheth, Kno.e.sis Center, Wright State University  Kerry Taylor, CSIRO  Amit Parashar  Holger Neuhaus  Laurent Lefort, CSIRO  Participants: 39 people from 20 organizations, including:  Universities in: US, Germany, Finland, Spain, Britain, Ireland  Multinationals: Boeing, Ericsson  Small companies in semantics, communications, software  Research institutes: DERI (Ireland), Fraunhofer (Germany), ETRI (Korea), MBARI (US), SRI International (US), MITRE (US), US Defense, CTIC (Spain), CSIRO (Australia), CESI (China)
  • 120. W3C SSN Incubator Group Two main objectives: The development of an ontology for describing sensing resources and data, and The extension of the SWE languages to support semantic annotations.
  • 122. SSN Ontology  OWL 2 DL ontology  Authored by the XG participants  Edited by Michael Compton  Driven by Use Cases  Terminology carefully tracked to sources through annotation properties  Metrics  Classes: 117  Properties: 148  DL Expressivity: SIQ(D) SSN Ontology –
  • 126. Stimulus-Sensor-Observation  The SSO Ontology Design Pattern is developed following the principle of minimal ontological commitments to make it reusable for a variety of application areas.  Introduces a minimal set of classes and relations centered around the notions of stimuli, sensor, and observations. Defines stimuli as the (only) link to the physical environment.  Empirical science observes these stimuli using sensors to infer information about environmental properties and construct features of interest.
  • 129. SSN Sensor  A sensor can do (implements) sensing: that is, a sensor is any entity that can follow a sensing method and thus observe some Property of a FeatureOfInterest.  Sensors may be physical devices, computational methods, a laboratory setup with a person following a method, or any other thing that can follow a Sensing
  • 130. SSN Measurement Capability  Collects together measurement properties (accuracy, range, precision, etc) and the environmental conditions in which those properties hold, representing a specification of a sensor's capability in those conditions.
  • 131. SSN Observation  An Observation is a Situation in which a Sensing method has been used to estimate or calculate a value of a Property.  Links to Sensing and Sensor describe what made the Observation and how; links to Property and Feature detail what was sensed; the result is the output of a Sensor; other metadata gives the time(s) and the quality.  Different from OGC’s O&M, in which an “observation” is an act or event, although it also provides the record of the event.
  • 133. What SSN does not model  Sensor types and models  Networks: communication, topology  Representation of data and units of measurement  Location, mobility or other dynamic behaviours  Animate sensors  Control and actuation  ….
  • 134. Semantic Annotation of SWE Recommended technique via Xlink attributes requires no change to SWE xlink:href - link to ontology individual xlink:role - link to ontology class xlink:arcrole - link to ontology object property
  • 135. How do we design the Sensor Web?  Integration through shared semantics  OGC Sensor Web Enablement  W3C SSN ontology and Semantic Annotation  Interpretation through integration of heterogeneous data and reasoning with prior knowledge  Semantic Perception/Abstraction  Linked Open Data as prior knowledge  Scale through distributed local interpretation  “intelligence at the edge”
  • 136. 136 Interpretation of data  A primary goal of interconnecting devices and collecting/processing data from them is to create situation awareness and enable applications, machines, and human users to better understand their surrounding environments.  The understanding of a situation, or context, potentially enables services and applications to make intelligent decisions and to respond to the dynamics of their environments.
  • 137. 137 Observation and measurement data Source: W3C Semantic Sensor Networks, SSN Ontology presentation, Laurent Lefort et al.
  • 138. 138 How to say what a sensor is and what it measures? Sink node Gateway
  • 139. 139 Data/Service description frameworks  There are standards such as Sensor Web Enablement (SWE) set developed by the Open Geospatial Consortium that are widely being adopted in industry, government and academia.  While such frameworks provide some interoperability, semantic technologies are increasingly seen as key enabler for integration of IoT data and broader Web information systems.
  • 140. 140 Sensor Markup Language (SensorML) Source: http://www.mitre.org/
  • 141. 141 W3C SSN Ontology makes observations of this type Where it is What it measures units SSN-XG ontologies SSN-XG annotations SSN-XG Ontology Scope
  • 142. 142 Semantics and IoT data  Creating ontologies and defining data models is not enough  tools to create and annotate data  data handling components  Complex models and ontologies look good, but  design lightweight versions for constrained environments  think of practical issues  make it as compatible as possible and/or link it to the other existing ontologies  Domain knowledge and instances  Common terms and vocabularies  Location, unit of measurement, type, theme, …  Link it to other resources  Linked-data  URIs and naming
  • 143. 143 Semantics and sensor data Source: W. Wang, P. Barnaghi, "Semantic Annotation and Reasoning for Sensor Data", In proceedings of the 4th European Conference on Smart Sensing and Context (EuroSSC2009), 2009.
  • 144. 144 Semantics and Linked-data  The principles in designing the linked data are defined as:  using URI’s as names for things;  using HTTP URI’s to enable people to look up those names;  provide useful RDF information related to URI’s that are looked up by machine or people;  including RDF statements that link to other URI’s to enable discovery of other related concepts of the Web of Data;
  • 146. 146 Myth and reality  #1: If we create an Ontology our data is interoperable  Reality: there are/could be a number of ontologies for a domain  Ontology mapping  Reference ontologies  Standardisation efforts  #2: Semantic data will make my data machine-understandable and my system will be intelligent.  Reality: it is still meta-data, machines don’t understand it but can interpret it. It still does need intelligent processing, reasoning mechanism to process and interpret the data.  #3: It’s a Hype! Ontologies and semantic data are too much overhead; we deal with tiny devices in IoT.  Reality: Ontologies are a way to share and agree on a common vocabulary and knowledge; at the same time there are machine-interpretable and represented in interoperable and re- usable forms;  You don’t necessarily need to add semantic metadata in the source- it could be added to the data at a later stage (e.g. in a gateway);  Legacy applications can ignore it or to be extended to work with it.
  • 148. 148 148 Symbolic Aggregate Approximation (SAX) Variable String Length and Vocabulary size. Length: 10, VocSize: 10 Length: 10, VocSize: 4 “gijigdbabd” “cdddcbaaab” Green Curve: consists of 100 Samples, Blue Curve: SAX
  • 149. 149 SAX representation SAX Pattern (blue) with word length of 20 and a vocabulary of 10 symbols over the original sensor time-series data (green) P. Barnaghi, F. Ganz, C. Henson, A. Sheth, "Computing Perception from Sensor Data", in Proc. of the IEEE Sensors 2012, Oct. 2012. fggfffhfffffgjhghfff jfhiggfffhfffffgjhgi fggfffhfffffgjhghfff
  • 150. 150 Data Processing Framework fggfffhfffffgjhghfff dddfffffffffffddd cccddddccccdddccc aaaacccaaaaaaaaccccdddcdcdcdcddasddd PIR Sensor Light Sensor Temperature Sensor Raw sensor data stream Raw sensor data stream Raw sensor data stream Attendance Phone Hot Temperature Cold Temperature Bright Day-time Night-time Office room BA0121 On going meeting Window has been left open …. Temporal data (extracted from SSN descriptions) Spatial data (extracted from SSN descriptions) Thematic data (low level abstractions) Parsimonious Covering Theory Observations Perceptions Domain knowledge SAX Patterns Raw Sensor Data (Annotated with SSN Ontology) … …. Perception Computation High-level Perceptions
  • 151. 151 SensorSAX F. Ganz, P. Barnaghi, F. Carrez, “Information Abstraction for Heterogeneous Real World Internet Data”, Feb. 2013.
  • 152. 152 Evaluation results of abstraction creation F. Ganz, P. Barnaghi, F. Carrez, “Information Abstraction for Heterogeneous Real World Internet Data”, Feb. 2013.
  • 153. 153 Data size reduction F. Ganz, P. Barnaghi, F. Carrez, “Information Abstraction for Heterogeneous Real World Internet Data”, Feb. 2013.
  • 154. 154 Enabling the Internet of Things - Diversity range of applications - Interacting with large number of devices with various types -Multiple heterogeneous networks -Deluge of data -Processing and interpretation of the IoT data
  • 155. 155 Challenges and opportunities  Providing infrastructure  Publishing, sharing, and access solutions on a global scale  Indexing and discovery (data and resources)  Aggregation and fusion  Trust, privacy and security  Data mining and creating actionable knowledge  Integration into services and applications in e-health, the public sector, retail, manufacturing and personalized apps.  Mobile apps, location-based services, monitoring control etc.  New business models
  • 156. 156 Part 6: Cognitive aspects of knowledge representation, reasoning and perception Image source: CISCO
  • 157. Abstraction provides the ability to interpret and synthesize information in a way that affords effective understanding and communication of ideas, feelings, perceptions, etc. between machines and people. Abstraction
  • 158.  People are excellent at abstraction; of sensing and interpreting stimuli to understand and interact with the world.  The process of interpreting stimuli is called perception; and studying this extraordinary human capability can lead to insights for developing effective machine perception. Abstraction
  • 160. Semantic Perception/Abstraction Fundamental Questions What is perception, and how can we design machines to perceive? What can we learn from cognitive models of perception? Is the Semantic Web up to the task of modeling perception?
  • 161. What is Perception? Perception is the act of  Abstracting  Explaining  Discriminating  Choosing
  • 162. What can we learn from Cognitive Models of Perception?  A-priori background knowledge is a key enabler  Perception is a cyclical, active process Ulric Neisser (1976)Ulric Neisser (1976) Richard Gregory (1997)Richard Gregory (1997)
  • 163. Is Semantic Web up to the task of modeling perception? Representation Heterogeneous sensors, sensing, and observation records Background knowledge (observable properties, objects/events, etc.) Inference Explain observations (hypothesis building) Focus attention by seeking additional stimuli (that discriminate between explanations) Difficult Issues to Overcome Perception is an inference to the best explanation Handle streaming data Real-time processing (or nearly)
  • 164. Both people and machines are capable of observing qualities, such as redness. * Formally described in a sensor/ontology (SSN ontology) observes Observer Quality
  • 165. The ability to perceive is afforded through the use of background knowledge, relating observable qualities to entities in the world. * Formally described in domain ontologies (and knowledge bases) inheres in Quality Entity
  • 166. With the help of sophisticated inference, both people and machines are also capable of perceiving entities, such as apples.  the ability to degrade gracefully with incomplete information  the ability to minimize explanations based on new information  the ability to reason over data on the Web  fast (tractable) perceives EntityPerceiver
  • 167. Perceptual Inference minimize explanations degrade gracefully tractabl e Abductive Logic (e.g., PCT) high complexity Deductive Logic (e.g., OWL) (relatively) low complexity Web reasoning Perceptual Inference (i.e., abstraction)
  • 168. The ability to perceive efficiently is afforded through the cyclical exchange of information between observers and perceivers. Traditionally called the Perceptual Cycle (or Active Perception) sends focus sends observation Observer Perceiver
  • 170.  1970’s – Perception is an active, cyclical process of exploration and interpretation. - Nessier’s Perception Cycle  1980’s – The perception cycle is driven by background knowledge in order to generate and test hypotheses. - Richard Gregory (optical illusions)  1990’s – In order to effectively test hypotheses, some observations are more informative than others. - Norwich’s Entropy Theory of Perception Cognitive Theories of Perception
  • 171. Key Insights Background knowledge plays a crucial role in perception; what we know (or think we know/believe) influences our perception of the world. Semantics will allow us to realize computational models of perception based on background knowledge. Contemporary Issues Internet/Web expands our background knowledge to a global scope; thus our perception is global in scope Social networks influence our knowledge and beliefs, thus influencing our perception
  • 172. observes inheres in Integrated together, we have an general model – capable of abstraction – relating observers, perceivers, and background knowledge. perceives sends focus sends observation Observer Quality EntityPerceiver
  • 173.  Ontology of Perception – as an extension of SSN  Provides abstraction of sensor data through perceptual inference of semantically annotated data
  • 174. Prior Knowledge W3C SSN Ontology Bi-partite Graph  Prior knowledge conformant to SSN ontology (left), structured as a bipartite graph (right)
  • 175. Explanation is the act of accounting for sensory observations (i.e., abstraction); often referred to as hypothesis building. Observed Property: A property that has been observed.   ObservedProperty ≡ ∃ssn:observedProperty— .{o1} ⊔ … ⊔ ∃ssn:observedProperty— .{on}   Explanatory Feature: A feature that explains the set of observed properties.   ExplanatoryFeature ≡ ∃ssn:isPropertyOf— .{p1} ⊓ … ⊓ ∃ssn:isPropertyOf— .{pn}   Semantics of Explanation
  • 176. Example Assume the properties elevated blood pressure and palpitations have been observed, and encoded in RDF (conformant with SSN):   ssn:Observation(o1), ssn:observedProperty(o1, elevated blood pressure) ssn:Observation(o2), ssn:observedProperty(o2, palpitations)   Given these observations, the following ExplanatoryFeature class is constructed: ExplanatoryFeature ≡ ∃ssn:isPropertyOf— .{elevated blood pressure} ⊓ ∃ssn:isPropertyOf— .{palpitations} Given the KB, executing the query ExplanatoryFeature(?y) can infer the features, Hypertension and Hyperthyroidism, as explanations: ExplanatoryFeature(Hypertension) ExplanatoryFeature(Hyperthyroidism) Semantics of Explanation
  • 177. Discrimination is the act of deciding how to narrow down the multitude of explanatory features through further observation. Expected Property: A property is expected with respect to (w.r.t.) a set of features if it is a property of every feature in the set. ExpectedProperty ≡ ∃ssn:isPropertyOf.{f1} ⊓ … ⊓ ∃ssn:isPropertyOf.{fn}   NotApplicable Property: A property is not-applicable w.r.t. a set of features if it is not a property of any feature in the set. NotApplicableProperty ≡ ¬∃ssn:isPropertyOf.{f1} ⊓ … ⊓ ¬∃ssn:isPropertyOf.{fn} Discriminating Property: A property is discriminating w.r.t. a set of features if it is neither expected nor not-applicable. DiscriminatingProperty ≡ ¬ExpectedProperty ⊓ ¬NotApplicableProperty Semantics of Discrimination
  • 178. Example Given the explanatory features from the previous example, Hypertension and Hyperthyroidism, the following classes are constructed: ExpectedProperty ≡ ∃ssn:isPropertyOf.{Hypertension} ⊓ ∃ssn:isPropertyOf.{Hyperthyroidism}   NotApplicableProperty ≡ ¬∃ssn:isPropertyOf.{Hypertension} ⊓ ¬∃ssn:isPropertyOf.{Hyperthyroidism}   Given the KB, executing the query DiscriminatingProperty(?x) can infer the property clammy skin as discriminating:   DiscriminatingProperty(clammy skin) Semantics of Discrimination
  • 179. How do we design the Sensor Web?  Integration through shared semantics  OGC Sensor Web Enablement  W3C SSN ontology and Semantic Annotation  Interpretation through integration of heterogeneous data and reasoning with prior knowledge  Semantic Perception/Abstraction  Linked Open Data as prior knowledge  Scale through distributed local interpretation  “intelligence at the edge”
  • 180. Efficient Algorithms for IntellegO  Use of OWL-DL reasoner too resource-intensive for use in resource constrained devices (such as sensor nodes, mobile phones, IoT devices)  Runs out of resources for problem size (prior knowledge) > 20 concepts  Asymptotic complexity: O(n3 ) [Experimentally determined]  To enable their use on resource-constrained devices, we now describe algorithms for efficient inference of explanation and discrimination.  These algorithms use bit vector encodings and operations, leveraging a-priori knowledge of the environment.
  • 181. Efficient Algorithms for IntellegO Semantic (RDF) Encoding Bit Vector Encoding Lower Lift  First, developed lifting and lowering algorithms to translate between RDF and bit vector encodings of observations.
  • 182. Efficient Algorithms for IntellegO Explanation Algorithm Discrimination Algorithm Utilize bit vector operators to efficiently compute explanation and discrimination Explanation: Use of the bit vector AND operation to discover and dismiss those features that cannot explain the set of observed properties Discrimination: Use of the bit vector AND operation to discover and indirectly assemble those properties that discriminate between a set of explanatory features. The discriminating properties are those that are determined to be neither expected nor not-applicable
  • 183. Efficient Algorithms for IntellegO Evaluation: The bit vector encodings and algorithms yield significant and necessary computational enhancements – including asymptotic order of magnitude improvement, with running times reduced from minutes to milliseconds, and problem size increased from 10’s to 1000’s.
  • 186. 186 J. McCarthyJ. McCarthy M. WeiserM. Weiser D. Engelbart D. Engelbart J. C. R. LickliderJ. C. R. Licklider Similar Visions of Computing
  • 187. 187 If people do not believe that mathematics is simple, it is only because they do not realize how complicated life is. - John von Neumann If people do not believe that mathematics is simple, it is only because they do not realize how complicated life is. - John von Neumann Computational paradigms have always dealt with a simplified representations of the real-world… Computational paradigms have always dealt with a simplified representations of the real-world… Algorithms work on these simplified representations Algorithms work on these simplified representations Solutions from these algorithms are transcended back to the real-world by humans as actions Solutions from these algorithms are transcended back to the real-world by humans as actions http://ngs.ics.uci.edu/blog/?p=1501 Grand challenges in the real-world are complex
  • 188. What has changed now? Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain There are more devices connected to the internet than the entire human population. There are more devices connected to the internet than the entire human population. Today we have around 10 billion devices connected to the internet making it an era of IoT (Internet of Things) Today we have around 10 billion devices connected to the internet making it an era of IoT (Internet of Things) http://www.cisco.com/web/about/ac79/docs/innov/IoT_IBSG_0411FINAL.pdf
  • 189. 189 What has not changed?What has not changed? We need computational paradigms to tap into the rich pulse of the human populace We need computational paradigms to tap into the rich pulse of the human populace We are still working on the simpler representations of the real- world! We are still working on the simpler representations of the real- world! Represent, capture, and compute with richer and fine-grained representations of real-world problems Represent, capture, and compute with richer and fine-grained representations of real-world problems What should change?What should change?
  • 190. PCS Computing: Asthma Scenario 190 Sensordrone – for monitoring environmental air quality Wheezometer – for monitoring wheezing sounds Can I reduce my asthma attacks at night? What are the triggers? What is the wheezing level? What is the propensity toward asthma? What is the exposure level over a day? What is the air quality indoors? Commute to Work Personal Public Health Population Level Closing the window at home in the morning and taking an alternate route to office may lead to reduced asthma attacks Actionable Information Actionable Information
  • 191. Personal, Public Health, and Population Level Signals for Monitoring Asthma Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain ICS= inhaled corticosteroid, LABA = inhaled long-acting beta2-agonist, SABA= inhaled short-acting beta2-agonist ; *consider referral to specialist Asthma Control and Actionable Information Sensors and their observations for understanding asthma
  • 192. Asthma Early Warning Model Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain 192 Personal Level Signals Personal Level Signals Societal Level Signals Societal Level Signals (Personal Level Signals) (Personalized Societal Level Signal) (Societal Level Signals) Societal Level Signals Relevant to the Personal Level Societal Level Signals Relevant to the Personal Level Personal Level Sensors (mHealth**) QualifyQualify QuantifyQuantify Action Recommendation Action Recommendation What are the features influencing my asthma? What is the contribution of each of these features? How controlled is my asthma? (risk score) What will be my action plan to manage asthma? StorageStorage Societal Level Sensors Asthma Early Warning Model (AEWM) Query AEWM Verify & augment domain knowledge Recommended Action Action Justification *http://www.slideshare.net/jain49/eventshop-120721, ** http://www.youtube.com/watch?v=btnRi64hJp4 (EventShop*)
  • 193. Health Signal Extraction to Understanding Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain Population Level Personal Wheeze – Yes Do you have tightness of chest? –Yes ObservationsPhysical-Cyber-Social System Health Signal Extraction Health Signal Understanding <Wheezing=Yes, time, location> <ChectTightness=Yes, time, location> <PollenLevel=Medium, time, location> <Pollution=Yes, time, location> <Activity=High, time, location> Wheezing ChectTightness PollenLevel Pollution Activity Wheezing ChectTightness PollenLevel Pollution Activity RiskCategory <PollenLevel, ChectTightness, Pollution, Activity, Wheezing, RiskCategory> <2, 1, 1,3, 1, RiskCategory> <2, 1, 1,3, 1, RiskCategory> <2, 1, 1,3, 1, RiskCategory> <2, 1, 1,3, 1, RiskCategory> . . . Expert Knowledge Background Knowledge tweet reporting pollution level and asthma attacks Acceleration readings from on-phone sensors Sensor and personal observations Signals from personal, personal spaces, and community spaces Risk Category assigned by doctors Qualify Quantify Enrich Outdoor pollen and pollution Public Health Well Controlled - continue Not Well Controlled – contact nurse Poor Controlled – contact doctor
  • 194. 194 J. McCarthyJ. McCarthy M. WeiserM. Weiser D. Engelbart D. Engelbart J. C. R. LickliderJ. C. R. Licklider
  • 195. 195 Part 8: IoT/PCS Systems: Applications Image source: CISCO
  • 196. SSN Applications Applications of SSN HealthcareWeather Rescue Traffic Fire Fighting Logistics
  • 197. SSN Application: Weather 50% savings in sensing resource requirements during the detection of a blizzard Order of magnitude resource savings between storing observations vs. relevant abstractions
  • 198. Linked Sensor Data Linked Sensor Data (~2 Billion Statements)
  • 199. Sensor Discovery Application Query w/ location name to find nearby sensors
  • 200. Real-Time Feature Streams Demo: http://www.youtube.com/watch? v=_ews4w_eCpg Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain
  • 201. SSN Application: Fire Detection Weather Application SECURE: Semantics-empowered Rescue Environment (detect different types of fires) DEMO: http://www.youtube.com/watch?v=in2KMkD_uqg
  • 202. SSN Application: Health Care MOBILEMD: Mobile app to help reduce re- admission of patients with Chronic Heart Failure
  • 203. SSN Application: Health Care Passive Monitoring PhasePassive Monitoring Phase • Abnormal heart rate • Clammy skin • Panic Disorder • Hypoglycemia • Hyperthyroidism • Heart Attack • Septic Shock Observed Symptoms Possible Explanations Passive Sensors – heart rate, galvanic skin response
  • 204. SSN Application: Health Care Active Monitoring PhaseActive Monitoring Phase Are you feeling lightheaded?Are you feeling lightheaded? Are you have trouble taking deep breaths? Are you have trouble taking deep breaths? yesyes yesyes Have you taken your Methimazole medication? Have you taken your Methimazole medication? Do you have low blood pressure?Do you have low blood pressure? yesyes • Abnormal heart rate • Clammy skin • Lightheaded • Trouble breathing • Low blood pressure • Panic Disorder • Hypoglycemia • Hyperthyroidism • Heart Attack • Septic Shock Observed Symptoms Possible Explanations nono Active Sensors – blood pressure, weight scale, pulse oxymeterDemo: http://www.youtube.com/watch?v=btnRi64hJp4
  • 205. Domain ExpertsDomain Experts ColdWeatherColdWeather PoorVisibilityPoorVisibility SlowTrafficSlowTraffic IcyRoadIcyRoad Declarative domain knowledge Causal knowledge Linked Open Data ColdWeather(YES/NO)IcyRoad (ON/OFF) PoorVisibility (YES/NO)SlowTraffic (YES/NO) 1 0 1 1 1 1 1 0 1 1 1 1 1 0 1 0 Domain ObservationsDomain Observations Domain Knowledge Domain Knowledge Structure and parameters 205 WinterSeasonWinterSeason Otherknowledge Correlations to causations using Declarative knowledge on the Semantic Web kTraffic
  • 206. Traffic jamTraffic jam Link Description Link Description Scheduled Event Scheduled Event traffic jamtraffic jambaseball gamebaseball game Add missing random variables Time of dayTime of day bad weather CapableOf slow traffic bad weatherbad weather Traffic data from sensors deployed on road network in San Francisco Bay Area time of daytime of day traffic jamtraffic jambaseball gamebaseball game time of daytime of day slow trafficslow traffic Three Operations: Complementing graphical model structure extraction Add missing links bad weatherbad weather traffic jamtraffic jambaseball gamebaseball game time of daytime of day slow trafficslow traffic Add link direction bad weatherbad weather traffic jamtraffic jambaseball gamebaseball game time of daytime of day slow trafficslow traffic go to baseball game Causes traffic jam Knowledge from ConceptNet5 traffic jam CapableOfoccur twice each day traffic jam CapableOf slow traffic 206
  • 207. 207 Scheduled Event Scheduled Event Active EventActive Event Day of weekDay of week Time of dayTime of day delaydelay Travel time Travel time speedspeed volumevolume Structure extracted form traffic observations (sensors + textual) using statistical techniques Scheduled Event Scheduled Event Active EventActive Event Day of weekDay of week Time of dayTime of day delaydelay Travel time Travel time speedspeed volumevolume Bad WeatherBad Weather Enriched structure which has link directions and new nodes such as “Bad Weather” potentially leading to better delay predictions kTraffic
  • 208. SemMOB Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain First responders have limited time to analyze sensor (on and around them) observations. First responders have limited time to analyze sensor (on and around them) observations. Fire fighters need to combine prior knowledge of fires and its behavior, extinguishers, and floor plan of the building for rescue strategy and operations. Fire fighters need to combine prior knowledge of fires and its behavior, extinguishers, and floor plan of the building for rescue strategy and operations. There are a variety of sensors used to monitor vitals of firefighters, location, and poisonous gases. There are a variety of sensors used to monitor vitals of firefighters, location, and poisonous gases. O2 O2 Heart rateHeart rate COCO CO2 CO2 GPSGPS AccelerometerAccelerometer CompassCompass FootstepsFootsteps There is a team of them making it further difficult for analysis There is a team of them making it further difficult for analysis Semantic Web allow us to describe the domain, sensors, and first responders -- apply reasoning techniques to derive actionable insights Semantic Web allow us to describe the domain, sensors, and first responders -- apply reasoning techniques to derive actionable insights Sensors on each responder provides different view of the event and they need to register dynamically as the firefighters arrive. Sensors on each responder provides different view of the event and they need to register dynamically as the firefighters arrive.
  • 209. SemMOB Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain PlatformPlatform SensingDeviceSensingDevice System_AGT C1 System_AGT C1 Sensor_LSM1 1 Sensor_LSM1 1 SensorOutputSensorOutput O2Concentraction_outputO2Concentraction_output Measuremen tCapability Measuremen tCapability MeasurementCa pability_LSM11 MeasurementCa pability_LSM11 PropertyProperty concentratio n concentratio n onPlatform hasOutput hasMeasurementCapability Observes ObservationObservation O2ConcentrationO2Concentration ObservedProperty O2Obs_1505201 2 O2Obs_1505201 2 observationResultTimehttp://geonames.org/629864 0 http://geonames.org/629864 0 hasLocation name xsd:floatxsd:float latlong Dayton, Dayton-Wright Brothers Airport^^xsd:string FeatureFeature Demo: http://www.youtube.com/watch?v=nX11OqgtQlw
  • 210. LOKILAS (LOst Key Identification, Localization and Alert System) Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain
  • 211. LOKILAS Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain
  • 212. LOKILAS Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain
  • 213. Future work  Creating ontologies and defining data models are not enough  tools to create and annotate data  Tools for publishing linked IoT data  Designing lightweight versions for constrained environments  think of practical issues  make it as much as possible compatible and/or link it to the other existing ontologies  Linking to domain knowledge and other resources  Location, unit of measurement, type, theme, …  Linked-data  URIs and naming
  • 214. Some of the open issues  Efficient real-time IoT resource/service query/discovery  Directory  Indexing  Abstraction of IoT data  Pattern extraction  Perception creation  IoT service composition and compensation  Integration with existing Web services  Service adaptation
  • 215. Selected references  Payam Barnaghi, Wei Wang, Cory Henson, Kerry Taylor, "Semantics for the Internet of Things: early progress and back to the future", (to appear) International Journal on Semantic Web and Information Systems (special issue on sensor networks, Internet of Things and smart devices), 2012.  Atzori, L., Iera, A. & Morabito, G. , “The Internet of Things: A survey”, Computer Networks, Volume 54, Issue 15, 28 October 2010, 2787- 2805.  Suparna De, Tarek Elsaleh, Payam Barnaghi , Stefan Meissner, "An Internet of Things Platform for Real-World and Digital Objects", Journal of Scalable Computing: Practice and Experience, vol 13, no.1, 2012.  Suparna De, Payam Barnaghi, Martin Bauer, Stefan Meissner, "Service modelling for the Internet of Things", in Proceedings of the Conference on Computer Science and Information Systems (FedCSIS), pp.949-955, Sept. 2011.  Cory Henson, Amit Sheth, and Krishnaprasad Thirunarayan, “Semantic Perception: Converting Sensory Observations to Abstractions”, IEEE Internet Computing, Special Issue on Context-Aware Computing, March/April 2012.  Payam Barnaghi, Frieder Ganz, Cory Henson, Amit Sheth, “Computing Perception from Sensor Data”, In proceedings of the 2012 IEEE Sensors Conference, Taipei, Taiwan, October 28-31, 2012.  Michael Compton et al, “The SSN Ontology of the W3C Semantic Sensor Network Incubator Group”, Journal of Web Semantics, 2012.  Harshal Patni, Cory Henson, and Amit Sheth , “Linked Sensor Data”, in Proceedings of 2010 International Symposium on Collaborative Technologies and Systems (CTS 2010), Chicago, IL, May 17-21, 2010.  Amit Sheth, Cory Henson, and Satya Sahoo , “Semantic Sensor Web IEEE Internet Computing”, vol. 12, no. 4, July/August 2008, pp. 78- 83.  Wei Wang, Payam Barnaghi, Gilbert Cassar, Frieder Ganz, Pirabakaran Navaratnam, "Semantic Sensor Service Networks", (to appear) in Proceedings of the IEEE Sensors 2012 Conference, Taipei, Taiwan, October 2012.  Wang W, De S, Toenjes R, Reetz E, Moessner K, "A Comprehensive Ontology for Knowledge Representation in the Internet of Things", International Workshop on Knowledge Acquisition and Management in the Internet of Things (KAMIoT 2012) in conjunction with IEE IUCC-2012, Liverpool, UK. Liverpool. 25-27 June, 2012.
  • 216. Some useful links related to IoT  Internet of Things, ITU  http://www.itu.int/osg/spu/publications/internetofthings/InternetofThings_summary.pdf  IoT Comic Book  http://www.theinternetofthings.eu/content/mirko-presser-iot-comic-book  Internet of Things Europe, http://www.internet-of-things.eu/  Internet of Things Architecture (IOT-A)  http://www.iot-a.eu/public/public-documents  W3C Semantic Sensor Networks  http://www.w3.org/2005/Incubator/ssn/XGR-ssn-20110628/

Hinweis der Redaktion

  1. Scalability and interoperability problems
  2. Intelligence at the edge or hub; still no good answer; could be app or design dependent
  3. Traditional networking: host to host Now: data-oriented communication; looking for data, not the host providing the data unless you want to manage the node
  4. IoT data different from traditional content; transient, small. The more important thing is how to find the service that can provide the data; could be huge stream of data
  5. Take about something on the web of data
  6. Since then bring in the semantic web technologies
  7. Images: http://www.google.com/imgres?q=abstract+earth+puzzle&amp;um=1&amp;hl=en&amp;safe=off&amp;biw=1548&amp;bih=829&amp;tbm=isch&amp;tbnid=fCWWmELEgLspwM:&amp;imgrefurl=http://depositphotos.com/4946293/stock-photo-Abstract-earth-puzzle.html&amp;docid=ObBXLAbfdYscyM&amp;imgurl=http://static5.depositphotos.com/1021974/494/i/450/dep_4946293-Abstract-earth-puzzle.jpg&amp;w=450&amp;h=397&amp;ei=QEKXTrSIFLCrsALi0LnqBA&amp;zoom=1&amp;iact=hc&amp;vpx=206&amp;vpy=160&amp;dur=463&amp;hovh=166&amp;hovw=201&amp;tx=86&amp;ty=75&amp;sig=102505865583293696354&amp;page=1&amp;tbnh=160&amp;tbnw=196&amp;start=0&amp;ndsp=30&amp;ved=1t:429,r:0,s:0
  8. Images: http://massthink.wordpress.com/2007/06/10/husserl-in-indubitable-response-to-descartes-and-kant/
  9. Four characteristics of perceptual inference
  10. Images: http://www.ida.liu.se/~eriho/COCOM_M.htm http://www.idemployee.id.tue.nl/g.w.m.rauterberg/lecturenotes/ucd%20lecture-3/sld019.htm
  11. Images: http://www.ida.liu.se/~eriho/COCOM_M.htm http://www.idemployee.id.tue.nl/g.w.m.rauterberg/lecturenotes/ucd%20lecture-3/sld019.htm
  12. Declarative knowledge + statistical correlation This slide illustrates the three operations to enrich the correlation structure extracted using statistical methods These operations utilize declarative knowledge form ConceptNet5 as shown in each step
  13. Statistical correlation structure shown above The enriched structure is shown below The enrichment of the graphical model will potentially allow us to capture the domain precisely and also improve our prediction as the model would get closer to the underlying probabilistic distribution in the real-world Log-Likelihood score is one way of quantifying how good a structure is based on the observed data There may be many candidate structures extracted from data which result in the log likelihood score Declarative knowledge will help us ground statistical models to reality which will allow us to pick one structure over the other Pramod Anantharam, Krishnaprasad Thirunarayan and Amit Sheth, &apos;Traffic Analytics using Probabilistic Graphical Models Enhanced with Knowledge Bases,&apos; 2nd International Workshop on Analytics for Cyber-Physical Systems (ACS-2013) at SIAM International Conference on Data Mining (SDM13), pp. 13--20, Texas, USA, May 2-4, 2013. We stopped at structure extraction for our workshop paper (SIAM ACS workshop) since the declarative knowledge we used (ConceptNet5) and statistical model (nodes and edges) are at the same level of abstraction