This document discusses an innovative semantic solution called SNAP to help transport stakeholders comply with EU Regulation 2017/1926, which requires the sharing of transport data through National Access Points using standard data formats. The SNAP solution uses a reference ontology derived from Transmodel to convert heterogeneous transport data into the required formats. It provides a semantic converter called Chimera that can customize data lifting from source formats into the ontology and lowering into target formats using mapping files. Current work involves finalizing the Transmodel ontology, piloting SNAP with different stakeholders and data, and investigating reversible mappings.
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Turn Transport Data into EU Compliance with Semantic Solution
1. An Innovative Semantic Solution to Turn Transport Data
into EU Compliance
Marco Comerio (Cefriel)
2. 3
• The EU Regulation 2017/1926
• Requirements
• Impact on Transport Stakeholders
• Challenges & Opportunities for the Semantic Web Community
• The SNAP solution
• Ongoing & Next Steps
Agenda
3. 4
• Objective: establish an interoperability framework enabling European industry players to
make their business applications ‘interoperate’ and provides the travelers with a new
seamless travel experience
• Barriers:
• insufficient accessibility of transport data
• lack of service and data interoperability
• Key enablers:
• appropriate data sharing mechanism
• making data interoperable by means of a common set of data exchange standards
EU Mission: multimodal travel information service
4. 5
EU Regulation 2017/19261: Requirements
• Each EU Member State is required to set up a National Access Point (NAP)
• allowing access to static data (e.g., timetables, network topology, list of services offered at a
station / airport) and dynamic data (e.g., delays, cancellations) related to different transport
modes (air, train, bus, ferry, metro, tram, car/bike-sharing, car-pooling, etc.)
• defined according to specific standard data formats such as NeTEx CEN / TS 16614 and
SIRI CEN / TS 15531
• described using national application profiles (e.g., DCAT-AP)
1 https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32017R1926
5. 6
EU Regulation 2017/1926: Status
• The first deadline for the provisioning of the first set of static transport data through NAPs is
December 1st, 2019
• According to [1], very few transport stakeholders are ready to provide their data and services
in compliance with the standards requested by the EU Regulation.
• To enable a data conversion process, they rely on their in-house support (which may lack
knowledge and skills related to the Regulation) or turn to external technology providers that
provide custom and often expensive solutions.
[1] Hendriks L., Jorna R., et Al. (2018). Monitoring and Harmonisation of National Access Points in Europe.
Available at: www.its-platform.eu/filedepot_download/1971/6491
6. 7
EU Regulation 2017/1926: Impact on Transport Stakeholders
Obligations
• Provide datasets to the NAP compliant to the requested data formats
• Provide metadata description of the datasets
Challenge
• Turn available data into the requested data formats and, in case,
enrich them with additional data sources
Benefit
• Enhance the knowledge and the adoption of their transport offerings
• Potential revenues from data trading
Transport Authorities
and Operators
Transport
Infrastructure Managers
7. 8
• Reference ontologies
• unambiguously describe the operational aspects of the transport domain
• Metadata profiles for transport datasets
• harmonize the metadata description of datasets through a semantically-enhanced metadata profile
• Semantic Web-based transport data converters
• turn available transport data into specific data formats and, in case, enrich them with additional data sources
• e.g., a specific (semantic) converter enables the translation of transportation schedule, geographic and fare
information expressed in GTFS to a NeTEx specification preserving the original meaning
EU Regulation 2017/1926: Challenges & Opportunities
8. Conceptualization
• Acquire knowledge of the
domain
• Acquire knowledge of
data formats/standards
• Define the reference
ontology
Sharing
• Define asset types
• Define asset descriptors
Governance
• Identify actors, roles and
associated tasks
• Define of the lifecycle of
each asset type
• Define processes to
automate the lifecycle
management
Conversion
• Define models and rules
to enable the
conversion process
Our Contributions
9. 10
• The EU Regulation 2017/1926
• Requirements
• Impact on Transport Stakeholders
• Challenges & Opportunities for the Semantic Web Community
• The SNAP solution
• Ongoing & Next Steps
Agenda
10. 11
• Snap supports the process of compliance to the EU Regulation 2017/1926 providing
• a reference ontology for the transport domain
• a new solution supporting the mapping of heterogeneous transport data to the target standard
requested by the EU Regulation
• Snap is an EIT Digital Innovation Activity: a co-financed innovation project carried out by
the EIT Digital Partners, with the goal of launching on the market a new product or service
The Snap Project
www.snap-project.eu
11. Customers’ Needs
• Make transport data compatible with the standards
established by EU Regulation 2017/1926
• Enrich own datasets with other data
• Render own datasets interoperable with other data
• Minimize the investment and time effort needed to
achieve compliance, enrichment and/or interoperability
Target Customers
• Transport authorities
• Transport operators
• Infrastructure Managers
Strategic Partners
• NAP Management Board
• Standardization bodies
Snap Desirability
12. • A reference ontology for
the transportation domain
The Snap Solution
Transmodel
Ontology
13. • A reference ontology for
the transportation domain
The Snap Solution
Transmodel
Ontology
Source Data
Preparation
• Open source components
• FIWARE modules for data
preparation
14. • A reference ontology for
the transportation domain
The Snap Solution
Target Data
Format
Source Data
Format
Transmodel
Ontology
Source Data
Preparation
• Open source components
• FIWARE modules for data
preparation
• Chimera converter for data
conversion and enrichment
through semantic Web
technologies
15. • A reference ontology for
the transportation domain
The Snap Solution
Target Data
Format
Source Data
Format
Transmodel
Ontology
LIFTING
Specification
LOWERING
Specification
Mapping
File
Mapping
File
Source Data
Preparation
• Customization services to enable
interoperability in transport:
• Mapping definition, enabling
lifting to the reference ontology
and lowering to target formats
• Open source components
• FIWARE modules for data
preparation
• Chimera converter for data
conversion and enrichment
through semantic Web
technologies
16. 17
The Transmodel Ontology
• The Snap solution is empowered by a reference ontology
• derived from a sub-part of the CEN Transmodel1, the European reference data
model for public transport information (e.g., timetable, fares)
• under finalization by the Universidad Politecnica de Madrid using the LOT (Linked
Open Terms) Methodology2
• The ontology is a starting point for a potential collaboration with the CEN
TC278-WG3-SG4 that has formally defined the Transmodel
1www.transmodel-cen.eu 2http://lot.linkeddata.es/
17. 18
The Chimera converter
• Semantic Interoperability allows avoiding the burden of point-to-point interoperability
• The ontology provides an interoperability layer to guarantee decoupled conversion pipelines
• Lifting and lowering processes can be defined once for each data format
• Chimera provides two different options for the lifting process
• RML mappings
• Java annotations
• Chimera provides two different options for the lowering process
• Java annotations
• Apache Velocity template
18. 19
LIFTING: RML Lifter
• This block accepts mappings defined
through the RML mapping language
• RML extends R2RML (SQL) allowing
also mappings from heterogeneous
data sources (CSV, XML, JSON)
through the definition of iterators.
• Built-in support for data enrichment
referencing mappings defined for sources
with different data formats
• This block uses the mapper from
https://github.com/RMLio/rmlmapper-java
19. 20
LOWERING: Template Lowerer block
• This block is able to apply an
Apache Velocity template
(https://velocity.apache.org) to
extract data and build a
structured document
• SPARQL queries defined at the
beginning of the template are
executed and template variables
are bound to their output
• Such variables are then used to
access the contents of the
results while filling the template
• This approach allows for high
flexibility (custom queries and
custom template structure)
but is not specific or optimized
for semantic applications
20. 21
LIFTING and LOWERING: Annotation-based Lifter and Lowerer
• Annotation-based block exploits Java
annotations to define mappings for Java
classes
Lifting:
• marshall from source data format to Java
instances
• unmarshall from Java instances to RDF
Lowering
• unmarshall from RDF to Java instances
• marshall from Java instances to target data
format
• This approach is useful for web-services
where an interface descriptor is used
(Java classes can be automatically
generated and annotated).
21. 22
The Chimera converter
• The Chimera converter can be customized to fulfill more complex requirements by means of a
set of building blocks supporting:
• Lifting
• Data enrichment
• Inference enrichment
• Lowering
• Chimera is based on Apache Camel and inspired by Enterprise Integration Patterns (EIP)
• Other Apache Camel blocks can be integrated in the pipeline (e.g., I/O, un/marshall,…)
https://camel.apache.org/
22. 23
The Chimera converter
Lowering blockLifting block
Source
Message
Target
Message
• A custom pipeline can be defined. For a basic conversion, a lifting and lowering block are required.
23. 24
The Chimera converter
Source
Message
• A custom pipeline can be defined. For a basic conversion, a lifting and lowering block are required.
• The RDF graph can be seen as a global variable shared among the blocks of the pipeline
RDF graph (Reference Ontology terms)
24. 25
The Chimera converter
Lifting block
Source
Message
Mapping to
Reference
Ontology
• A custom pipeline can be defined. For a basic conversion, a lifting and lowering block are required.
• The RDF graph can be seen as a global variable shared among the blocks of the pipeline
RDF graph (Reference Ontology terms)
25. 26
The Chimera converter
Lifting block Data enricher
Source
Message
Mapping to
Reference
Ontology
RDF data
• A custom pipeline can be defined. For a basic conversion, a lifting and lowering block are required.
• The RDF graph can be seen as a global variable shared among the blocks of the pipeline
RDF graph (Reference Ontology terms)
26. 27
The Chimera converter
Lifting block Data enricher
Source
Message
Inference enricher
Mapping to
Reference
Ontology
RDF data
• A custom pipeline can be defined. For a basic conversion, a lifting and lowering block are required.
• The RDF graph can be seen as a global variable shared among the blocks of the pipeline
RDF graph (Reference Ontology terms)
Reference
Ontology
27. 28
The Chimera converter
Lowering blockLifting block Data enricher
Source
Message
Inference enricher
Mapping to
Reference
Ontology
Target
Message
Mapping
from
Reference
Ontology
RDF data
• A custom pipeline can be defined. For a basic conversion, a lifting and lowering block are required.
• The RDF graph can be seen as a global variable shared among the blocks of the pipeline
Reference
Ontology
Export RDF graph
28. 29
SWOT Analysis
STRENGTHS
• Flexibility
• Reusability
WEAKNESSES
• Handmade mappings (no tooling)
• Semantic/Logic skills required
OPPORTUNITIES
• Conceptualization of the domain
• Applicability to different domains
• Semantic NAPs with Transmodel RDF data
THREATS
• Lack of domain knowledge leads to bad
ontologies and mappings
INTERNAL
EXTERNAL
POSITIVE
NEGATIVE
29. 30
• The EU Regulation 2017/1926
• Requirements
• Impact on Transport Stakeholders
• Challenges & Opportunities for the Semantic Web Community
• The SNAP solution
• Ongoing & Next Steps
Agenda
30. 31
Ongoing & Next Steps
• Finalization of the Transmodel ontology
• Expected to be published in October on https://github.com/oeg-upm/transmodel-ontology
• 3 Pilots of the Snap solution
• covering different metropolitan areas (Madrid, Milano and Genova)
• different stakeholders (transport authorities, transport operators, infrastructure managers)
• different transport modes (air, bus, metro)
• different source data formats (GTFS, proprietary format)
• Reversible mappings: investigation of the feasibility to exploit a ReversibleRMLMapper capable
of using RML mappings defined for the lifting block also for the lowering process