Stream Reasoning: a summary of ten years of research and a vision for the next decade

Stream Reasoning:a summary of ten years of research and

a vision for the next decadehttp://content.iospress.com/articles/data-science/ds006

Emanuele Della ValleDEIB - Politecnico di Milano

@[email protected]://emanueledellavalle.org

Linköping University, Sweden - 5.10.2017

http://content.iospress.com/articles/data-science/ds006

mailto:[email protected]


http://emanueledellavalle.org

Me

• Assistant Professor at DEIBPolitecnico di Milano

• Expert in semantic technologies and stream computing

• Brander of stream reasoning: an approach to master the velocity and variety dimension of Big Data• https://scholar.google.com/scholar?hl=

en&q="stream+reasoning"

• 16 years of experience in research and innovation projects

• Startupper: • http://www.fluxedo.com

2

INTRODUCTION

SR 2015, Austria - 3.11.2015 @manudellavalle - http://emanueledellavalle.org 3

It's a streaming world …

• Off-shore oil operations

• Smart Cities

• Power turbine

• Social networks

• Generate data streams!

E. Della Valle, S. Ceri, F. van Harmelen, D. Fensel It's a Streaming World! Reasoning upon Rapidly Changing Information. IEEE Intelligent Systems 24(6): 83-89 (2009)


… looking for reactive answers …• When a sensor on a drill in an oil-rig indicates that it is

about to get stuck, how long can I keep drilling?

• Where am I likely going to run into atraffic jam during my commute tonight?

• Which electricity-producing turbine hassensor readings similar to any turbine thatsubsequently had a critical failure?

• Who is driving the discussion about the top 10 emerging topics ?

• Require continuous processing and reactive answer


… and many more conflicting requirements

A system able to answer those queries must be able to

• handle volume

• handle velocity

• handle variety

• cope with incompleteness

• cope with noise

• provide reactive answers

• support fine-grained access

• integrate complex domain models

• offer high-level languages


Grand challenge

• Volume + Velocity + Variety = hard deal

SR 2015, Austria - 3.11.2015 @manudellavalle - http://emanueledellavalle.org

vo

lum

e

months days hours min. sec. ms. velocity

ZB

EB

PB

TB

GB

MB

KB

Variety

7

A good reason to embrace it!

• ++ Variety ++ value


valu

e

ms. sec. min. hours days months yearsvelocity

Variety

8

From challenges to opportunities

• Formally data streams are : – unbounded sequences of time-varying data elements

• Less formally, in many application domains, they are: – a “continuous” flow of information – where recent information is more relevant as it describes the

current state of a dynamic system

• Opportunities– Forget old enough information– Exploit the implicit ordering (by recency) in the data

time


State-of-the-art: DSMS and CEP

• A paradigmatic change!

• Continuous queries registered over streams thatare observed trough windows

window

input streams streams of answerRegistered Continuous Query

Dynamic System


DSMS and CEP vs. requirements

RequirementDSMSCEP

volume

velocity

variety

incompleteness

noise

reactive answers

fine-grained information access

complex domain models

high-level languages


State of the art: Semantic Web

• Given ontology O and query Q, use O to rewrite Qas Q’ so that, for any set of ground facts A contained in multiple databases:

– answer(Q,O,A) = answer(Q’,,A)The answer of the query Q using the ontology O for any set of ground facts A is equal to answer of a query Q’ without considering the ontology O

• Use mapping M to map Q’ to multiple SQL queries to the various databases

Rewrite

O

QQ’

MapSQL

M

answerA

UiO, Norway - 3.11.2015 @manudellavalle - http://emanueledellavalle.org 12

Solutions vs. requirements

RequirementDSMSCEP

SemWeb

volume

velocity

variety

incompleteness

noise

reactive answers





Stream Reasoning

• Research question

– is it possible to make sense in real time ofmultiple, heterogeneous, gigantic and inevitably noisy and incomplete data streams in order to support the decision processes of extremely large numbers of concurrent users?


Emanuele Della Valle: On Stream Reasoning. PhD thesis, Vrije Universiteit Amsterdam, 2015. Available online at http://dare.ubvu.vu.nl/handle/1871/53293 .

http://dare.ubvu.vu.nl/handle/1871/53293

Is this feasible?

• Proposed approach: cascading Stream Reasoning

Complexity

Raw Stream Processing

Semantic Streams

DL-Lite

DLAbstraction

Selection

Interpretation

Reasoning

Querying

Re-writing

Change Frequency

PTIME

NEXPTIME

104 Hz

1 Hz

Complexity vs. Dynamics

AC0

H. Stuckenschmidt, S. Ceri, E. Della Valle, F. van Harmelen: Towards Expressive Stream Reasoning. Proceedings of the Dagstuhl Seminar on Semantic Aspects of Sensor Networks, 2010.


A model to describe stream reasoning

SR 2015, Austria - 3.11.2015 @manudellavalle - http://emanueledellavalle.org 16D. Dell’Aglio, On Unified Stream Reasoning, PhD thesis, Politecnico di Milano, 2016.

Stream Processing (DSMS)

Event Processing (CEP)Window merge

Window operator

Streams

Graph-level entailment

Window-level entailment

Stream-level entailment

Application

Stre

am R

easo

nin

g

GRAPH LEVEL UNDER SIMPLE ENTAILMENT REGIME


State-of-the-art: RDF model

• RDF: Resource Description Framework– It allows to make statements about resources

in the form of subject-predicate-object expressions• In RDF terminology triples• E.g.

@TimBernersLee posts "This is for everyone"

– A collection of RDF statements represents a labelled, directed graph• In RDF terminology a graph• E.g., the tweet above by Tim Berners Lee is connected to

– Thousands of twitter user profiles via retweets– Thousands of twitter user profiles via favorite– …

subject predicate object

UiO, Norway - 3.11.2015 @manudellavalle - http://emanueledellavalle.org 18

RDF stream Models

• RDF Stream (the C-SPARQL way)

– Unbound sequence of time-varying triples

– each represented by a pair made of an RDF triple and its timestamp

– Timestamp are non-decreasing (allowing for simultaneity)…

TimBernersLee posts "This is for everyone", 10:16.55 PM 22 Aug 2012

@Alice posts "RT: This is for everyone", 10:17.03 PM 22 Aug 2012

…

D.F. Barbieri, D. Braga, S. Ceri, E. Della Valle, M. Grossniklaus: Querying RDF streams with C-SPARQL. SIGMOD Record 39(1): 20-26 (2010)

subject predicate object timestamp


RDF stream Models

• RDF Stream (the Streaming Linked Data way)

– Unbound sequence of time-varying graphs

– each represented by a pair made of an RDF graph and its timestamp (all triples in a graph are simultaneous)

– Timestamps (if present) are monotonically increasing

– Graphs act as a form of punctuation


A. Mauri, J-P Calbimonte, D.Dell'Aglio, M.Balduini, M.Brambilla, E.Della Valle, K. Aberer: TripleWave: Spreading RDF Streams on the Web. ISWC (2) 2016: 140-149

20

D.F. Barbieri, E. Della Valle: A Proposal for Publishing Data Streams as Linked Data –A Position Paper. LDOW (2010)

Continuous-SPARQL


Continuous-SPARQLWho are the opinion makers? i.e., the users who are

likely to influence the behavior their followers

REGISTER STREAM OpinionMakers COMPUTED EVERY 5m AS

CONSTRUCT { ?opinionMaker sd:about ?resource }

FROM STREAM <http://…> [RANGE 30m STEP 5m]

WHERE {

?opinionMaker ?opinion ?res .

?follower sioc:follows ?opinionMaker.

?follower ?opinion ?res.

FILTER (cs:timestamp(?follower ?opinion ?res) >

cs:timestamp(?opinionMaker ?opinion ?res) )

}

HAVING ( COUNT(DISTINCT ?follower) > 3 )

SR 2015, Austria - 3.11.2015 @manudellavalle - http://emanueledellavalle.org 22D.F. Barbieri, D. Braga, S. Ceri, E. Della Valle, M. Grossniklaus: Querying RDF streams with C-SPARQL. SIGMOD Record 39(1): 20-26 (2010)

Continuous-SPARQLWho are the opinion makers? i.e., the users who are

likely to influence the behavior their followers

REGISTER STREAM OpinionMakers COMPUTED EVERY 5m AS

CONSTRUCT { ?opinionMaker sd:about ?resource }

FROM STREAM <http://…> [RANGE 30m STEP 5m]

WHERE {

?opinionMaker ?opinion ?res .

?follower sioc:follows ?opinionMaker.

?follower ?opinion ?res.

FILTER (cs:timestamp(?follower ?opinion ?res) >

cs:timestamp(?opinionMaker ?opinion ?res) )

}

HAVING ( COUNT(DISTINCT ?follower) > 3 )

Query registration(for continuous execution)

FROM STREAM clause

WINDOW

RDF Stream added asnew ouput format

Builtin to accesstimestamps

SR 2015, Austria - 3.11.2015 @manudellavalle - http://emanueledellavalle.org 23D.F. Barbieri, D. Braga, S. Ceri, E. Della Valle, M. Grossniklaus: Querying RDF streams with C-SPARQL. SIGMOD Record 39(1): 20-26 (2010)

More than modeling, it's for reactive answers!

• C-SPARQL engine time window-based selection outperforms SPARQL filter-based selection (Jena-ARQ)

D. Barbieri, D. Braga, S. Ceri, E. Della Valle, Y. Huang, V. Tresp, A.Rettinger, H. Wermser: Deductive and Inductive Stream Reasoning for Semantic Social Media AnalyticsIEEE Intelligent Systems, 30 Aug. 2010.

Our In-memoryRDF stream processing

engine


Alternatives to C-SPARQL

• SPARQLStream

– What: window in the past, focus on RDF to Stream operators– Ref: Calbimonte, J.-P., Corcho, O., & Gray, A. J. G. Enabling ontology-based

access to streaming data sources. In ISWC, 2010, pages 96–111.

• CQELS– What: STREAM clause, focus on new answer – Ref: Le-Phuoc, D., Dao-Tran, M., Xavier Parreira, J., & Hauswirth, M.

A native and adaptive approach for unified processing of linked streams and linked data. In ISWC 2011, pages 370–388.

• EP-SPARQL– What: focus on event specific operators– Ref: Anicic, D., Fodor, P., Rudolph, S., & Stojanovic, N. EP-SPARQL: a unified

language for event processing and stream reasoning. In WWW 2011, pages 635–644.

• TEF-SPARQL– What: adds "facts" as first class elements – Ref: https://www.merlin.uzh.ch/publication/show/8467


https://www.merlin.uzh.ch/publication/show/8467

Work in progress

• RSP-QL– Syntax

• https://github.com/streamreasoning/RSP-QL/blob/master/RSP-QL%20Sample%20Queries.md

– Proposed semantics• D.Dell'Aglio, E.Della Valle, J.-P.Calbimonte, Ó. Corcho: RSP-QL

Semantics: A Unifying Query Model to Explain Heterogeneity of RDF Stream Processing Systems. Int. J. Semantic Web Inf. Syst. 10(4): 17-44 (2014)

– Semantics (work in progress)• https://github.com/streamreasoning/RSP-

QL/blob/master/Semantics.md

– Quick ref.• D. Dell'Aglio, J.-P. Calbimonte, E. Della Valle, Ó. Corcho: Towards a

Unified Language for RDF Stream Query Processing. ESWC (Satellite Events) 2015: 353-363


https://github.com/streamreasoning/RSP-QL/blob/master/RSP-QL Sample Queries.md

https://github.com/streamreasoning/RSP-QL/blob/master/Semantics.md

GRAPH LEVEL UNDER OTHER ENTAILMENT REGIME


continuous deductive reasoning

• DL Ontology Stream ST

– A ontology stream with respect to a static Tbox T is a sequence of Abox axioms ST(i)

• A Windowed Ontology Stream ST(o,c]

– A windowed ontology stream with respect to a static Tbox T is the union of the Abox axioms ST(i) where o<i≤c

• Reasoning on a Windowed Ontology Stream ST(o,c] is as reasoning on a static DL KB

SR 2015, Austria - 3.11.2015 @manudellavalle - http://emanueledellavalle.org 28Emanuele Della Valle, Stefano Ceri, Davide Francesco Barbieri, Daniele Braga, Alessandro Campi: A First Step Towards Stream Reasoning. FIS 2008: 72-81

discusses discusses discusses

discusses discusses

discusses

discusses

Example of continuous deductive reasoning

What impact has been my micropost p1 creating in the last hour?

Let’s count the number of microposts that discuss it …

REGISTER STREAM ImpactMeter AS

SELECT (count(?p) AS ?impact)

FROM STREAM <http://…/fb> [RANGE 60m STEP 10m]

WHERE {

:Alice posts [ sr:discusses ?p ]

}

p1 p3 p5 p8

p2 p4 p7

p6

Transitive

property

Alice posts p1 .


Not so naïve approach tostream reasoning

• The problem is that materialization (the result of data-driven processing) are very difficult to decrement efficiently.– State-of-the-art: DRed algorithm

• Over delete• Re-derive• Insert

ReasonerInferred

data

ontology

window

insertions

deletions

Incremental !!!

SPARQL


Is DRed needed?

• DRed works with random insertions and deletions

• In a streaming setting, when a triple enters the window, given the size of the window, the reasoner knows already when it will be deleted!

• E.g., – if the window is 40 minutes

long, and,

– it is 10:00, the triple(s) entering now

– will exit on 10:40.

• Conclusion– deletions are predictable

Time

Enter window

Exit window

Explicitly in window

Inferred in window # Exp. # Inf. SUM

10:00 AßB

1 1

10:10 BßC

1 1 2

10:20 AßE

2 1 3

10:30 EßC

2 1 3

10:40 AßB

2 1 3

10:50 BßC

2 1 3

11:00 AßE

0 0 0

A B

A B C A C

A B C

E

A C

A B C

E

A C

A C

E

A C

A B C

E

A C

C

E


IMaRS algorithm

• Idea:

– add an expiration time to each triple and

– use an hash table to index triples by their expiration time

• The algorithm

1. deletes expired triples

2. Adds the new derivations that are consequences of insertions annotating each inferred triple with an expiration time (the min of those of the triple it is derived from), and

3. when multiple derivations occur, for each multiple derivation, it keeps the max expiration time.


IMaRS algorithm

• Incremental Reasoning on RDF streams (IMaRS): new reasoning algorithm optimized for reactive query answering

D.F. Barbieri, D. Braga, S.Ceri, E. Della Valle, M. Grossniklaus: Incremental Reasoning on Streams and Rich Background Knowledge. ESWC (1) 2010: 1-15D. Dell'Aglio, E. Della Valle: Incremental Reasoning on RDF Streams. In A.Harth, K.Hose, R.Schenkel (Eds.) Linked Data Management, CRC Press 2014, ISBN 9781466582408

§ Re-materialize after each window slide

§ Use DRed

§ IMaRS

% of deletions w.r.t. the content of the window


IMaRS algorithm

• comparison of the average time needed to answera C-SPARQL query, when 2% of the content exits the window each time it slides, using – A backward reasoner on the window content– DRed + standard SPARQL on the materialization– IMaRS + standard SPARQL on the materialization


Some alternatives to IMaRS

• TROWL– How: DRed in the context of approximate reasoning– Ref: Y. Ren, J. Z. Pan. Optimising ontology stream reasoning

with truth maintenance system. In CIKM (2011)

• Sparkwave– How: extended RETE algorithm for windows and RDFS– Ref: Sparkwave: Continuous Schema-Enhanced Pattern

Matching over RDF Data Streams. Komazec S, Cerri D. DEBS 2012

• The Backward/Forward Algorithm– How: optimizing DRed– B. Motik, Y. Nenov, R.E.F. Piro, I. Horrocks: Incremental Update

of Datalog Materialisation: the Backward/Forward Algorithm. AAAI 2015: 1560-1568


WINDOW LEVEL ENTAILMENT


window level entailment

• Graph-level entailment considers data item contents, but it does not use the temporal annotations

• Window-level entailment applies the inference process on the non-merged stream items.

• E.g.,– A door cannot be open and close at the same time– A window contains: door A is open @1, door A is close @2– At graph-level the reasoner tells that there is an inconsistency in

the window because it ignore the parts in italics– At window-level the reasoner does not

• Best approaches I saw so far– ÖL Özçep, R Möller. Ontology Based Data Access on Temporal and

Streaming Data. Reasoning Web, 2014– D. Dell'Aglio et al. :A Query Model to Capture Event Pattern Matching

in RDF Stream Processing Query Languages EKAW 2016: 145-162


STREAM LEVEL ENTAILMENT



• Window-level entailment only considers a recent portion of the stream

• Stream-level entailment aims at considering the entire stream

• This is not just a theoretical dream, CEP does so– E.g., rise C for every A that follows a B without a C in the middle

• Best approaches I saw so far– ETALIS

• Anicic, D., Rudolph, S., Fodor, P., & Stojanovic, N. Stream reasoning and complex event processing in ETALIS Semantic Web,3(4), 2012, 397–407.

– LARS• H. Beck, M. Dao-Tran, T. Eiter, M. Fink: LARS: A Logic-Based Framework

for Analyzing Reasoning over Streams. AAAI 2015: 1431-1438H.


OPEN PROBLEMS AND CHALLENGES FOR THE NEXT YEARS


Requirements vs. state-of-the-art

Requirement SotA@2017Q2

volume

velocity

variety

incompleteness

noise

reactive answers





Towards queries that capture user needs

• Capturing a wider set of tasks in queries

– Descriptive analytics

• Which electricity-producing turbine has sensor readings similar (i.e., Pearson correlated by at least 0.75) to any turbine that subsequently had a critical failure in the past year?

– Predictive analytics (Machine Learning)

• Where am I likely going to run into a traffic jam during my commute tonight and how long will it take, given current weather and traffic conditions?

– Geo-spatial reasoning

– Including preferences

– Exploiting the graph nature of the data



• Tightening CEP and ontological reasoning – ETALIS/EP-SPARQL is the corner stone with high-

expressivity but very low efficiency

– Work in progress:


CEP

DL

RDF STREAM PROCESSING

RAW STREAM PROCESSING

Abstraction

Selection

Annotation

Reasoning

Querying

Rewriting

R. Tommasini, P. Bonte, E. Della Valle, E. Mannens, F. De Turck, F. Ongenae: Towards Ontology-Based Event Processing. OWLED 2016: 115-127


• Forgetting knowledge

– Is IMaRS semantics what users want?

– Consumption ≠ expiration

• Consumed facts are not useful for processing

• Expired facts are not true anymore

– Should we introduce

• a notion of semantic importance?

• a notion of time-annotated facts?

• …


Towards sophisticated Stream Reasoning

• Extending the range of logical formalisms – Pioneers:

• Metric Temporal logic– F. Heintz and P. Doherty, DyKnow: An approach to middleware for

knowledge processing, Journal of Intelligent and 11 Fuzzy Systems 15(1) (2004), 3–13.

• Answer Set Programming (ASP)– A. Mileo, A. Abdelrahman, S. Policarpio and M. Hauswirth, StreamRule:

A nonmonotonic stream reasoning system for the semantic web, in: RR, LNCS, Vol. 7994, Springer, 2013, pp. 247–252.

– More alternatives: • action logics

• step logics

• active logics

• event calculus



• Integrating other types of reasoning/processing

– My contribution


D.F. Barbieri, D. Braga, S. Ceri, E. Della Valle, Y. Huang, V. Tresp, A. Rettinger, H. Wermser: Deductive and Inductive Stream Reasoning for Semantic Social Media Analytics. IEEE Intelligent Systems 25(6): 32-41 (2010)


• Integrating other types of reasoning/processing

– Other options

• More on inductive stream reasoning

• Probabilistic reasoning

• Planning

• Natural Language Processing

• Sentiment analysis

• …



• Towards semantic streams

– Describe data stream to make them discoverable, composable, etc.

– My work in progress

• Y. A. Sedira, R.Tommasini, E. Della Valle: Towards VoIS: A Vocabulary of Interlinked Streams. DeSemWeb@ISWC 2017

– There is much more to describe, e.g.

• Stream of state (the door is close) vs. Stream of actions (the door was closed)



• Towards scalable stream reasoners – Rewriting

• e.g., MorphStream– How: rewriting in DSMS languages (one at a time)– Ref: Calbimonte, J.-P., Corcho, O., & Gray, A. J. G. Enabling ontology-

based access to streaming data sources. In ISWC, 2010, pages 96–111.

– Using parallelization and distribution• e.g., DynamiTE

– How: Truth maintenance for DF (a fragment of RDFS)– J. Urbani, A. Margara, C. J. H. Jacobs, F. van Harmelen, H.E. Bal:

DynamiTE: Parallel Materialization of Dynamic RDF Data. ISWC (1) 2013: 657-672

– Using approximation• e.g., TR-OWL

– How: Truth maintenance for EL++ with syntactic approximations– Ref: Y. Ren, J. Z. Pan. Optimising ontology stream reasoning with truth

maintenance system. In CIKM (2011)


Towards robustness to imperfect data

• Overcoming types of heterogeneity that do not exist in the static reasoning settings– It is not just extending OBDA to continuous queries

– There are more sources of heterogeneity than in the static database settings• Different execution semantics

• No possibility for a priori homogenous– Synchronisation*

– discretization

• …

* Pioneered in F.Heintz, DyKnow: A stream Based Knowledge Processing Middleware Framework, PhD thesis,Linkoping University, Sweden, 2009.


Towards robustness to imperfect data

• Copying with noise

– Data as signals instead of data as truths!

– Easy to state, hard to achieve

– Still of paramount importance!

– Notable approaches I run into• Matthias Nickles, Alessandra Mileo: Web Stream Reasoning Using

Probabilistic Answer Set Programming. RR 2014: 197-205

• Anastasios Skarlatidis, Georgios Paliouras, Alexander Artikis, George A. Vouros: Probabilistic Event Calculus for Event Recognition. ACM Trans. Comput. Log. 16(2): 11:1-11:37 (2015)

• Anni-Yasmin Turhan, Erik Zenker: Towards Temporal Fuzzy Query Answering on Stream-based Data. HiDeSt@KI 2015: 56-69


Conclusions


Stream Processing

Event ProcessingWindow merge

Window operator

Streams

Graph-level entailment

Window-level entailment


Application

Stre

am R

easo

nin

g

Very well only some initial studies still very open

New abstractions to capture user needs

Application

Stream Reasoning:a summary of ten years of research and a

vision for the next decadehttp://content.iospress.com/articles/data-science/ds006

Q/AEmanuele Della ValleDEIB - Politecnico di Milano

@[email protected]://emanueledellavalle.org

Linköping University, Sweden - 5.10.2017

http://content.iospress.com/articles/data-science/ds006



http://emanueledellavalle.org

Data & Analytics

Stream Reasoning: a summary of ten years of research and a vision for the next decade