Beneﬁts of Publishing the Norwegian Petroleum Directorate ...folk.uio.no/martige/pub/SkLi-NIK2013_presentation.pdf · Beneﬁts of Publishing the Norwegian Petroleum Directorate’s

Benefits of Publishing the NorwegianPetroleum Directorate’s FactPages as Linked

Open Data

Martin G. Skjæ[email protected]

NIK 201319 November 2013

Outline

NPD and NPD FactPages

Linked Open Data

Converting NPD’s FactPages to Semantic Web Data

Examples

Map Visualization

Query Formulation

2 / 20

Outline


Linked Open Data


Examples

Map Visualization

Query Formulation

3 / 20

Norwegian Petroleum Directorate (NPD)What is it?

• A governmental specialist directorate and administrative body

• Reports to the Ministry of Petroleum and Energy

• Main objective is to look after the national interests in the energysector

• One of four functions:

The NPD has a national responsibility for data from the Norwegiancontinental shelf. The NPD’s data, overview and analyses constitutea crucial factual basis on which the activities are founded

4 / 20

NPD FactPagesWhat is it?

• The FactPages contain data about petroleum activities on the NCS

• Data is collected from companies that operate on the NCS

• Important functions:• forms the basis for the authorities’ planning of future activity and their

judgement of existing activity

• makes companies share information

• provides information to the general public

• Acts as a national reference data library for the activities on the NCS

5 / 20

NPD FactPagesDemonstration

11 main categoriesSelecting Field ...

... lists availableviews andsubviews.SelectingPage view →All ...

... lists recordedmembers for theview. SelectingTROLL ...

... displays a (pageview) report, i.e., alldata,about the fieldTroll

DownloadCSV, XML,Excel

Shortcomings to the NPD FactPages:

• No query interface• Arbitrary queries not possible,• only predefined categories and views

• Poor identifiers• Difficult information retrieval• Integration “impossible”

6 / 20

http://factpages.npd.no/


11 main categories

Selecting Field ...








6 / 20



11 main categoriesSelecting Field ...








6 / 20


Outline


Linked Open Data


Examples

Map Visualization

Query Formulation

7 / 20

5-star Linked Open Data

Tim Berners-Lee’s 5 Linked Open Data stars

:

? Available on the web with an open licence

?? + machine-readable structured data

(Excel)

??? + non-proprietary format

(CSV, XML)

???? + use W3C standards (URI, RDF) to identify and serve

????? + link to other data

Even better:

?????? + provide query interface (SPARQL)

??????? + define vocabulary semantics (OWL)

8 / 20


Tim Berners-Lee’s 5 Linked Open Data stars:


?? + machine-readable structured data

(Excel)

??? + non-proprietary format

(CSV, XML)



Even better:



8 / 20


Tim Berners-Lee’s 5 Linked Open Data stars:


?? + machine-readable structured data (Excel)

??? + non-proprietary format (CSV, XML)



Even better:



8 / 20

Linked Open DataUniform Resource Identifier (URI)

• Schema for global identifiers

• Example: Core sample no. 3 of wellbore 1/6-A-7http://sws.ifi.uio.no/data/npd-v2/wellbore/903/core/3

http://factpages.npd.no/factpages/default.aspx?culture=en&nav1=wellbore&nav2=PageView|Development|With|WithCores&nav3=903

• Crucial for integration

• Makes retrival of information easy

http://sws.ifi.uio.no/data/npd-v2/page/wellbore/903/core/3

http://sws.ifi.uio.no/data/npd-v2/data/wellbore/903/core/3

http://sws.ifi.uio.no/data/npd-v2/wellbore/903/core/3

• Using existing "web machinery"

9 / 20

http://factpages.npd.no/factpages/default.aspx?culture=en&nav1=wellbore&nav2=PageView|Development|With|WithCores&nav3=903http://factpages.npd.no/factpages/default.aspx?culture=en&nav1=wellbore&nav2=PageView|Development|With|WithCores&nav3=903http://sws.ifi.uio.no/data/npd-v2/page/wellbore/903/core/3http://sws.ifi.uio.no/data/npd-v2/data/wellbore/903/core/3http://sws.ifi.uio.no/data/npd-v2/wellbore/903/core/3


• Schema for global identifiers• Example: Core sample no. 3 of wellbore 1/6-A-7




• Makes retrival of information easy

http://sws.ifi.uio.no/data/npd-v2/page/wellbore/903/core/3




9 / 20



• Schema for global identifiers• Example: Core sample no. 3 of wellbore 1/6-A-7




• Makes retrival of information easyhttp://sws.ifi.uio.no/data/npd-v2/page/wellbore/903/core/3




9 / 20


Linked Open DataResource Description Framework (RDF)

• Simple, generic data model• It’s all triples!

subject predicate objectnpd:wellbore/903 rdf:type npdv:Wellborenpd:wellbore/903 npdv:hasCoreSample npd:wellbore/903/core/3

• No/One schema/schema independent, so (at a technical level):• Easy to extend• Easy to exchange• Easy to consume• Easy to query

• Use with “any” RDF tool• tools to browse, visualise, edit

10 / 20

Linked Open DataSPARQL Protocol and RDF Query Language (SPARQL)

• RDF query language, similar to SQL

• Protocol for• sending queries• receiving results• result format(s)• using existing web infrastructure

• Safely expose data to querying over Internet• Examples

? Total production of oil and gas jan–jun 2010 for Statoil operated fields? Find things close to Svalbard? Ekofisk oil and gas production

Not possible to answer with existing NPD FactPages

11 / 20

http://sws.ifi.uio.no/snorql/npd-v2/?query=SELECT+%3Ffield%0D%0A++++++%28SUM%28%3Fg%29+AS+%3Fgas%29%0D%0A++++++%28SUM%28%3Fo%29+AS+%3Foil%29%0D%0AWHERE+{%0D%0A++[+npdv%3AproductionYear+2010+%3B%0D%0A++++npdv%3AproductionMonth+%3Fm+%3B%0D%0A++++npdv%3AproducedGas+++++%3Fg+%3B%0D%0A++++npdv%3AproducedOil+++++%3Fo+%3B%0D%0A++++npdv%3AproductionForField%0D%0A++++[+rdf%3Atype+npdv%3AField+%3B%0D%0A++++++npdv%3Aname+%3Ffield+%3B%0D%0A++++++npdv%3AcurrentFieldOperator%0D%0A++++++[+npdv%3AshortName+%22STATOIL+PETROLEUM+AS%22%5E%5Exsd:string+]+]+]%0D%0A++FILTER%28%3Fm+%3E%3D+01+%26%26%0D%0A+++++++++%3Fm+%3C%3D+06%29%0D%0A}+GROUP+BY+%3Ffield+ORDER+BY+%3Ffield%0D%0Ahttp://sws.ifi.uio.no/sparql/npd?query=PREFIX+npd%3A+%3Chttp%3A%2F%2Fsws.ifi.uio.no%2Fnpd%2F%3E%0D%0APREFIX+npdv%3A+%3Chttp%3A%2F%2Fsws.ifi.uio.no%2Fvocab%2Fnpd%23%3E%0D%0A%0D%0ASELECT+%3Fname+%3Flat+%3Flong+%3Furl%0D%0AWHERE+%0D%0A+{+%3Furl+npdv%3AnsDecDeg+%3Flat+%3B%0D%0A+npdv%3AewDecDeg+%3Flong+%3B%0D%0A+npdv%3Aname+%3Fname+%3B%0D%0A+FILTER+%28%3Flat+%3E+72.5%29%0D%0A+}&output=xml&stylesheet=http%3A%2F%2Fsws.ifi.uio.no%2Fproject%2Fnpd%2Fxsl%2Fgmaps-markers.xslhttp://sws.ifi.uio.no/sgvizler/npd/?query=PREFIX%20fn%3A%20%3Chttp%3A%2F%2Fwww.w3.org%2F2005%2Fxpath-functions%23%3E%0A%0ASELECT%20xsd%3Adate%28%3Fdate%29%20%3FOil_millSm%20%22%22%20%22%22%20%3FGas_billSm%20%22%22%20%22%22%0AWHERE{%20%0A%09{[]%20%0A%09%09a%20npdv%3AFieldProductionPeriod%20%3B%0A%09%09npdv%3AhasField%20%3Chttp%3A%2F%2Fsws.ifi.uio.no%2Fnpd%2Ffield%2FEkofisk%3E%20%3B%0A%09%09npdv%3Amonth%20%3Fmonth%20%3B%0A%09%09npdv%3Ayear%20%3Fyear%20%3B%0A%09%09npdv%3AproducedNetOilMillSm3%20%3FOil_millSm%20%3B%0A%09%09npdv%3AproducedNetGasBillSm3%20%3FGas_billSm%20%3B%0A%09%09LET%20%28%3Fdate%20%3A%3D%20fn%3Aconcat%28fn%3Aconcat%28fn%3Aconcat%28%3Fyear%2C%20%22-%22%29%2C%20%3Fmonth%29%2C%20%22-01%22%29%29%0A%09%09FILTER%20%28xsd%3Aint%28%3Fyear%29%20%3C%202011%20%26%26%20xsd%3Aint%28%3Fmonth%29%20%3E%209%29%0A%09}%0A%09UNION%0A%09{%0A%09%09[]%20%0A%09%09a%20npdv%3AFieldProductionPeriod%20%3B%0A%09%09npdv%3AhasField%20%3Chttp%3A%2F%2Fsws.ifi.uio.no%2Fnpd%2Ffield%2FEkofisk%3E%20%3B%0A%09%09npdv%3Amonth%20%3Fmonth%20%3B%0A%09%09npdv%3Ayear%20%3Fyear%20%3B%0A%09%09npdv%3AproducedNetOilMillSm3%20%3FOil_millSm%20%3B%0A%09%09npdv%3AproducedNetGasBillSm3%20%3FGas_billSm%20%3B%0A%09%09LET%20%28%3Fdate%20%3A%3D%20fn%3Aconcat%28fn%3Aconcat%28fn%3Aconcat%28%3Fyear%2C%20%22-0%22%29%2C%20%3Fmonth%29%2C%20%22-01%22%29%29%0A%09%09FILTER%20%28xsd%3Aint%28%3Fyear%29%20%3C%202011%20%26%26%20xsd%3Aint%28%3Fmonth%29%20%3C%2010%29%0A%09}%20}ORDER%20BY%20xsd%3Adate%28%3Fdate%29&chart=gTimeline


• RDF query language, similar to SQL• Protocol for

• sending queries• receiving results• result format(s)• using existing web infrastructure




11 / 20





• Safely expose data to querying over Internet

• Examples? Total production of oil and gas jan–jun 2010 for Statoil operated fields? Find things close to Svalbard? Ekofisk oil and gas production


11 / 20








11 / 20


Linked Open DataWeb Ontology Language (OWL)

• Language for formally defining vocabulary semantics• what the data means, not how to store it

• Domain-centric vocabulary• Data is accessible to the “right people”• Easier to formulate queries

• Sophisticated intregration using relations between ontologies• Formal semantics allows reasoning

• infer new facts• check for inconsistencies• explain consequences and query answers

12 / 20




• Sophisticated intregration using relations between ontologies

• Formal semantics allows reasoning• infer new facts• check for inconsistencies• explain consequences and query answers

12 / 20




• Sophisticated intregration using relations between ontologies• Formal semantics allows reasoning

• infer new facts• check for inconsistencies• explain consequences and query answers

12 / 20

Outline


Linked Open Data


Examples

Map Visualization

Query Formulation

13 / 20

Conversion Process

14 / 20

CSVCSVCSV

DB schema

1a. download

Transformationspecification

Rel. DB

D2RQ map

RDF Triple store

SPARQL

Triple store

OWLOWLOWL

1b. gen.

1c. create

1d. load

2a. generate

2b. dump 2c. load

3b. load

3a. generate

SQL

Conversion Process

1. Create relational database representation

14 / 20

CSVCSVCSV

DB schema

1a. download


Rel. DB

D2RQ map

RDF Triple store

SPARQL

Triple store

OWLOWLOWL

1b. gen.

1c. create

1d. load

2a. generate

2b. dump 2c. load

3b. load

3a. generate

SQL

Conversion Process


a) Download CSV files from NPD FactPages website

14 / 20

CSVCSVCSV

DB schema

1a. download


Rel. DB

D2RQ map

RDF Triple store

SPARQL

Triple store

OWLOWLOWL

1b. gen.

1c. create

1d. load

2a. generate

2b. dump 2c. load

3b. load

3a. generate

SQL

Conversion Process


b) Generate database schema—based on CSV files and transformation spec.

14 / 20

CSVCSVCSV

DB schema

1a. download


Rel. DB

D2RQ map

RDF Triple store

SPARQL

Triple store

OWLOWLOWL

1b. gen.

1c. create

1d. load

2a. generate

2b. dump 2c. load

3b. load

3a. generate

SQL

Conversion Process


c) Create and

d) load the database

14 / 20

CSVCSVCSV

DB schema

1a. download


Rel. DB

D2RQ map

RDF Triple store

SPARQL

Triple store

OWLOWLOWL

1b. gen.

1c. create

1d. load

2a. generate

2b. dump 2c. load

3b. load

3a. generate

SQL

Conversion Process

2. Create RDF/LOD representation

14 / 20

CSVCSVCSV

DB schema

1a. download


Rel. DB

D2RQ map

RDF Triple store

SPARQL

Triple store

OWLOWLOWL

1b. gen.

1c. create

1d. load

2a. generate

2b. dump 2c. load

3b. load

3a. generate

SQL

Conversion Process


a) Generate D2RQ map—based on transformation spec. and data

14 / 20

CSVCSVCSV

DB schema

1a. download


Rel. DB

D2RQ map

RDF Triple store

SPARQL

Triple store

OWLOWLOWL

1b. gen.

1c. create

1d. load

2a. generate

2b. dump 2c. load

3b. load

3a. generate

SQL

Conversion Process


b) Dump database to RDF using map

14 / 20

CSVCSVCSV

DB schema

1a. download


Rel. DB

D2RQ map

RDF Triple store

SPARQL

Triple store

OWLOWLOWL

1b. gen.

1c. create

1d. load

2a. generate

2b. dump 2c. load

3b. load

3a. generate

SQL

Conversion Process


c) and load into triple store

14 / 20

CSVCSVCSV

DB schema

1a. download


Rel. DB

D2RQ map

RDF Triple store

SPARQL

Triple store

OWLOWLOWL

1b. gen.

1c. create

1d. load

2a. generate

2b. dump 2c. load

3b. load

3a. generate

SQL

Conversion Process

3. Create ontology representation

14 / 20

CSVCSVCSV

DB schema

1a. download


Rel. DB

D2RQ map

RDF Triple store

SPARQL

Triple store

OWLOWLOWL

1b. gen.

1c. create

1d. load

2a. generate

2b. dump 2c. load

3b. load

3a. generate

SQL

Conversion Process


a) Generate OWL ontology—based on transformation spec. and data—and extend manually

14 / 20

CSVCSVCSV

DB schema

1a. download


Rel. DB

D2RQ map

RDF Triple store

SPARQL

Triple store

OWLOWLOWL

1b. gen.

1c. create

1d. load

2a. generate

2b. dump 2c. load

3b. load

3a. generate

SQL

Conversion Process


b) Load RDF + OWL into triple store

14 / 20

CSVCSVCSV

DB schema

1a. download


Rel. DB

D2RQ map

RDF Triple store

SPARQL

Triple store

OWLOWLOWL

1b. gen.

1c. create

1d. load

2a. generate

2b. dump 2c. load

3b. load

3a. generate

SQL

Conversion Process

We test the different representations in our query evaluation experiment

14 / 20

CSVCSVCSV

DB schema

1a. download


Rel. DB

D2RQ map

RDF Triple store

SPARQL

Triple store

OWLOWLOWL

1b. gen.

1c. create

1d. load

2a. generate

2b. dump 2c. load

3b. load

3a. generate

SQL

Results

• Availability of data is better• Each thing has an identifier• Find information using identifer

• Easier to exhange and integrate data• More data is available

• Arbitrary SPARQL queries• New facts added by reasoning with semantics

• Semantics are more explicit

• All using standardised languages• application independent• general-purpose tools available

15 / 20

Outline


Linked Open Data


Examples

Map Visualization

Query Formulation

16 / 20

Example: Map Visualization

NPD FactMap vs. “our” SPARQL map visualiser.

• Official map app.

• Server side• Predefined queries only• Map results only

• Generic map app• Client side, SPARQL based, open

source• Predefined generic ontology driven

queries and any SPARQL query• Results link to LOD

17 / 20

http://sws.ifi.uio.no/project/npd-v2/viz/



• Official map app.• Server side

• Predefined queries only• Map results only




17 / 20




• Official map app.• Server side• Predefined queries only

• Map results only




17 / 20




• Official map app.• Server side• Predefined queries only• Map results only




17 / 20





• Generic map app

• Client side, SPARQL based, opensource

• Predefined generic ontology drivenqueries and any SPARQL query

• Results link to LOD

17 / 20






source

• Predefined generic ontology drivenqueries and any SPARQL query


17 / 20







queries and any SPARQL query


17 / 20








17 / 20


Example: Visual Query Formulation

• We want data to be available to domain experts

• Most domain experts don’t know query languages

• Developed query formulation tool

• Generic, ontology/data driven application

• Customized to dataset

18 / 20

Example: Visual Query Formulation

Query: Fields operated by Statoil Petroleum AS with gas producingfacilities

• http://sws.ifi.uio.no/project/optique/qf/interface/

• Field• Company

• Name: Statoil Petroleum ASA

• Facility• facility function: GAS PRODUCER

• Show SPARQL

• Run query

• Select SNØHVIT

19 / 20

http://sws.ifi.uio.no/project/optique/qf/interface/

NPD FactPagesProject website

For more information:http://sws.ifi.uio.no/project/npd-v2/

Available for download:

• CSV• SQL• Mappings• RDF dumps• OWL ontologies• SPARQL endpoint• LOD front-end• Queries

20 / 20

http://sws.ifi.uio.no/project/npd-v2/

NPD and NPD FactPagesLinked Open DataConverting NPD's FactPages to Semantic Web DataExamplesMap VisualizationQuery Formulation

Documents

Beneﬁts of Publishing the Norwegian Petroleum Directorate ...folk.uio.no/martige/pub/SkLi-NIK2013_presentation.pdf · Beneﬁts of Publishing the Norwegian Petroleum Directorate’s