TWC LOGD: A Portal for Linking Open Government Data Dominic DiFranzo, Li Ding, John S. Erickson, Xian Li, Tim Lebo, James Michaelis, Alvaro Graves, Gregory

TWC LOGD: A Portal for Linking Open

Government Data Dominic DiFranzo, Li Ding, John S. Erickson, Xian Li, Tim Lebo, James Michaelis, Alvaro Graves, Gregory Todd Williams, Jin Guang Zheng, Johanna Flores, Zhenning Shangguan, Gino Gervasio, Deborah L. McGuinness, Jim Hendler

Tetherless World ConstellationRensselaer Polytechnic Institute

Semantic Web Challenge 2010

Nov 10, 2010

2

The TWC LOGD Portal for SWC2010

Real World Data US, UK, China,… Health, energy, economy

End User Applications Community Portal Fast, Low-cost Mashups

Semantic Web in Gov Domain Major partner of Data.gov 8.5 billion triples in LOD

3

“Semantic Web” and RDF logo showed up on the frontpage of the US Data.gov website

4

RPI brought Semantic Web to US government

5

Major Partner of US Data.gov Project

Semantic Web Tech deployed in Data.gov RDF data, SPARQL endpoint, semantic mashups

6

Government Adoption Process

Data-gov Wiki@RPIonline

May

21,

20

09

May

, 20

10

data.gov online SPARQL End Point& RDF data& DemosReplicated atData.gov

July

,20

09

2009 2010 …DemosTutorialsVideosSPARQL Endpoint20

09-

2010

Oct

, 20

10

New Applicationpublished bya team at DOE

Two-day Mashathon in Washington DC

Aug

, 20

10

May

21,

20

10

data.gov relaunchwith semantic webfeatured

TWC LOGD Drupal SiteannouncedO

ct,

2010

7

The Largest Real World LOD Dataset

8.5+ billion triples from real world 7500+ LOD links Accessible via Data Browser, e.g. Tabulator

8

Trends in Smoking Prevalence, Tobacco Policy Coverage and Tobacco Prices (1991-2007)

Smoking Prevalence vs. Tax, Policy …Extensible and accountable Mashups for NIH project

Extensible Mashups via Linked Data Diverse datasets from NIH Potentially linking to “unemployment rate”Accountable Mashups via Provenance Annotate datasets used in demos Feedback users’ comment to gov contact

9White House Visitor SearchLeveraging linked data (DBpedia & New York Times)

“POTUS”

dbpedia:Barack_Obama

Created by Dominic DiFranzo, Evan Patton, RPI, http://data-gov.tw.rpi.edu/demo/stable/white-house-visitor/top100-visitees.php

[Person Mashup] Data.gov (statistics) + DBpedia (personal profiles)+ NYTimes (news) [Technologies] Semantic MediaWiki, Google Visualization, IPad Apps available in Apple Store

The White House

Semantic Wiki

WikipediaNYTimes

10Linking GDP of the US and ChinaLinking international government data meaningfully

GDP of China (Billion Chinese Yuan )

GDP of the US (Billion Dollar)

[Temporal Mashup] bea.gov + federalreserve.gov +stats.gov.cn

8.3

6.32000 2010

http://www.federalreserve.gov/releases/

http://www.google.com.hk/imgres?q=us+flag+gov&um=1&hl=zh-CN&newwindow=1&safe=strict&client=firefox-a&rls=org.mozilla:en-US:official&source=og&biw=1104&bih=562&tbs=isch:1&tbnid=Pe3I56NVicHenM:&imgrefurl=http://www.timtim.com/drawing/view/drawing_id/2815&imgurl=http://www.timtim.com/public/images/drawings/large/2815_American_Flag.gif&zoom=1&w=486&h=346&iact=hc&vpx=118&vpy=110&dur=3287&hovh=189&hovw=266&tx=125&ty=84&ei=KvPYTMHBI4qkuAPi48XACA&oei=KvPYTMHBI4qkuAPi48XACA&esq=1&page=1&tbnh=117&tbnw=165&start=0&ndsp=18&ved=1t:429,r:0,s:0

http://www.bea.gov/index.htm

http://www.google.com.hk/imgres?q=china+flag+gov&um=1&hl=zh-CN&newwindow=1&safe=strict&client=firefox-a&rls=org.mozilla:en-US:official&source=og&biw=1104&bih=562&tbs=isch:1&tbnid=tEa8fO2m3Bs4hM:&imgrefurl=http://adoption.state.gov/country/china.html&imgurl=http://adoption.state.gov/images/adoptions/flags/china_flag.gif&zoom=1&w=457&h=302&iact=hc&vpx=119&vpy=251&dur=1512&hovh=182&hovw=276&tx=123&ty=84&ei=PfPYTNC1AYuwuAOQzIDICA&oei=PfPYTNC1AYuwuAOQzIDICA&esq=1&page=1&tbnh=117&tbnw=177&start=0&ndsp=15&ved=1t:429,r:5,s:0

11Reaching Open Source CommunitiesLinking semantic web with web developers

• Social Semantic Web extensions/modules to popular CMS, e.g. Semantic Wiki, Drupal

• Process/consume integrated gov data in a number of different ways: social networks, natural language technologies, workflows, search…

http://www.google.com.hk/imgres?q=drupal&um=1&hl=zh-CN&safe=strict&client=firefox-a&sa=N&rls=org.mozilla:en-US:official&source=og&biw=808&bih=551&tbs=isch:1&tbnid=Z2e0zN1mO9s0FM:&imgrefurl=http://www.chinaz.com/Webbiz/Free/041M29152009.html&imgurl=http://www.chinaz.com/upimg/userup/0903/251GGW641.jpg&zoom=1&w=349&h=400&iact=hc&vpx=118&vpy=84&dur=185&hovh=240&hovw=210&tx=94&ty=129&ei=0ULLTLTBJ4LyvwPA6IDEDw&oei=0ULLTLTBJ4LyvwPA6IDEDw&esq=1&page=1&tbnh=121&tbnw=106&start=0&ndsp=12&ved=1t:429,r:0,s:0

http://research.microsoft.com/c/1040

http://www.google.com/imgres?imgurl=http://www.locallytype.com/images/yahoo_boss_logo_340x146.png&imgrefurl=http://www.locallytype.com/2008/10/24/yahoo-boss-hack-evening-in-london/&usg=__7z_wmdaPHsHuqfGNbNacbbIVIQI=&h=146&w=340&sz=23&hl=en&start=5&um=1&itbs=1&tbnid=bX2fSSkix6cdsM:&tbnh=51&tbnw=119&prev=/images%3Fq%3Dyahoo%2Bboss%26um%3D1%26hl%3Den%26client%3Dfirefox-a%26sa%3DN%26rls%3Dorg.mozilla:en-US:official%26tbs%3Disch:1

http://upload.wikimedia.org/wikipedia/en/a/ad/Yahoo_Logo.svg

http://www.google.com/imgres?imgurl=http://briancroxall.net/TimelineTutorial/timelineTutorialPics/SimileLogo.png&imgrefurl=http://briancroxall.net/TimelineTutorial/WritingWebPageBody.html&usg=__nmatvE0YTLZdixdtCIOR_jf50dc=&h=84&w=138&sz=6&hl=en&start=8&um=1&itbs=1&tbnid=CKBoFNtmAiy4_M:&tbnh=57&tbnw=93&prev=/images%3Fq%3Dsimile%2Bmit%2Blogo%26um%3D1%26hl%3Den%26safe%3Dactive%26client%3Dfirefox-a%26rls%3Dorg.mozilla:en-US:official%26tbs%3Disch:1

http://www.google.com.hk/imgres?q=ibm&um=1&hl=zh-CN&newwindow=1&safe=strict&client=firefox-a&sa=N&rls=org.mozilla:en-US:official&source=og&biw=1104&bih=540&tbs=isch:1&tbnid=pQtsZ1HWHWuE0M:&imgrefurl=http://www.ubuntuchina.com/viewthread.php%253Ftid%253D5721&imgurl=http://blogs.sun.com/jonathan/resource/ibm-logo.jpg&zoom=1&w=609&h=330&iact=hc&vpx=347&vpy=118&dur=1193&hovh=165&hovw=305&tx=128&ty=92&ei=nd3ZTMKBIoHuvQPe35HDCA&oei=nd3ZTMKBIoHuvQPe35HDCA&esq=1&page=1&tbnh=83&tbnw=154&start=0&ndsp=18&ved=1t:429,r:2,s:0

http://www.google.com.hk/imgres?q=facebook+logo&um=1&hl=zh-CN&newwindow=1&safe=strict&client=firefox-a&rls=org.mozilla:en-US:official&source=og&biw=1104&bih=540&tbs=isch:1&tbnid=zJxMgbsojbq3jM:&imgrefurl=http://www.hotels-saintgermain.fr/&imgurl=http://www.hotels-saintgermain.fr/wp-content/uploads/2010/09/facebook-logo.jpg&zoom=1&w=800&h=301&iact=hc&vpx=71&vpy=141&dur=341&hovh=59&hovw=156&tx=227&ty=59&ei=JT7aTLjVBYfmvQO_rfjzCQ&oei=Ez7aTKytKJC-vgOqj4G4CA&esq=3&page=1&tbnh=59&tbnw=156&start=0&ndsp=18&ved=1t:429,r:0,s:0

http://www.google.com.hk/imgres?q=twitter+logo&um=1&hl=zh-CN&newwindow=1&safe=strict&client=firefox-a&rls=org.mozilla:en-US:official&source=og&biw=1104&bih=540&tbs=isch:1&tbnid=HlgeaHb1DzF49M:&imgrefurl=http://www.brightscope.com/blog/&imgurl=http://www.brightscope.com/blog/wp-content/uploads/2009/03/twitter-logo.jpg&zoom=1&w=800&h=295&iact=hc&vpx=204&vpy=263&dur=888&hovh=136&hovw=370&tx=159&ty=74&ei=Rz7aTN24NJHovQP79oW5CA&oei=Rz7aTN24NJHovQP79oW5CA&esq=1&page=1&tbnh=63&tbnw=171&start=0&ndsp=18&ved=1t:429,r:7,s:0

12

TWC LOGD Status: Website Statistics

• 378,128 page hits • 28,481 visits• 16,041 visitors• 4126 cities• 34 countries

Note: the above statistics are about http://data-gov.tw.rpi.edu. Dataset access not counted.

http://data-gov.tw.rpi.edu/

13

Summary of the TWC LOGD Portal

Real World Data8.5+ billion triples 400+ datasets10+ sources Many domainsSemantic Web Technology completely open source Demos/tutorials/videosCommunity and Users partner of US government open source community education in university

http://logd.tw.rpi.edu

Linking Open Government Data Now!

http://logd.tw.rpi.edu/

14

The Team and Sponsors

• Jim Hendler• Deborah L. McGuinness• Li Ding• Dominic DiFranzo• Sarah Magidson• James Michaelis • Alvaro Graves• Jin Guang Zheng • Xian Li• Gregory Todd Williams• Tim Lebo• Zhenning Shangguan• Devin Gaffney• Peter Coons• Adam Bell• William Cooper • Brian Zaik• Johanna Flores

Government Sponsors

DARPANSFNASAIARPANIH/NCI…

15

Backups

16Data.gov and World-Wide Open Government Data Activities

Janu

ary

1,

2009

“Openness will strengthen our democracy and promote efficiency and effectiveness in Government.”

--- President Obama

Putting Government Data online

May

21,

20

09

Janu

ary

19,

2010

data.gov.uk onlineM

ay 2

1,

2010

data.gov online data.gov relaunchwith semantic webfeatured

June

30,2

009

2009 2010 …

Many countries• US• UK• Australia• New Zealand …

17

Data-gov Wiki: Innovations at RPI

The Data-gov Wiki explores and educates the use of semantic web technologies, esp. linked data, in producing, processing and utilizing government data from data.gov.

The Data-gov Wiki is run by the Tetherless World Constellation at RPI, headed by Professors Jim Hendler and Deborah McGuinness and led by Li Ding. Other student team members include: Dominic DiFranzo, Sarah Magidson ,James Michaelis, Alvaro Graves, Adam Bell, Jin Guang Zheng, Xian Li, Tim Lebo, Gregory Todd Williams, Peter Coons, Zhenning Shangguan, Devin Gaffney, William Cooper, Brian Zaik, and Johanna Flores .

40+ Demos 400+ Datasets Tutorials & Videos

18

Dataset

Version

TableSource

Record

ConversionLayer

OGD (part1)Snapshot

LOGD(raw)

LOGD(e1)

high Levels of structural data granularity low

Dat

a pu

blis

hing

sta

ges

OGD (part2)Snapshot

…

…

…

Tech: Abstraction and Versioning

19

Tech: Provenance in LOGD data

Convert

derive derive

create

derive

revision

Access

Enhance

Version

SemDiff

http://images.google.com/imgres?imgurl=http://www.rotetinte.com/chart2008/iconCSV.gif&imgrefurl=http://www.rotetinte.com/chart2008/results-1.htm&usg=__XwSISwrZDdGXXgPTXARBXLyyYVw=&h=50&w=50&sz=3&hl=en&start=2&um=1&tbnid=Lh5exFkoMowafM:&tbnh=50&tbnw=50&prev=/images?q=csv+icon&hl=en&client=firefox-a&rls=org.mozilla:en-US:official&um=1

http://images.google.com/imgres?imgurl=http://www.mckinneytexas.org/Associations/144/data/images/home/1056/RSS%20FEED%20Icon.jpg&imgrefurl=http://www.mckinneytexas.org/homeright.asp?id=&o_n=Library%20Home&aid=144&Category=Library&usg=__J55EohbeyISO8l9bbCV0A0ZPASs=&h=300&w=300&sz=90&hl=en&start=4&um=1&tbnid=euie1pYC7IefuM:&tbnh=116&tbnw=116&prev=/images?q=rss+feed&hl=en&client=firefox-a&rls=org.mozilla:en-US:official&um=1

http://images.google.com/imgres?imgurl=http://reareports.realestate.com.au/images/icons/news_archive_icon.jpg&imgrefurl=http://reareports.realestate.com.au/news_archive.do&usg=__-n4dA-553aUwh-azFt-5RyHPZMM=&h=201&w=223&sz=9&hl=en&start=15&um=1&tbnid=uMpHYrk8TMgLXM:&tbnh=96&tbnw=107&prev=/images?q=archive+icon&hl=en&client=firefox-a&rls=org.mozilla:en-US:official&um=1

http://images.google.com/imgres?imgurl=http://www.rotetinte.com/chart2008/iconCSV.gif&imgrefurl=http://www.rotetinte.com/chart2008/results-1.htm&usg=__XwSISwrZDdGXXgPTXARBXLyyYVw=&h=50&w=50&sz=3&hl=en&start=2&um=1&tbnid=Lh5exFkoMowafM:&tbnh=50&tbnw=50&prev=/images?q=csv+icon&hl=en&client=firefox-a&rls=org.mozilla:en-US:official&um=1

20

XHTML+RDFa

ARC2

http://data-gov.tw.rpi.edu/

Data-gov Semantic Search

HTML

RDFa

WebSearchResults

RDF

Annotation

Consume LOGD data in Semantic Search

http://www.google.com/imgres?imgurl=http://www.locallytype.com/images/yahoo_boss_logo_340x146.png&imgrefurl=http://www.locallytype.com/2008/10/24/yahoo-boss-hack-evening-in-london/&usg=__7z_wmdaPHsHuqfGNbNacbbIVIQI=&h=146&w=340&sz=23&hl=en&start=5&um=1&itbs=1&tbnid=bX2fSSkix6cdsM:&tbnh=51&tbnw=119&prev=/images%3Fq%3Dyahoo%2Bboss%26um%3D1%26hl%3Den%26client%3Dfirefox-a%26sa%3DN%26rls%3Dorg.mozilla:en-US:official%26tbs%3Disch:1

http://upload.wikimedia.org/wikipedia/en/a/ad/Yahoo_Logo.svg

http://www.google.com/imgres?imgurl=http://www.247ha.com/images/rdf.jpg&imgrefurl=http://www.247ha.com/mark_logic.html&usg=__lAzcJNZWffL5x7H7cEsCVhI9g88=&h=500&w=458&sz=44&hl=en&start=3&um=1&itbs=1&tbnid=FnyC4_HzbWEJDM:&tbnh=130&tbnw=119&prev=/images%3Fq%3Drdf%26um%3D1%26hl%3Den%26client%3Dfirefox-a%26rls%3Dorg.mozilla:en-US:official%26tbs%3Disch:1

Documents

TWC LOGD: A Portal for Linking Open Government Data Dominic DiFranzo, Li Ding, John S. Erickson, Xian Li, Tim Lebo, James Michaelis, Alvaro Graves, Gregory