Linked Data at the National Széchényi Library : road to the publication

Linked Data at the National Linked Data at the National Széchényi Library : road to the Széchényi Library : road to the

publicationpublication

SWIB10 : SEMANTIC WEB IN BIBLIOTHEKEN SWIB10 : SEMANTIC WEB IN BIBLIOTHEKEN

Cologne, 29–30 November, 2010 Cologne, 29–30 November, 2010

ÁdámÁdám HorváthHorváth

National Széchényi LibraryNational Széchényi Library

2 Linked Data at the National Széchényi Library - Ádám Horváth - NSZL

ContentsContents

Why I am hereWhy I am hereBackground information on NSZLBackground information on NSZLRoad to the publicationRoad to the publicationCurrent developments and future plansCurrent developments and future plans


The newsThe news

The National Széchényi Library (NSZL) has The National Széchényi Library (NSZL) has recently published its entire OPAC and recently published its entire OPAC and Digital Library and the corresponding Digital Library and the corresponding authority data as Linked Open Data.authority data as Linked Open Data. (2010.04.20)(2010.04.20)


The newsThe news

The used vocabularies areThe used vocabularies are– RDFDC for bibliographic data,RDFDC for bibliographic data,– FOAF for nameFOAF for namess, and, and– SKOS for subject terms and geographical SKOS for subject terms and geographical

namesnames


The newsThe news NSZL uses CoolURIsNSZL uses CoolURIs Every resource has both RDF and HTML Every resource has both RDF and HTML

representationrepresentation Our RDFDC, FAOF and SKOS statements are Our RDFDC, FAOF and SKOS statements are

linked together linked together Our name authority is matched with the DBPedia Our name authority is matched with the DBPedia

name filesname files URI aliases are handled as owl:sameAsURI aliases are handled as owl:sameAs

statementsstatements NSZL also supports the HTML link auto-discoveryNSZL also supports the HTML link auto-discovery


Information infrastructure of NSZLInformation infrastructure of NSZL

Integrated library systemIntegrated library system– AmicusAmicus

• Consortium systemConsortium system• ViewsViews• Oracle basedOracle based• Authority handlingAuthority handling• Do not handle all thesaurus relation typesDo not handle all thesaurus relation types• Products moduleProducts module• Z39.50 serverZ39.50 server



OPACOPAC– LibriVisionLibriVision

• HTMLHTML• XML and XSLT basedXML and XSLT based• Z39.50 clientZ39.50 client• Session basedSession based



Thesaurus handlingThesaurus handling– RelexRelex

• Contains general terms and geographical namesContains general terms and geographical names• It uses all possible relation type available in ISO2788It uses all possible relation type available in ISO2788• It also contains UDC equivalents and coordinatesIt also contains UDC equivalents and coordinates• Relex can produce MARC outputRelex can produce MARC output• Relex uses the descriptors themselves as identifiersRelex uses the descriptors themselves as identifiers• Subject terms in Amicus is based on RelexSubject terms in Amicus is based on Relex



MARC is HUNMARCMARC is HUNMARC– MARC21 basedMARC21 based– No punctuation marc in recordsNo punctuation marc in records– More subfieldsMore subfields– Punctuation is program generatedPunctuation is program generated– MARC21 tools and utilities can’t be usedMARC21 tools and utilities can’t be used



The IT department containsThe IT department contains– ProgrammersProgrammers– System librariansSystem librarians– Maintenance stuffMaintenance stuff



The IT department is responsible forThe IT department is responsible for– Integrated library systemIntegrated library system– Developing digital library Developing digital library – Developing other utilitiesDeveloping other utilities– Maintaining the whole IT infrastructureMaintaining the whole IT infrastructure

Not responsible for Not responsible for – DigitisationDigitisation– HomepageHomepage


Road to the publicationRoad to the publication


Motivation of our semantic web Motivation of our semantic web developmentsdevelopments

Personal interestPersonal interest Interested colleaguesInterested colleagues

– Kornél Horváth ([email protected])Kornél Horváth ([email protected])– Zsolt Zachár ([email protected])Zsolt Zachár ([email protected])

Semantic web is a cool thingSemantic web is a cool thingMy friends are also publishing their dataMy friends are also publishing their dataOur role is to provide dataOur role is to provide data


Carrying out the developmentCarrying out the development

There were no specific projectThere were no specific projectSmall developments pointing to the same Small developments pointing to the same

directiondirectionWe developed it when time permittedWe developed it when time permitted


The very first stepsThe very first steps

ONE2 projectONE2 project– Interoperability projectInteroperability project– Z39.50Z39.50– SRU emerged by the end of the projectSRU emerged by the end of the project– Semantic web was also mentionedSemantic web was also mentioned


SRU interface for AmicusSRU interface for Amicus

TEL-ME-MOR projectTEL-ME-MOR project– To make NSZL searchable on the TEL portal via To make NSZL searchable on the TEL portal via

SRUSRUYAZ Proxy was usedYAZ Proxy was used

– SRU/Z39.50 gatewaySRU/Z39.50 gateway– User defined XSLTUser defined XSLT– Result set is according to the TEL Application Result set is according to the TEL Application

profileprofile


SRU interface for AmicusSRU interface for Amicus

The important resultsThe important results– URL based searchURL based search– XML result setXML result set

• TTELAPELAP• MARCXMLMARCXML

– RDFDC is provided via YAZ ProxyRDFDC is provided via YAZ Proxy


Development of LibriUrlDevelopment of LibriUrl

The requirement was to The requirement was to – make LibriVision OpenUrl compatiblemake LibriVision OpenUrl compatible– provide access to OPAC records via URLprovide access to OPAC records via URL

The problem was that LibriVision isThe problem was that LibriVision is– Session based and requires loginSession based and requires login

LibriUrlLibriUrl– URL based search interface for the OPACURL based search interface for the OPAC– DDeveloped by NSZL on the bases of a vendor eveloped by NSZL on the bases of a vendor

software codesoftware code



LibriUrl search:LibriUrl search:

http://link.oszk.hu/libriurl.php?http://link.oszk.hu/libriurl.php?LN=en&DB=any&SRY=an&SRE=2616972LN=en&DB=any&SRY=an&SRE=2616972

LibriUrl side effectsLibriUrl side effects– Search for Amicus number is a link to a specific Search for Amicus number is a link to a specific

recordrecord– Our records became bookmarkable and linkableOur records became bookmarkable and linkable

and OpenSearchableand OpenSearchable


How does it work?How does it work?


How does it work?How does it work?




DemonstrationDemonstration


DemonstrationDemonstration



The importance of LibriUrlThe importance of LibriUrl– It was behind of our CoolURIIt was behind of our CoolURI

http://link.oszk.hu/libriurl.php?http://link.oszk.hu/libriurl.php?LN=en&DB=any&SRY=an&SRE=2616972LN=en&DB=any&SRY=an&SRE=2616972


SKOSifying the thesaurusSKOSifying the thesaurus

TelPlus projectTelPlus project– Thesaurus in SKOS for search refinement Thesaurus in SKOS for search refinement

SKOS conversionSKOS conversion– SKOS is converted from the MARC output of SKOS is converted from the MARC output of

RelexRelex– Not every thesaurus relation type is convertedNot every thesaurus relation type is converted– UDC and coordinates are also not includedUDC and coordinates are also not included



Concept identifier is an IRI compatible Concept identifier is an IRI compatible descriptor:descriptor:– 150 ** a 150 ** a abszurd drámaabszurd dráma– <skos:Concept<skos:Concept

rdf:about="http://nektar.oszk.hu/auth/rdf:about="http://nektar.oszk.hu/auth/abszurd_drámaabszurd_dráma">">



Serving SKOS to TelPlusServing SKOS to TelPlus– The SKOS XML file is indexed by Zebra and The SKOS XML file is indexed by Zebra and

served via SRUserved via SRU– http://193.6.201.195:9996/skos?http://193.6.201.195:9996/skos?

version=1.1&operation=searchRetrieve&query=version=1.1&operation=searchRetrieve&query=%22Gravenhage%22Gravenhage%22&startRecord=1&maximumRecords=10%22&startRecord=1&maximumRecords=10

– It is not RDF/XMLIt is not RDF/XML



Importance of SKOS developmentImportance of SKOS development– Having a conversion tool for creating SKOS Having a conversion tool for creating SKOS

recordsrecords


LIBRIS as an exampleLIBRIS as an example

ELAG, 2008, WageningenELAG, 2008, Wageningen– LIBRIS’ linked open data was presentedLIBRIS’ linked open data was presented– Content negotiationContent negotiation– Cool URICool URI– Link relLink rel


Focusing to the publicationFocusing to the publication

Realised that we have almost everything to Realised that we have almost everything to be able to publish our data as LODbe able to publish our data as LOD

We hadWe had– SKOSSKOS– LibriUrl (accessing OPAC records via URL)LibriUrl (accessing OPAC records via URL)– YAZ Proxy - SRU (URL based search in YAZ Proxy - SRU (URL based search in

Amicus)Amicus)– LIBRIS as an exampleLIBRIS as an example


Focusing to the publicationFocusing to the publication

What were missingWhat were missing– Name convention of resourcesName convention of resources– IdentifiersIdentifiers– Content negotiationsContent negotiations– RDFDCRDFDC– RDF databaseRDF database– FOAFFOAF– Link rel metatags in the OPAC headerLink rel metatags in the OPAC header– Creating linksCreating links


Name conventionName convention

Resource name for documentsResource name for documents– /resource/manifestation/2645471/resource/manifestation/2645471

Name for RDF representationName for RDF representation– /data/manifestation/2645471/data/manifestation/2645471

Name for HTML representationName for HTML representation– /hu/manifestation/2645471/hu/manifestation/2645471– /en/manifestation/2645471/en/manifestation/2645471


Name conventionName convention

Resource name for authorityResource name for authority– /resource/auth/33589/resource/auth/33589

Name for RDF representationName for RDF representation– /data/auth/33589/data/auth/33589

Name for HTML representationName for HTML representation– /auth/33589/auth/33589


IdentifiersIdentifiers

DocumentsDocuments• Amicus number (MARC 001)Amicus number (MARC 001)

Subject authority (thesaurus)Subject authority (thesaurus)• The descriptor itself with some conversion rulesThe descriptor itself with some conversion rules

NamesNames• Special number stored in the Amicus databaseSpecial number stored in the Amicus database


Content negotiationContent negotiation

Implementation of content negotiationImplementation of content negotiation– 303 redirection was chosen303 redirection was chosen


Thing

http://nektar.oszk.hu/resource/manifestation/2645471

RDF

303 redirectand

Contentnegotiation

applicatoin/rdf+xml wins text/html wins

Content location:http://nektar.oszk.hu/data/manifestation/2645471 Content location:

http://nektar.oszk.hu/hu/manifestation/2645471

HTML

DOCUMENTS (books, journals, etc.)


Thing

http://nektar.oszk.hu/resource/auth/33589

RDF

303 redirectand

Contentnegotiation


Content location:http://nektar.oszk.hu/data/auth/33589

Content location:http://nektar.oszk.hu/auth/33589

HTML

AUTHORITY ENTRIES (thesaurusterms, geo names, person names)


Creating RDF for catalogue recordsCreating RDF for catalogue recordsCreating „RDFDC”Creating „RDFDC”

XSLT does the jobXSLT does the job– It is MARCXML RDF/XML conversionIt is MARCXML RDF/XML conversion

• Modification of the MARC to TEL Application Modification of the MARC to TEL Application profile conversionprofile conversion

– Creates links to subjects and namesCreates links to subjects and namesUsed vocabulariesUsed vocabularies

– Dublin CoreDublin Core– BIBOBIBO


Installation of RDF databaseInstallation of RDF database

JenaJenaJoseki SPARQL endpointJoseki SPARQL endpoint


Creating FOAF for namesCreating FOAF for names

Batch processBatch process– The name index of Amicus is usedThe name index of Amicus is used

FOAF is stored in and served from JenaFOAF is stored in and served from JenaDuring update always the entire FOAF During update always the entire FOAF

dataset is rebuiltdataset is rebuilt


Contents of JenaContents of Jena

Names (FOAF)Names (FOAF)Subject authority (SKOS)Subject authority (SKOS)

– It is still available from Zebra via SRUIt is still available from Zebra via SRUCatalogue records (RDFDC)Catalogue records (RDFDC)All of our linked data can be searched All of our linked data can be searched

via SPARQLvia SPARQL


Creating HTML link auto-discoveryCreating HTML link auto-discovery

In the head of our OPAC extended In the head of our OPAC extended view pagesview pages

<link rel="meta" <link rel="meta" type="application/rdf+xml" type="application/rdf+xml" title="RDF Version" title="RDF Version" href="http://nektar.oszk.hu/data/manifestathref="http://nektar.oszk.hu/data/manifestation/2645471" />ion/2645471" />


Creating linksCreating links

Links to NSZL resourcesLinks to NSZL resources– Link from RDFDC to nameLink from RDFDC to namess and and

subjectsubjectss

<dcterms:creator <dcterms:creator rdf:resource="http://nektar.oszk.hu/resource/ardf:resource="http://nektar.oszk.hu/resource/auth/33589"/>uth/33589"/>

<dc:creator>Jókai Mór (1825-1904)</dc:creator> <dc:creator>Jókai Mór (1825-1904)</dc:creator>


Creating linksCreating links Links to external resourcesLinks to external resources

– Link from the name authority records to DBpediaLink from the name authority records to DBpedia

<foaf:Person <foaf:Person rdf:about="http://nektar.oszk.hu/resource/auth/33589">rdf:about="http://nektar.oszk.hu/resource/auth/33589">

<foaf:name>Jókai Mór (1825-1904)</foaf:name><foaf:name>Jókai Mór (1825-1904)</foaf:name>

<<owl:sameAs owl:sameAs rdf:resource="http://rdf:resource="http://dbpedia.orgdbpedia.org/resource/M/resource/M%C3%B3r_J%C3%B3kai"/>%C3%B3r_J%C3%B3kai"/>


Resolving the URL of documentsResolving the URL of documents

The URL of the documentThe URL of the document– /resource/manifestation/2645471/resource/manifestation/2645471

RDF is requestedRDF is requested– Redirection to /data/manifestation/2645471Redirection to /data/manifestation/2645471– PHP program gathersPHP program gathers

• MARCXMLMARCXML• Name idsName ids

– XSLT creates RDF/XML with linksXSLT creates RDF/XML with links



Link generation to SKOSLink generation to SKOS– Automatic conversion Automatic conversion

• from the literalfrom the literal– <dcterms:subject>abszurd dráma<dcterms:subject>abszurd dráma

</dcterms:subject></dcterms:subject>

• to the Conceptto the Concept– <dcterms:subject <dcterms:subject

rdf:resource="http://nektar.oszk.hu/resource/auth/abszurdrdf:resource="http://nektar.oszk.hu/resource/auth/abszurd_dráma"/>_dráma"/>



Link generation to FOAFLink generation to FOAF– The XSLT gets the name ids as parameters and The XSLT gets the name ids as parameters and

creates the linkscreates the links

<dc:creator>Jókai Mór (1825-1904)</dc:creator><dc:creator>Jókai Mór (1825-1904)</dc:creator>

<dcterms:creator <dcterms:creator rdf:resource="http://nektar.oszk.hu/resource/authrdf:resource="http://nektar.oszk.hu/resource/auth/33589"/>/33589"/>

Resolving the URL of a documentRDF is requested

Integrated Library System (Amicus)

SRU/Z39.50 gateway Servlet

Response RDF/XMLRequest

WebserverContent negotiation

Application program (PHP) XSLT

RDFDC with links to SKOS and FOAFRDF+XML accepted

SRU

Z39.50 MARC ISO2709

MARCXML ManifestId

JDBC NameIds

/resource/manifestation/2645471

/data/manifestation/2645471



Sample URL of a document Sample URL of a document /resource/manifestation/2645471/resource/manifestation/2645471

HTML is requestedHTML is requested– Redirection to Redirection to

/hu/manifestation/2645471 or /hu/manifestation/2645471 or /en/manifestation/2645471/en/manifestation/2645471

– Nektar proxy calledNektar proxy called



Nektar proxyNektar proxy– Java servlet (substitution of LibriUrl)Java servlet (substitution of LibriUrl)– It doesIt does

• Loging into LibriVision and getting session idLoging into LibriVision and getting session id• Searching for the manifestation id and getting Searching for the manifestation id and getting

query id, etcquery id, etc• Sending present request and getting the search Sending present request and getting the search

result pageresult page• Creating the final HTML pageCreating the final HTML page

– Replacing URL-s, filling in the header, etcReplacing URL-s, filling in the header, etc

Resolving the URL of a documentHTML is requested

LibriVision (OPAC)

Nektar proxy (Servlet)

Response HTMLRequest


OPAC extended viewtext/html accepted

/hu/manifestation/2645471

SessionId

/resource/manifestation/2645471

/hu/manifestation/2645471

Loging in Searching QueryId

Final OPAC record

View Raw OPAC record


Resolving the URL of authority dataResolving the URL of authority data

Sample URL of an authority record Sample URL of an authority record /resource/auth/33589/resource/auth/33589

RDF is requestedRDF is requested– Jena is searched via the Joseki SPARQL Jena is searched via the Joseki SPARQL

endpointendpoint– RDF/XML is given backRDF/XML is given back

Resolving the URL of authority dataRDF is requested

JenaJoseki SPARQL endpoint

Response RDF/XMLRequest


FOAF or SKOS in RDF/XMLRDF+XML accepted

SPARQL

/resource/auth/33589

/data/auth/33589

FOAF or SKOS in RDF/XML



Resolving the URL of authority dataResolving the URL of authority data

Sample URL of an authority record Sample URL of an authority record /resource/auth/33589/resource/auth/33589

HTML is requestedHTML is requested– Jena is searched via the Joseki SPARQL Jena is searched via the Joseki SPARQL

endpointendpoint– RDF/XML is given backRDF/XML is given back– XSLT creates the HTML pageXSLT creates the HTML page

Resolving the URL of authority dataHTML is requested

JenaJoseki SPARQL endpoint

Response HTMLRequest


FOAF or SKOS in HTMLtext/html accepted

SPARQL

/resource/auth/33589

/auth/33589

FOAF or SKOS in RDF/XML



Current activitiesCurrent activities

Improving RDFDCImproving RDFDC– Making RDFDC 100% compliant with the Making RDFDC 100% compliant with the

DCMI Metadata TermsDCMI Metadata Terms• Proper usage of dc:creator and dcterms:creatorProper usage of dc:creator and dcterms:creator• Better conversion of title and publisherBetter conversion of title and publisher



Improving SKOSImproving SKOS– Inserting UDC equivalent of Concepts Inserting UDC equivalent of Concepts – Inserting USE AND and other relationshipsInserting USE AND and other relationships– Inserting coordinates of geographical Inserting coordinates of geographical

namesnames



Improving FOAFImproving FOAF– Including organisationsIncluding organisations



VIAFVIAF– Creating links from our FOAF to VIAFCreating links from our FOAF to VIAF

Studying VirtuosoStudying Virtuoso– Jena might be changed to VirtuosoJena might be changed to Virtuoso

FRBRising the OPACFRBRising the OPAC



MethodMethod– Collecting works, expressions and Collecting works, expressions and

manifestations from the „MARC dump” of manifestations from the „MARC dump” of our catalogueour catalogue

– Creating RDF representations based on Ian Creating RDF representations based on Ian Davis: Expression of Core FRBR Concepts Davis: Expression of Core FRBR Concepts in RDFin RDF

– Loading the RDF representations into JenaLoading the RDF representations into Jena– Showing the work tree based on Showing the work tree based on

manifestation idmanifestation id



Scope (at first)Scope (at first)– Printed monographsPrinted monographs

• Excluding multivolumeExcluding multivolume

– This is a „post” FRBRisationThis is a „post” FRBRisationUsageUsage

– OPAC/Extended view/Link to the work treeOPAC/Extended view/Link to the work treeFirst resultsFirst results



Digital Library: OSZKDKDigital Library: OSZKDK– Mainly contains deposit copiesMainly contains deposit copies– It is also published as LODIt is also published as LOD

OAI-ORE is being implemented as a OAI-ORE is being implemented as a container of structural metadatacontainer of structural metadata– Used parts: resource map, aggregationUsed parts: resource map, aggregation– Used vocabularies: DC, BIBOUsed vocabularies: DC, BIBO– It is in test phase nowIt is in test phase now

Thing

http://oszkdk.oszk.hu/resource/DRJ/404

RDF

303 redirectand

Contentnegotiation


Content location:http://oszkdk.oszk.hu/data/DRJ/404

Content location:http://oszkdk.oszk.hu/DRJ/404

HTML

DIGITAL DOCUMENTS (books,journals, etc.)


Planned discovery of OAI-OREPlanned discovery of OAI-ORE

„„Link rel” in the head of the OPAC of Link rel” in the head of the OPAC of OSZKDKOSZKDK– <link rel="meta" <link rel="meta"

type="application/rdf+xml" type="application/rdf+xml" title=„OAI-ORE resource map" title=„OAI-ORE resource map" href="http://oszkdk.oszk.hu/rem/drj/869" />href="http://oszkdk.oszk.hu/rem/drj/869" />


One resource map of test OSZKDKOne resource map of test OSZKDK

<rdf:Description <rdf:Description rdf:about="rdf:about="http://oszkdk.oszk.hu/rem/drj/876http://oszkdk.oszk.hu/rem/drj/876">"> < <rdf:typerdf:type rdf:resource="http://www.openarchives.org/ore/termsrdf:resource="http://www.openarchives.org/ore/terms//ResourceMapResourceMap"/>"/>

<<dcterms:modifieddcterms:modified rdf:datatype="http://www.w3.org/2001/XMLSchema#drdf:datatype="http://www.w3.org/2001/XMLSchema#date">2010-04-07</dcterms:modified>ate">2010-04-07</dcterms:modified>

<<dcterms:creatordcterms:creator rdf:resource="http://oszkdk.oszk.hu/"/>rdf:resource="http://oszkdk.oszk.hu/"/>

<<ore:describes ore:describes rdf:resource="rdf:resource="http://oszkdk.oszk.hu/aggr/drj/876http://oszkdk.oszk.hu/aggr/drj/876"/>"/>

</rdf:Description></rdf:Description>

[email protected]@oszk.hu

Thank you for your attentionThank you for your attention

Technology

Linked Data at the National Széchényi Library : road to the publication