Introduction to XQuery and eXist

Week 21

DSA

DSA - XQuery 2

• Refresher on XPath• XML databases• XQuery Applications

– Whiskies (again)– A simple blog

• The XQuery Wikibook

DSA - XQuery 3

XPath (1)

• Hierarchical file systems have been navigable with path expression since Unix– /abc/cde/../../efg i.e. ?

• Problem – Only child, parent and root can be accessed

in one step– Only one node addressable

DSA - XQuery 4

XPath (2)

• Solution – extend the path language– select multiple items

• /abc/cde/../../efg/* - all children of the node /efg• //x - all x nodes anywhere in the tree

– select a subset of items by appending predicate (filter) – nodes selected if predicate evaluates to true• //x[@size>1000]

– functions in predicate• //x[ends-with(@name,’.jpeg’)]

– multiple filters• //x[@owner=‘fred’]//images[ends-with(@name,’.jpeg’)]

– select items by position• /abc[2] short for /abc[position() = 2]

DSA - XQuery 5

XPath (3)

• Core data type in XPath 2.0 is a sequence of items – Items are atomic values or XML elements– Sequence is flat

• deep-equal('fred',('fred')) is true• deep-equal((‘fred’, ‘fred’), (‘fred’)) is false• deep-equal(('fred', ('bill','joe')),('fred', 'bill','joe')) is true

– () is the empty sequence• count((‘fred’,() )) is 1

• Equality between sequences defined as non-empty intersection

• (‘fred’,’joe’) = ‘joe’ is true• (‘fred’,’joe’) = (‘joe’,’bill’) is true• (‘fred’,’joe’) eq ‘joe’ is false

DSA - XQuery 6

XQuery .• Problem

– Need to be able to construct a tree as well as select nodes in an existing tree– Need for more complex searches a la SQL

• Solution 1 – XML addition to existing languages – PHP + Simple XML – XML structures not compatible with language

• Solution 2 – XSLT – Push (declarative, pattern-matching) or pull processing– XML- based– Two versions

• 1.0 most common, in Browsers, Xalan• 2.0 e.g. Saxon

• Solution 2 – XQuery – Functional– Pull only – Non XML

• XQuery 1.0 and XSLT 2.0 share the same XPath 2.0 data model and function library

DSA - XQuery 7

eXist Native XML Database

• Open source• Wolfgang Meier is the chief architect• Written in Java• Deployable in different ways

– Embedded in a Java application– Part of a Cocoon pipeline– As web application in Apache/Tomcat– With embedded Jetty HTTPserver (as on stocks)

• Multiple Interfaces– REST – to Java servlet – SOAP– XML:RPC

DSA - XQuery 8

Native XML database

• Well-formed XML documents can be added to the database

• They are stored in an efficient, searchable B+ tree structure

• Documents (files) are organised into collections in a filestore

• Non-XML resources (XQuery, CSS, JPEG ..), etc can be stored as binary

DSA - XQuery 9

Whisky example

• A base XML file • Some XQuery Scripts

• Simple list• Another simple list• Select Whiskies• Select with a Form• Generate a kml map

DSA - XQuery 10

Simple Xquery – list1.xql

for $d in //Distilleryreturn <div> <h2>{$d/Name}</h2> <p>{$d/WhiskyDescription}</p> </div>

DSA - XQuery 11

Executing an XQuery

eXist DB

a.xql

XQuery Engine

parameters

html

Client Browser eXist: Server

Get a.xqlparameters

servlet

fetch a.xql

render

User clickslink

DSA - XQuery 12

Simple XQuery – list2.xql

declare option exist:serialize "method=xhtml media-type=text/html";<table border='2'> <tr> <th>Name</th><th>Address</th> </tr> {for $w in //Distillery return <tr> <td>{data($w/Name)}</td> <td>{data($w/Address)}</td> </tr> } </table>

DSA - XQuery 13

declare option exist:serialize "method=xhtml media-type=text/html";

<table border='2'> <tr> <th>Name</th><th>Address</th> </tr> { for $w in //Distillery return <tr> <td>{data($w/Name)}</td> <td>{data($w/Address)}</td> </tr> } </table>

XQuery explainedXQuery ‘variable’

Setting the output mime type

XPath expression to select nodes

XQuery inside XML

Another XPath

expression

Function to get the text

DSA - XQuery 14

XQuery ..• Nesting XML and XQuery

– XML to XQuery • { … }

– XQuery to XML <tag> … </tag>

• Must be well-formed XML – single root - <div> and <span> handy

• Constructed structures– XML element

<Position latitude=’51’ longitude=‘-2.5’ />or element Position {

attribute latitude ’51’,attribute longitude ‘2.5’}

– Sequence• ('fred', <tag a='4'/>, 5)

DSA - XQuery 15

XQuery ..

• control constructs are expressions and hence composable– if (cond) then exp else exp– for ..let.. where .. order by .. return – FLWOR– function call– variable value ($d)

• ‘Atomisation’ sometimes needed in output– Element to characters– $d/Name

• <Name>Glenfiddich</Name> – data($d/Name)

• Glenfiddich

DSA - XQuery 16

XQuery …• Functions and Modules

– Typed arguments and return– Recursion– XPath functions– eXist functions

• Database management• HTTP interface• ..

• Functional language– let $x := 5 binds the value 5 to the variable $x– Can’t write iterative code – what’s the output?

• let $y := 1• return• for $x in (1 to 5)• let $y := $y * 2• return $y

DSA - XQuery 17

XQuery FLWOR expression• FLWOR

for $x in sequencelet $a := expression, $b := expressionwhere conditionorder by $varreturn expression

– Returns a Sequence of nodes• Compare with SQL

select columnsfrom tableswhere conditionorder by

– Returns a Relation (table)

DSA - XQuery 18

Search Query

• Enter a name or part name• Match against the Distillery name• List the matches

DSA - XQuery 19

List 3

let $name := request:request-parameter("name","")

return <table border='2'> <tr><th>Name</th><th>Address</th></tr> {for $d in //Distillery[contains(Name,$name)] return <tr><td>{data($d/Name)}</td> <td>{data($d/Address)}</td> </tr> } </table>

eXist function

FLWOR expression

DSA - XQuery 20

An XQuery sticky form

• The script here has several shifts between XML to XQuery

• Whole interface in one script – Equivalent to PHP + MySQL.

DSA - XQuery 21

let $name := request:request-parameter("name","")return

<html>

<form method="get" action=""> <label>Enter Name or part of Name</label> <input type="text" name="name" size="10"/> </form>

<table border='2'> <tr><th>Name</th><th>Address</th></tr> { for $d in //Distillery[contains(Name,$name)] return <tr> <td>{data($d/Name)}</td> <td>{data($d/Address)}</td> </tr> } </table></html>

list4.xql

DSA - XQuery 22

Generate a kml overlay

• Output is kml• Need to specify the mime type so that the

browser will link to Google Earth (if set up correctly)

• Or create a Google Map link:• http://maps.google.com/maps?q=http://www.cems.

uwe.ac.uk/xmlwiki/whisky/map.xql

DSA - XQuery 23

Kml declare option exist:serialize "method=xhtml indent=yes media-type=application/vnd.google-earth.kml+xml";

<Document> <name>Distilleries of Scotland 2</name> {for $d in //Distillery return <Placemark> <name>{data($d/Name)}</name> <description>{data($d/WhiskyDescription)}</description> <Point> <coordinates> {data($d/Position/@longitude)}, {data($d/Position/@latitude)},0 </coordinates> </Point> </Placemark> } </Document>

distkml.xql

DSA - XQuery 24

Reading an API

• Brooklyn Museum API• Simple search

– Returning XML – Returning HTML

DSA - XQuery 25

Sourcelet $uri := "http://www.brooklynmuseum.org/opencollection/api/"let $apikey := "yPbZKsKZLC"let $keyword:= request:get-parameter("keyword","Anubis")let $request := concat($uri,"?api_key=",$apikey,"&amp;version=1&amp;include_item_caption=true&amp;require_image=true&amp;method=collection.search&amp;keyword=",$keyword)return doc($request)

DSA - XQuery 26

Uploading DSA CW2 metadata

• upload.xql

DSA - XQuery 27

FOLD and StudentsOnline

• StudentsOnline

DSA - XQuery 28

XQuery Wikibook

• A large number of worked examples– http://en.wikibooks.org/wiki/XQuery

DSA - XQuery 29

Trying eXist

• Demo site– http://exist-db.org/

• Local installation– Simple install from the exist-db site– Use the Java client to

• load files• move, copy, rename files• edit files in situ (but no Save-as)• execute queries• backup

DSA - XQuery 30

Other Native XML databases

• MarkLogic– MarkMail

• Sedna• Berkleydb• Many Relational databases include partial

support for XML