DavQL - Querying WebDAV Properties · Idea WebDAV provides: file sharing via HTTP(S) (firewall friendly) distinct methods for file management GET, PUT, DELETE, MOVE, COPY, LOCK,

DavQL - Querying WebDAV Properties

Syntax : Development and Challenges

Mike Becker, 4 June 2015

This talk

What this talk is not about:

it’s not about semantics

it’s not (really) about implementing languages

it’s not about designing Turing complete languages

it’s not reading a spec (you can do this by yourself)

What this talk is about:

it’s about concrete syntax

it’s about a domain specific language

it’s about why things are as they are

4 June 2015 Mike Becker DavQL - Querying WebDAV Properties Page 2

What is WebDAV?

Web-based Distributed Authoring and Versioning

For historians: RFC 2518For us: RFC 4918

But there is also more advanced stuff in other RFCs


Idea

WebDAV provides:

file sharing via HTTP(S) (firewall friendly)

distinct methods for file managementGET, PUT, DELETE, MOVE, COPY, LOCK, UNLOCK and MKCOL

more methods to get or set (custom) propertiesPROPFIND, PROPPATCH

But: output format is XML (so programs need to parse that)

So how about querying these properties with an SQL-like language?

Make it so.


Quick EBNF Crash-Course

NonTerminal = " terminal "| OtherNonTerminal| A l t e r n a t i v e P r o d u c t i o n| { A r b i t r a r y C o n c a t i n a t i o n }| [ Opt iona l ]| ?Specia lClass?| ?Specia lClass? − "w/ o th is "| Sequence , Of , " terminals " , And , NonTerminals| Choice , ( " within " | " in " ) , "a" , Sequence;

And in this case we forget about white spaces that are not either

explicitly part of a terminal

part of a literal or identifier

part of a path string


Idea for setting properties

SetStatement = " set " , SetExpressions ," at " , Path ,[ " with " , WithClause ] ,( " where " , Log ica lExpress ion ) | " anywhere" ;

Not today!


Idea for selecting properties

Let’s first look at an example:

select contentlength, lastmodifiedfrom /path/to/ collectionwhere lastmodified > %torder by 2 desc

We have:fields, a path, conditions, a placeholder and an order criterion


Coarse syntax for selecting properties

SelectStatement = " select " , F ie ldExpress ions ," from " , Path ,[ " with " , WithClause ] ,[ " where " , Log ica lExpress ion ] ,[ " order by " , OrderByClause ] ;

Let’s have a look at each of them non terminals:

1. WithClause very easy (contains key value pairs only)2. Path easy (it might be just a string)3. OrderByClause medium (some choices and options)4. FieldExpressions hard (needs lots of rules)5. LogicalExpressions interesting (promised!)


Some prerequisites

Keyword = " select " | " set " | "from" | " at " | "as"| "where" | "anywhere" | " l i k e " | " unlike "| "and" | "or" | "not" | "xor" | " with " | " i n f i n i t y "| "order" | "by" | "asc" | "desc" ;

L i t e r a l = Number | S t r i n g | Timestamp ;Number = ? D i g i t ? , {? D i g i t ?} | "%d" ;S t r i n g = " ’ " , {? Character? − " ’ " | " ’ ’ ’ " } , " ’ " | "%s" ;Timestamp = "%t " ;

I d e n t i f i e r = I d e n t i f i e r C h a r − ? D i g i t ? , { I d e n t i f i e r C h a r }| " 8 " , ?Character?−" 8 " , {? Character?−" 8 " } , " 8 " ;

I d e n t i f i e r C h a r = ?Character? − ( " " | " , " ) ;


With clause and Path

WithClause = "depth" , "=" , ( Number | " i n f i n i t y " ) ;

Path = S t r i n g| " / " , [ PathNode , { " / " , PathNode } ] , [ " / " ] ;

PathNode = { { ? Character? − " / " } − Keyword } ;

The second rule for Path is merely a convenience feature.

Lesson:Good things for users are (often) bad things for language designers.


With clause and Path

Examples for with clause:

with1. depth = 02. depth = infinity3. depth = %d4. depth = −2 # syntactically correct − but it will throw an error !

Examples for paths:

from1. /2. /some/path3. / this /contains two spaces/4. ’ /here/with/keyword’5. %s


Order by clause

OrderByClause = OrderByCr i te r ion , { " , " , OrderByCr i te r ion } ;OrderByCr i te r ion = ( I d e n t i f i e r | Number ) , [ " asc" | " desc" ] ;

Straight forward - examples:

order by1. 1, 2, %d2. lastmodified desc, 23. name asc, creationdate desc4. name, %d desc


Field list

The basic idea:

Fie ldExpress ions = "−"| " ∗ " , { " , " , NamedField }| F ie ldExpress ion , { " , " , F ie ldExpress ion } ;

F ie ldExpress ion = NamedField | I d e n t i f i e r ;

NamedField = Expression , " as " , I d e n t i f i e r ;

dash (-) for selecting no properties (just resources)

star (*) for selecting any property

specific properties denoted by identifiers

arbitrary expressions (they need an alias name!)


Expression

But what is an expression?


Expression

Examples:

42

contentlength

1 + a

replacenull (someprop, ’n/a’)

−b

(5 + a) ∗ (3 − b)


Expression

This motivates the following approach:

Expression = Expression , BinaryOperator , Expression| UnaryOperator , Expression| Func t ionCa l l | I d e n t i f i e r | L i t e r a l| " ( " , Expression , " ) " ;

Func t ionCa l l = I d e n t i f i e r , " ( " , ArgumentList , " ) " ;ArgumentList = Expression , { " , " , Expression } ;

UnaryOperator = "−" | "~" ;B inaryOperator = "+" | "−" | " ∗ " | " / " | "&" | " | " | "^" ;

Looks good? It is not.


Expression


Some questions:

How is operator precedence handled?

How does the syntax tree for (a+b)+c look like?

Can you implement this right away?


Expression


Answers:

Operator precedence is not expressed by the syntax.

(a+b)+c possibly has no syntax tree and is invalid.

Cannot be implemented with recursive parsing (no LL(1) grammar)


Expression


Lesson:A grammar might look nice. But parsers don’t care for aesthetics.


Expression

Next try:

Expression = AddExpression ;AddExpression = MultExpression , [ AddOperator , AddExpression ] ;Mul tExpression = Bi twiseExpress ion , [ MultOperator , Mul tExpression ] ;B i tw iseExpress ion = UnaryExpression , [ B i twiseOperator , B i tw iseExpress ion ] ;UnaryExpression = [ UnaryOperator ] , ( ParExpression | AtomicExpression ) ;AtomicExpression = Func t ionCa l l | I d e n t i f i e r | L i t e r a l ;ParExpression = " ( " , Expression , " ) " ;

B i tw iseOpera tor = "&" | " | " | "^" ;Mul tOperator = " ∗ " | " / " ;AddOperator = "+" | "−" ;UnaryOperator = "+" | "−" | "~" ;


Expression


Expression

Abbreviated notation can make things clearer:

Expr = Add ;

Add = Mult , [ "+" | "−" , Add ] ;Mul t = B i tw ise , [ " ∗ " | " / " , Mul t ] ;B i tw ise = Unary , [ "&" | " | " | "^" , B i tw ise ] ;

Unary = [ "+" | "−" | "~" ] , ( Par | Atomic ) ;

Atomic = Func t ionCa l l | I d e n t i f i e r | L i t e r a l ;Par = " ( " , Expr , " ) " ;

The binary operations follow the same principle which is based on a stack.⇒ This allows recursive parsing!


Expression - Function calls

Oh wait - we didn’t talk about function calls yet:

Func t ionCa l l = I d e n t i f i e r , " ( " , [ ArgumentList ] , " ) " ;

ArgumentList = Expression , { " , " , Expression } ;

Question: how should that end up in a binary syntax tree?


Expression - Example for a function call


Achievements

What can we do until now? Some examples:

select name, contentlength, lastmodifiedfrom /mycollection with depth = infinityorder by 1, lastmodified desc

select name, isnull(cryptokey) as encrypted from %s

select ∗ , substr(name, lastindexof(name, ’. ’ ) ) as fname_extfrom ’/ collection /with/ files ’order by fname_ext

Last thing missing: where clause.


Where clause

Reminder:

[ " where " , Log ica lExpress ion ]

Examples:

where1. name like ’ talk%’2. lastmodified >= %t3. name unlike ’%.out’4. not iscollection and (fname_ext = ’.c’ or fname_ext = ’.h’ )


Logical Expression

This motivates the following approach:

Logica lExpress ion = Logica lExpress ion , Logica lOperator , Log ica lExpress ion| "not " , Log ica lExpress ion| Expression , Comparison , Expression| Expression , ( " l i k e " | " unlike " ) , S t r i n g| " ( " , Logica lExpress ion , " ) " ;

Log ica lOpera tor = " and " | " or " | " xor " ;Comparison = | "=" | "<" | ">" | "<=" | ">=" | " != " ;

Looks good again? Guess what - it is not.

Bonus: we can’t even use a single identifier as condition!


Logical Expression

This is the correct result:

Logica lExpress ion = BooleanExpression , [ Logica lOperator , Log ica lExpress ion ] ;BooleanExpression = "not " , BooleanExpression

| " ( " , Logica lExpress ion , " ) "| BooleanPrimary ;

BooleanPrimary = Expression , ( " l i k e " | " unlike " ) , S t r i n g| Expression , Comparison , Expression| Func t ionCa l l | I d e n t i f i e r ;

Note: here logical operators are equivalent in its binding strength! (not every language)


Statistics

Finally some facts:

only about 80 lines of grammar specification

37 non terminals

round about 1500 lines of parser code

set statement missing (yet)

some semantics enforced by grammar

generally no semantic analysis (yet)

language extensions will follow (group by, join, ...)


Documents

DavQL - Querying WebDAV Properties · Idea WebDAV provides: file sharing via HTTP(S) (firewall friendly) distinct methods for file management GET, PUT, DELETE, MOVE, COPY, LOCK,