Publishing Repositories with mod_dav

Publishing Repositories with mod_dav

Greg SteinCollabNet, Inc.

[email protected], http://www.lyra.org/greg/

July 24, 2002

Greg Stein, CollabNet 222

Agenda

• WebDAV introduction

• mod_dav introduction

• The mod_dav API for backends– Example: mod_dav_fs– Example: mod_dav_svn

• Future

July 24, 2002


WebDAV IntroductionWhat is it? (1 of 2)

• Web-based Distributed Authoring and Versioning– “DAV” is the usual short form

• Goal: enable interoperability of tools for distributed web authoring

• Turns the Web into a writeable medium

July 24, 2002


WebDAV Introduction What is it? (2 of 2)

• Applies to all kinds of content - not just HTML and images

• Based on extensions to HTTP

• Uses XML for properties, control, status

• RFC 2518 and RFC 3253

July 24, 2002


WebDAV IntroductionScenarios

• Collaborative authoring

• Network file system

• Remote software engineering

• Unified repository-access protocol

July 24, 2002


WebDAV Introduction Some Tools and Clients

• Open source– Joe Orton’s sitecopy and cadaver– Nautilus (gnome-vfs), KDE (kio)– Subversion– Python, Perl, C, Java client APIs

• Commercial– MacOS X, most Adobe and Microsoft

products– Dreamweaver and other authoring tools

• Lots of (commercial) DAV servers

July 24, 2002


mod_dav Introduction

• mod_dav is an Apache module

• Separate module for Apache 1.3

• Integrated into Apache 2.0

• On June 1, securityspace.com reported 171505 sites using mod_dav (6th most popular Apache module)

July 24, 2002


mod_dav Introduction

• mod_dav has an API for a “pluggable back-end repository”– Over three years old, but still evolving– Suggestions welcome!

• The default back-end repository uses the native filesystem

• Oracle and ClearCase are shipping products using mod_dav with custom back-ends

July 24, 2002


mod_dav Back-ends

• Sometimes called “providers”

• Implemented as Apache modules

• Register themselves with mod_dav when they are first loaded

• Each has a short name for reference– Used in the DAV directive (e.g. DAV svn)

• Associated with a portion of the URL tree

July 24, 2002


mod_dav Back-ends

Apache 2.0

DSO loader

mod_dav

mod_dav_fs

FS

July 24, 2002


mod_dav Hooks

• mod_dav does the protocol work and most of the interaction with Apache

• Back-ends provide groups of functionality– Each functional group defines a set of

“hooks” (a vtable)– Some entire groups, and some individual

hooks, are optional• The provider fills in the groups and hooks

that it understands and can respond to

July 24, 2002


Groups of Functionality

• Content (“repository”) required

• “Dead” properties required

• “Live” properties optional, recommended

• Locks optional

• Versioning optional, very complex

• Binding optional, ill-defined

• Searching optional, ill-defined

July 24, 2002


How to Start

• Look at mod_dav_fs– Set up your Apache module– Register your provider with mod_dav

• Set up your dav_provider structure– Begin with the repository hooks– Stub out the dead property hooks– Leave the other groups as NULL

July 24, 2002


Key Structures

• dav_resource - center of the universe– The backend object being operated upon– The “info” field for your private data

• dav_error - returning errors• dav_response - returning multistatus

info

• See mod_dav.h

July 24, 2002


dav_hooks_repository

• get_resource

• open/close/seek/write_stream

• set_headers, deliver

• copy/move/remove_resource

• create_collection

• walk

• miscellaneous

July 24, 2002


get_resource

• Identify a resource based on the user request

• Four parameters– The Apache request_rec pointer– The root of this DAV-enabled space– Label header and DAV:checked-in

property

• Remember stuff in resource->info

July 24, 2002


Dead Properties

• Client-defined name/value pairs

• XML fragments for values– Namespaces, xml:lang

• Namespace management (dav_xmlns_info)– Share namespace definitions to shrink output– Coordinate among different types of

fragments that occur in a response

July 24, 2002


dav_hooks_propdb

• open, close

• define_namespaces, output_value

• map_namespaces, store

• remove

• exists

• first_name, next_name

• get_rollback, apply_rollback

July 24, 2002


Live Properties

• Server-defined properties– Typically, values are generated (not stored)

• Managed through Apache’s formal hook system and a mod_dav vtable

• Via the hooks, standard properties are handled by mod_dav, but can be overridden

• Modules can add their own properties

July 24, 2002


Live Property Hooks

• Apache hooks– gather_propsets– insert_all_liveprops– find_liveprop

• mod_dav hooks– insert_prop– is_writable– patch_validate, patch_exec, patch_commit,

patch_rollback (these are for PROPPATCH)

July 24, 2002


mod_dav_fsOverview

• Storage is in the filesystem

• Implements basic function groups– Repository– Dead and live properties– Locking

July 24, 2002


mod_dav_fsRepository

• “Collections” are directories

• “Member resources” are files

• Straight-forward mapping– Apache’s Alias directive handles all of

the URI to filesystem translation

• Eek! Permissions and ownership

July 24, 2002


mod_dav_fsDead Properties

• Each directory has a .DAV subdirectory

• DBM files for each file, plus one for the directory itself (all are lazily-created)– Simple key/value storage

• Need to copy/move/delete DBM files

July 24, 2002


mod_dav_fsLive Properties

• Most are pulled from stat()– DAV:getcontentlength– DAV:creationdate

– etc.

• Some require Apache sub-requests

• Custom property to tweak the “executable bit” on files – assists with uploading CGIs

July 24, 2002


mod_dav_fsLocks

• Oh, the pain…

• “Locknull” resources complicate matters

• Implementation– Server-wide lock database– Per-directory (text) file records the

locknull resources

July 24, 2002


Subversion

• Subversion (“SVN”) is an Open Source project to replace CVS– Well, replace and improve

• Network protocol is WebDAV– Apache 2.0 is the network server– mod_dav handles the frontend and

protocol, SVN handles the backend

• http://subversion.tigris.org/

July 24, 2002


mod_dav_svnOverview

• Storage is in Subversion’s repository

• Implements these function groups– Repository– Dead and live properties– Versioning (!)

• No locking

July 24, 2002


mod_dav_svnRepository

• All data is stored in Subversion

• Subversion models a filesystem– Move, copy, delete are simple API calls

• We need to generate the content for a GET

July 24, 2002


mod_dav_svnDead Properties

• The Subversion repository provides arbitrary properties for each file / directory

• Oh. Gee. Simple.

July 24, 2002


mod_dav_svnLive Properties

• Live properties are generated by querying the Subversion API

• Most of the live properties are for dealing with DeltaV

• Some custom live properties that simplify the client’s life

July 24, 2002


mod_dav_svnVersioning

• Big topic (DeltaV has a lot of stuff)

• SVN implements a subset of DeltaV– Subversion is not a true DeltaV server– Full compatibility is deferred for now

• Look at the code for more info

July 24, 2002


Your Repository

• Define mappings from URLs to “resources”– Resources can be anything: people,

documents, rows in a table

• Define what the operations mean for your repository (“move a person? that is changing their name”)

• Are dead properties applicable?

• What live props are available? Writable?

July 24, 2002


Future

• Evolve the mod_dav API– More integration with Apache, apr(-util)– Performance and memory usage changes– Documentation(!)

• New Open Source providers– Catacomb (a MySQL backend)– Shims to implement a backend in Perl,

Python, or PHP

Documents

Publishing Repositories with mod_dav