Getting MAD: San Francisco’s Master Address Database Project BAAMA January 28, 2010

Agenda

Getting MAD: San Francisco’s Master Address Database

Project

BAAMAJanuary 28, 2010

SFGIS

• Jeff Johnson – Enterprise GIS Coordinator, Department of

Technology, City and County of San Francisco– With City since 1993

Getting MAD

• Why the City needs an enterprise-level address system

• The key data (streets, parcels, and address numbers) – Extract, Transform and Load (ETL)

• Why Open Source • Why the Cloud

Why does the City need an enterprise-level address system?

An Address Problem

Assessor/Recorder Board of Supervisors

City Attorney

District Attorney Public Defender

Sheriff

Law Library

Board of Appeals

Dept. of Status of Women

Civil Service Commission

Taxi Commission

Children, Youth & Families

Rent Arbitration Board

Children and Families Commission

Police – Office ofCitizen Complaints

Health Services System

Human RightsCommission

Mayor

Asian Art Museum

Elections

Environment Economic & WorkforceDevelopment

War Memorial

Retirement SystemHuman Resources

Adult Probation

Fine Arts MuseumsChild Support Services

City Planning

Department ofBuilding Inspection

Port of San Francisco Controller

PUC – Hetch Hetchy

Treasurer/Tax Collector

Juvenile Probation

Department of Emergency Management

Public Works

Public Utilities Commission

PUC – Wastewater Enterprise

MTA – Parking and Traffic

PUC –Water Department

Administrative Services(Real Estate, et al.)

Recreation and ParkFire DepartmentHuman Services

Police

MTA - MUNI

Public Health

Ethics Commission

Art Commission

Department of Technology

many address lists in the City varying needs and uses

varying formats, standards, and levels of accuracy or completeness

The Situation

• Duplicate Address Data• Data inconsistencies• No central coordination• No trust• Silos of Data

The Issues

“Another silo successfully deployed” – Sam Valdez

What’s needed? MAD

• Master Address Database system (or EAS)• An authoritative, reliable source of common address

information • Easily maintained and updated• Enterprise-based, i.e., benefits more than one

department and utilized by all departments• Data model incorporates key datasets• Web services for application development and

system integration at department level

Assessor/Recorder

Department ofBuilding Inspection

Public Works

Who’s responsible for what?(according to City Code)

Parcels(Assessor Parcel Number)

Streets(Street Name)

Address Number

Key Data Elements

Assessor/Recorder

Public Works

Parcels(Assessor Parcel

Number)

Streets(Street Name)

Department ofBuilding Inspection

Address Number

Block 1244 Lot 022

Ashbury St600 – 698

634 Ashbury St

Why SFGIS for MAD?

• Existing enterprise function• Well-developed key datasets

– Parcels– Street Centerline Network– Address Points

• Ability to ramp up ETL from existing data store to MAD

Why SFGIS for MAD?

• SFGIS: established relationships with departments in place

• Funding made available through City’s IT steering committee, COIT (Formerly EISPC)

• Not tied to a particular business process or application or vendor platform

Why Open Source?

Good timing for Open Source

• Open source standards are prevalent– Web Feature Service (WFS), vector– Web Map Service (WMS), raster

• ‘Legacy’ open source paves the path– Apache most popular webserver since April 1996 – 54% websites use Apache today

• Netcraft Web Server Survey (January 2010)

Maturity of Open Source options

• Linux ~ University of Helsinki, 1991• Apache ~ NCSA, 1994• PostgreSQL ~ Cal 1970’s• PostGIS ~ Refractions Research, 2001• MapServer ~2000• Python ~1667

Cost

• Software licensing is available to the City free of charge

• Bulk of funding could go to development work and creating solution

• Free to share with other organizations (GPL3)• What about support?

– So far, not an issue; in fact a plus.– Support options available

MAD Open Source Software

• PostgreSQL/PostGIS: An Open Source, spatially enabled relational database management system

• GeoServer: An Open Source application server capable of delivering spatial data using standard formats (such as Web Mapping Services and Web Feature Services) as specified by the Open Geospatial Consortium.

• OpenLayers: An Open Source JavaScript Library that permits the development of web mapping applications similar to Google Maps

• Django/GeoDjango: A spatially enabled Open Source web application development framework

• ExtJS: A modern javascript library (ajax, etc.)

Why the Cloud?

Why use the Cloud?

• Necessity• Opportunity to evaluate offerings• Virtualized Data Center• Development to QA to Production• Hot Site potential

MAD on the Cloud

• Provider: 3Tera • Cloud product: AppLogic• Granular security access• Granular control over allocated resources• Drag and drop interface for designing and

deploying all elements (VMs) of an application• Grids in Dallas and San Diego

Enterprise Address System (EAS)

• Roll out to departments for integration• Perfect Periodic ETL of key datasets into MAD• Work with other municipalities on

improvements to system (open source) • Migrate to internal cloud • Incorporate building geometry

Thank you

Jeff.Johnson@sfgov.org