NIWA Systems Development TeamAndrew Watkins

SDT Overview

Taking dull repetitive work from scientists and giving it to robots since 2008.

People Processes & Tools

The Team

The Team

• Colocated • Agile Methods• Diverse Skills• Mix of Youth and Experience• Capacity and Capability• 80:20 Development & Research

Work on

• Dataware• Middleware• Web services• Web User interfaces• Thick and thin application clients• Mobile, fax, and email notification services• Other Technical Stuff

Don’t Work on

• Static Web content - Communications• Office Systems – IT Operations• Intranet• GIS• Scientific programming• Model Development• High Performance Computing Facility

Key Relationships

• IT General Manager – Arian de Wit• IT Operations – Anthony Cole• Chief Scientist EI – Jochen Schmidt• GIS – Arne Pallentin• Scientists• Public and Science Users• Peer groups

People Processes & Tools

Processes

• Agile development methodology

• Full Software Engineering Life Cycle

• Strategic Planning

Agile Software Development

• Scrum• Iterative development• Test driven development• Continuous Integration• Virtual Environments• Evolving process

Software Engineering Baseline Standard

• Revision control.• Auto build process and development project• Test Coverage• Deployment System• Documentation• Issue Tracking and change control process• Run Time monitoring• Technical support plan and SLA.• Users and Access levels are identified• Data is managed

Data Management Process

• Work in progress• A big sell to the science staff• Overcoming hoarding• Making metadata available• Quality control chain• What can software do?

Managing Multiple Projects

• Over 80 online services• Using Wiki to document – basic metadata• Many legacy systems• Ongoing Maintenance• Sprints – burst of focus to delivery• JIRA Issue Tracking – custom workflow• Prioritisation

People Processes & Tools

Atlassian Confluence – One.NIWA

Atlassian Confluence – One.NIWA

Atlassian Confluence – One.NIWA

Atlassian JIRA

Atlassian JIRA

Eclipse

Subversion

Cruise Control

Selenium Web App Test

Programming Languages and Frameworks

• Java• Frameworks – Struts, Seam• N-Tier systems, Components• Web services, Web UI

• PHP• Frameworks – Symfony, Zend• Simpler systems, Front ends

Relational Database

• PostgreSQL/ PostGIS– Free, Open Source, Full Featured– Strong Geospatial Support

• Oracle– Perceived Quality, Performance– License costs – Distribution issues.

• Others - Legacy– Informix, Empress, Paradox, MySQL– Actively retiring

Non Relational Database

• eXist– XML datastore– Good for reference information– Lookups, Code lists– XQuery– XSLT

NIWA Systems

NIWA Systems Goals

• Web Friendly• Restful• Documented APIS• Open Data by default• Open Source tools first choice• Mostly Free• Standards Compliant

NIWA Systems Goals

• Avoid Vendor Lock in

• Avoid CMS Embedding

• Avoid Brand Embedding

• Don’t reinvent wheel

• Have Fun