Django Infrastructure @ UGentKevin Van Wilder

Why We Set Up the Django Infrastructure

As an Alternative to Plomino● Rapid application development● yet Reliable, Maintainable, Scaleable● Better at Handling Large Loads● Seperate Applications from Portal Site

Source Code Infrastructure

Git Repositories

Each project has 2 repositories

● site-<projectname>.git○ Django Source Code○ Business Logic of the application

● pkg-<projectname>.git○ Debian Package Source Code○ Installation Logic of the application

Each repository has a specific code structure.

○ Why? Automation!

Git Flow

http://nvie.com/posts/a-successful-git-branching-model/

Server Infrastructure

Load Balancerdjprd

Web + Appdjprd1

Web + Appdjprd2

...

DB

Overview Production Infrastructure

Managed by Developers

Managed by Jenkins Pipeline & Puppet

Load Balancerdjprd

Web + Appdjprd1

Web + Appdjprd2

DBframboo

s

Load Balancerdjqas

Web + Appdjqas1

Web + Appdjqas2

Web + Appdjtst1

Web + Appdjdev1

Complete Infrastructure

“Web + App” Servers

PuppetizedContinuous Deployment via JenkinsEach server contains all the appsLoad Balanced

Database

framboos.ugent.beMaar migratiemogelijkheden naar:● PostgreSQL● MySQL● SQLite● Oracle

Support Tools

Version Overview

Supporting Tools

Supporting Tools

Other Tools

AnsibleMaintenance Task Orchestration● db syncs● process restarts● etc

Continuous Deployment Pipeline

Jenkins

Project Specific Buildsteps

Project Independent Buildsteps

Build Unit Test Acceptance Test Package Deploy TST

Deploy QAS

Deploy PRD

Onderwijstips

Kwalifacts

Telefoonboek

UGent Mobile

Abstract Jenkins Pipeline

Abstract Jenkins Pipeline

Build

Unit Test

Acceptance Test

Package

Deploy TST

Deploy QAS

Deploy PRD

Start Project Build The only part of the pipeline that is specific to a project

Starts execution of the pipeline

Calls the “Build” job with the project-specific parameters:

PROJECT_NAME

PACKAGE_NAME

To add a new project pipeline, simply...

Duplicate this job

Change the values of the parameters

Abstract Jenkins Pipeline

Build

Unit Test

Acceptance Test

Package

Deploy TST

Deploy QAS

Deploy PRD

Start Project Build

Build

Clones the Git code for the web application

Unit Test

Finds all the written unit tests and executes them

Acceptance Test

Currently not yet implemented.

Finds all the written acceptance tests and executes them

Package

Clones the Git code for the debian package

Copies over the tested source code

Builds a Debian Package

Abstract Jenkins Pipeline

Package

Deploy TST

Deploy QAS

Deploy PRD

Start Project Build

Deploy TST

Built packages are automatically deployed to Testing

Deploy QAS

Manual trigger for deployment to Quality Assurance

Can only be triggered if successful deploy to TST

Deploy PRD

Manual trigger for deployment to Production

Can only be triggered if successful deploy to QAS

Build

Unit Test

Acceptance Test

Future

Future

Asynchronous Messaging (RabbitMQ)Caching

Response Caching (Varnish)Low-Level Caching (Memcached)

Acceptance Testing (Selenium)Seperate database for each environmentMore Monitoring (Graphite)

Kevin Van Wilder@KevinVanWilderOpen Source Consultant

INUITSDuboisstraat 50 - 2060 Antwerphttp://www.inuits.eu

QUESTIONS ?