Codecoon A Technical Case Study

Michael Lihs

@kaktusmimi lihs@codecoon.com

Fabian Stein

@docmak stein@codecoon.com

Outline

→ Continuous Integration Workflow → The Codecoon Vision → Codecoon Components → Usage of TYPO3 Flow → Integration of 3rd-party Applications → Outlook & Résumé

Developer

Developer

StagingServer

ProductionServer

Customer

Developer

Git Jenkins

Monitoring

Commit Stage

Acceptance Stage

Monitoring Stage

How does your

Project Lifecycle

look like?

1.Setup Dev

Environment

2.Setup

Versioning

3.Setup CIStages

4.Setup

StagingServer

5.Setup

ProductionServer

6.Setup

Deployment

7.WriteCode

Fix / RepairDev

Environment

Fix CI Server

UpdateProduction

Server

9.Deploy to

Production

8.Deploy to

Staging

UpdateGit

Server

Fix Deployment

Developer’s „Daily Activity Distribution“ *)

45% Development

30% Infrastructure Setup

25% Infrastructure Maintenance

*) numbers might differ slightly

„Infrastructure as a Service“

~99% Development

The Codecoon Vision

Provide CI as a service

→ focus on your project again

→ avoid all the orange activities

→ avoid as much light-blue activities as possible

Codecoon Management Portal

Continuous IntegrationCodecoon Developer Box

Staging ProductionERP

Developer

Staging Production

Continuous Integration

Codecoon Management Portal

Codecoon Developer Box

ERP

Developer

NOCCommand Line

Test-Harness

NOCWeb Service

NOC Services

ERP Services

DeploymentServices

NOC Domain Models

TYPO3 Flow

NOCUI

Testing

Unit Tests

Functional Tests

Acceptance Tests

export PPH_OUTPUT=JSON

PHYSICALSERVER_ID=`flow physicalServer:create '{"serialNumber" : "0815-2", "nodeName": "shared1.live.codecoon.com", "externalNodeName": "shared1.codecoon.com", "isAvailable" : true, "interfaces" : { "Management Interface" : { "macAddress" : "bla:bla:bla", "ipAddresses" : ["a.b.c.d","a.b.c.d"] } } }' | jq -r .uuid`

http://stedolan.github.io/jq/

{"uuid" : "89259562398239", ... }

Webspace Version 4

Webspace Version 4

Webspace Version 1

Abstract Webspace

Webspace Version 3

Webspace Version 2

/* * * @Annotation * @Target("CLASS") */final class Artifact {}

Wir wollen zeigen, * dass es mit wenig Code

möglich ist, die Persistierung in Flow zu beeinflussen

* wie wir dies genutzt haben und ein Domain-Objekt in unterschiedlichen Implementierungsversionen in der Datenbank zu persistieren

/** * @Flow\Entity * @PPH\Artifact * @ORM\Table(name="pphv2_a_webspace") */abstract class Webspace extends AbstractArtifact {

//...

}Wir wollen zeigen, * wie die Annotation für ein

„abstraktes Produkt“ aussieht

* dass wir hier für jedes Produkt eine Tabelle angeben können, in der das jeweilige Objekt persistiert wird

Codecoon Management Portal

Codecoon Developer Box

Developer

ERP

Continuous Integration

Chef Server

Staging

Production

GIT

Motivation for a mobile Development Environment

• Work wherever you are • Development = Production • Don’t touch your system • Distribute environments & cooperate with othershttp://www.die-mobile-gesellschaft.de/presse/fotos/themen/03_Mobil_arbeiten/10_Mobil_arbeiten_2.jpg

Debugging Tools

FAMP Stack

Frontend Tools

Developer

Freelancer

StagingServer

ProductionServer

Customer

Customer

Git Jenkins

Monitoring

Commit Stage

Acceptance Stage

Monitoring Stage

TrainingParticipant

CodecoonBox

TrainingParticipant

CodecoonBox

TrainingParticipant

CodecoonBox

IDE

ApacheCC ManagmentPortal

Git

Staging

Production

MySQL

PHP

SOLRFileSystem SSH Key

BOX-API

SURFSMB /NFS

https://vbox33.box.codecoon.com https://my.codecoon.com

APIAJAX/CORS

Codecoon Management Portal

gem 'sinatra'gem 'sinatra-contrib'gem 'sinatra-cross-origin'gem 'sintra-base', '~>1.4.0'gem 'webrick', '~>1.3.1'

Das ist das Äquivalent zur composer.json in Ruby.

Statt json/yaml wird bei Ruby oft auch für solche Dateien Ruby Code benutzt.

Vorteil: Es sind sehr einfach eigene Methoden möglich, die sehr mächtige Manipulation erlauben.

get '/status' do { :response => 'status', :box_initialized => is_initialized(), :project_type => settings.project_type, :environment => settings.environment }.to_json()end

webrick_options = { :Port => 4567, :AccessLog => [], :Logger => WEBrick::Log::new($stderr, WEBrick::Log::DEBUG), :DocumentRoot => DOCUMENT_ROOT, :SSLEnable => true, :SSLVerifyClient => OpenSSL::SSL::VERIFY_NONE, :SSLCertificate => OpenSSL::X509::Certificate.new( File.open(File.join(CERT_PATH, "vbox33.box.codecoon.com.crt")).read), :SSLPrivateKey => OpenSSL::PKey::RSA.new( File.open(File.join(CERT_PATH, "vbox33.box.codecoon.com.key")).read), :SSLCertName => [ [ "CN",WEBrick::Utils::getservername ] ], :app => BoxManagementApi} Rack::Server.start webrick_options

Gesamte Webserver - Konfiguration mit Sinatra.

Das ist alles, was man braucht, um

* einen eigenen Webservice* mit eigenem Webserver* mit SSL Verschlüsselung* mit eigenem Logging* auf einem eigenen HTTP

Port

laufen zu lassen.

$ rackup

. ├── Gemfile ├── application.rb ├── certificates │   ├── ca-bundle.crt │   ├── vbox33.box.codecoon.com.crt │   └── vbox33.box.codecoon.com.key ├── config │   ├── boot.rb │   ├── config_development.yaml │   └── config_production.yaml ├── config.ru ├── controllers │   ├── management_controller.rb │   └── status_controller.rb ├── lib │   └── helpers.rb ├── log └── readme.md

Vagrant BoxChef Cookbooks

DataBags

CodecoonInstaller

Scripts

Mount into Chef solo

4. Shared Host • Acts as Chef Node • Deployed with Surf using rsync over ssh • Holds git repository for each project

Codecoon Management Portal

DataBags

Chef CookbooksVagrant Box

Chef solo

creates

uploadspull

mount info

upload

Surf package build locally

Cache on Production Stage

Release Directory

more task on prod.f.e. flush cahces

tranfere withrsync cp release

5. ERP • Our ERP has crappy „Web Service“ • No user management / queue in the Web Service • Used MessageQueue (RabbitMQ)

• Made ERP Multi-User ready

Was wollen wir mit dieser Folie zeigen?

* Anbindung des ERP Systems an die Hosting-Oberfläche

* Verwaltung von Kunden, Rechnungs- und Vertragsdaten

* Anbindung über einen „Webservice“* ** eine Art SOAP

Codecoon Management Portal

ERP

Outlook • Implement the missing parts of our vision

• Jenkins and Gitlab as integrated services • Production Stage

• User driven development • 10 Early Adopters

Flow - the good things * Fluid is a great Template Engine * Functional testing with Flow is pure fun * Business (Domain) Logic can be implemented with very little

Framework overhead * Command Controllers are great

FLOW

Flow - the not so nice things * Flow Packages are not modular

* „Everything or nothing“ * Code-Rewrite makes feedback cycles during development slow * Remote Debugging in proxy classes can be annoying * All the „magic things in the background“ and declarative stuff

makes understanding hard * Code-Completion and IDE tools not very helpful

* Webservices (REST) are not easy to implement * There is no ready-to-use Controller à la Sinatra

* Not much „Best-Practices“ on Google…

FLOW

Resume • Flow is a powerful Framework • Integrates well with 3rd party components • Don’t use it „unreflected“ • Best detail solution vs. smoothes integration

• Duplicated configuration • Duplicated functionality

We will go out

and try it with you!

Thank you! We hope you had fun!

If you want to stay in contact:

Michael Lihs: @kaktusmimi Fabian Stein: @docmac

@codceoon

Or outside at our booth