One Click Deployment with Jenkins

© 2011 Mayflower GmbH

One Click Deployment with Jenkins

Thorsten Rinne I 12th October 2011, Mainz, Germany

http://webtechcon.de/2011/sessions/#session-18672

http://webtechcon.de/2011/sessions/#session-18672

Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I

Team Lead / Scrum MasterHead of Open Source Labs

@ThorstenRinne2


Who are you?

3


Developers?

4


Administrators?

5


DevOps?

6


Continuous Integration?

7


How much time do you need for a release?

8


More than a year?

9


6 - 12 months?

10


1 - 6 months?

11


2 - 4 weeks?

12


1 - 14 days?

13


1 - 24 hours?

14


Less than 15 minutes?

15


15 Minutes!

16


Why One Click Deployment?

17


Everything is getting faster.

18


Faster feedback from your customer / user

19


Faster ROI (return on invest)

20


Faster responseif the market changes

21

Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I 22Developers are lazy.


Basic requirements

23


Agile software development

24


Scrum or Kanban

25


Test Driven Development

26

Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I 27


Continuous Integration

28


Continuous Inspection

29


Refactoring

30



ScrumKanban


ScrumKanban

ContinuousIntegration


ScrumKanban


ContinuousInspection


Continuous Deployment

31

ScrumKanban


ContinuousInspection


Things you don‘t want.

32


Having your SVN repo in your webserver‘s docroot.

33


Using SSH and vi and your root account on the live

server. Never.

34


Who‘s still doing this? :-)

35



What do we need?

37


First: a central build server

38



Continous Testing with PHPUnit

40


Advantages

41


Test results at one central page

42


Software metrics at one central page

43


Errors can be identified fast and then fixed

44


Awareness

45


Awareness!


Continuous Integration in PHP projects

47


PHPUnitDocBlox

48


Continuous Inspection in PHP projects

49


PHPCSPHP Depend

PHPMDphpcpd

jslint

50


jslint?yes!

51


Infrastructure

52

Building a Cloud-based social network I Mayflower GmbH I 11th October 2011 I 53

http://www.flickr.com/photos/nirak/644337888/




One Ubuntu package to rule them all.

54


Development VMsJenkins CI ServerSelenium Server

Monitoring ServerTesting Server

55

Building a Cloud-based social network I Mayflower GmbH I 11th October 2011 I

Only on production:Frontend Nodes 11.04

Backend Nodes 10.04 LTS

56





☁☁ ☁

Mayflower DMZ

Eucalyptus Cloud

amazon AWS





☁☁ ☁

Mayflower DMZ

Eucalyptus Cloud

amazon AWSDeveloper Laptopwith Ubuntu VM

(Git repo + completesoftware stack)





☁☁ ☁

Mayflower DMZ

Eucalyptus Cloud

amazon AWSGitorious ServerJenkins CI Server

Puppet MasterSelenium Server





☁☁ ☁

Mayflower DMZ

Eucalyptus Cloud

amazon AWS

2 Frontnodes3 Backendnodes

Development MonitorStaging Monitor





☁☁ ☁

Mayflower DMZ

Eucalyptus Cloud

amazon AWS2 Frontnodes2 Backendnodes

DrupalLive Monitor





Configuration Management done with





Configuration changes are on every VM in 5 Minutes.

#WIN





Frontnode 1Varnish nginx

PHP 5.3

Frontnode 2Varnish nginx

PHP 5.3

Backendnode 3RabbitMQ

Apache SolrPHP 5.3

Backendnode 2MySQL Slave

MogileFSMemcached

PHP 5.3

Backendnode 1MySQL Master

MogileFSMemcached

PHP 5.3


Development workflow

65


1. git workflow

66

$ git add Foo.php$ git commit -m ″Bar″$ ant jslint$ ant phpunit ...$ git push


2. Post commit hook

67

Build triggered on Jenkins


Jenkins build checks

68


1. PHPUnit Build:Unittests

Controller Tests (Zend_Test)

69


2. Debian Package Build

70


3. Deploy to staging4. Deploy staging to live

71


At midnight:DocBlox

Static code analysis

72


What is a build process?

73


ant syncant clean

ant db-updateparallelTasks:

ant phpunitant phpcbant deb

74


<target name="phpunit" depends="db-test" description="Run all tests"> <exec executable="phpunit" failonerror="true"> <arg line="--configuration ${basedir}/tests/phpunit-verbose.xml ${basedir}/tests/AllTests.php" /> </exec></target>

75

ant phpunit


parallelTasks:php_depend

phpmdphpunitphpcpdphpcsjslint

76


Build Debian Packageant autoloaderant phpunitant clean

ant compilejsant compilecss


<target name="deb" depends="autoloader,phpunit-coverage,clean,autoloader,compilejs,compilecss" description="Generate Debian package"> <exec executable="gpg"> <arg value="--armor" /> <arg value="--import" /> <arg value="scripts/build/deb/mapolis.mayflower.de.pgp" /> </exec> <buildnumber />

78

ant deb (I)


<exec executable="git" outputproperty="githash"> <arg value="describe" /> <arg value="--always" /> <arg value="--match" /> <arg value="" /> </exec> <touch file="debian/changelog" /> <move file="debian/changelog" tofile="debian/changelog.old" />

79

ant deb (II)


<exec executable="git" output="debian/changelog.new"> <arg value="log" /> <arg value="-n" /> <arg value="1" /> <arg value='--format=format:mapolis-web (${build.number}-${githash}) unstable; urgency=low%n%n * %s%n%n -- %an <%ae> %aD%n%n' /> </exec>

80

ant deb (III)


<concat destfile="debian/changelog"> <fileset file="debian/changelog.new" /> <fileset file="debian/changelog.old" /> </concat> <delete file="debian/changelog.new" /> <delete file="debian/changelog.old" />

81

ant deb (IV)


<exec executable="dpkg-buildpackage"> <arg value="-b" /> <arg value="-v${build.number}-${githash}" /> <arg value="-Zlzma" /> <arg value="-z9" /> <arg value="-k78532445" /> <arg value="-pgpg" /> <arg value="-sgpg" /> <arg value="-mMayflower Mapolis Team <[email protected]>" /> </exec>

82

ant deb (V)

mailto:[email protected]



<move todir="build/deb/"> <fileset dir="${basedir}/../"> <include name="mapolis*.deb" /> <include name="mapolis*.changes" /> </fileset> </move> <exec executable="dpkg-buildpackage"> <arg value="-Tclean" /> </exec> </target>

83

ant deb (VI)


Deploy Stagingant clean

ant deb-signant db-staging-upscp to all hosts

84


Deploy Staging to livescp to all hosts


Disadvantages and problems

86


Build time15 min on harddisk

massive disk i/o4 min on ramdisk :-)

87


Complexity beginning such a infrastructure

88


Jenkins:Single point of failure!

89


Cool stuff which is now possible

90


Deploy by customer. :-)


Jenkins Chuck Norris Plugin


git bisect start $BAD $GOOD

93


Minimum time from bugreport to deploy:

30 minutes

94


$ git branch hotfix-2011-08-22$ git checkout hotfix-2011-08-22$ git reset --hard "last stable sha-1"$ git cherry-pick "hotfix sha-1"$ echo build.number=23 > build.number$ ant deb$ scp build/deb/*.deb xyz.amazonaws.com:/opt/repository/$ git checkout master$ git push origin hotfix-2011-08-10

95

LiveDemo


Questions?

97

Thank you very much for your attention!

© 2011 Mayflower GmbH

Contact Thorsten [email protected]+49 89 242054-31

Mayflower GmbHMannhardtstr. 680538 MünchenGermany

Please rate me @ http://joind.in/talk/view/3904



http://joind.in/talk/view/3884

http://joind.in/talk/view/3884

Technology

One Click Deployment with Jenkins