How to host an app for $20 in 20min using buildout and hostout

[email protected] June 2009 Dylan Jay

Buildout and Hostoutor

“How to host an app for $20 in 20min”

Dylan [email protected]

Technical Solutions Manager

PretaWeb

(Thanks to Darryl Cousins for slides 1-10 from NZPUG presentation)


A Python Module

A block of code imported by some other code


A python module: hello.py

We can import the method from the module:

def helloworld(): print u"Hello World

from hello import helloworld


A Python Package: hello

A package is a module that contains other modules: hello/

__init__.py hello.py

Now we must import the method from the module within the module.

from hello.hello import helloworld


Disutils

Distutils was written so we have a unified way to install python modules.

python setup.py install


Disutils – creating a distribution

To distribute the hello module we need have it in a directory with a setup.py file. workingdir/

setup.py hello/


The setup.py file needs at the least the following.

from distutils.core import setup setup(name="hello",

)


Disutils creating a distribution continued

Now we can create a distribution tarball with disutils. python setup.py sdist

Our directory now looks like this workingdir/

setup.py hello/


dist/ hello-1.0.tar.gz

If we unpack the source distribution it looks like this:

Hello-1.0/ PKG-INFO setup.py hello/



setuptools

• Setuptools is built on top of distutils • uses the setup.py • uses eggs for distribution • allows us to save our modules as eggs to pypi

Installing setuptools Installing setuptools wget http://peak.telecommunity.com/dist/ez_setup.py wget http://peak.telecommunity.com/dist/ez_setup.py python ez_setup.pypython ez_setup.py


Eggs

To create an egg change the import line in setup.py

from setuptools import setup setup(name="hello",

version="1.0", )

We can call that with: python setup.py bdist_egg

Which creates a binary egg in our dist directory dist/

hello-1.0-py2.4.egg


pypi

If we want that egg available on pypi and we have an account we can do that with a single command. python setup.py sdist upload

Which all the world can use easy_install hello


virtualenv

If we want to install without affecting whole system$ easy_install virtualenv$ virtualenv myenv$ source myenv/bin/activate(myenv)$ easy_install hello. . . (myenv)$ deactivate


zc.buildout – what is it?“Buildout is a system of configuring repeatable steps for

assembling complicated systems (applications) from multiple parts.”

Configure different parts using one config fileGlue the parts togeatherSpecify versionsIsolated in one directoryRepeat that configuration over and over


Installing buildout

$ easy_install zc.buildout$ cd myproj$ buildout init

This creates the following directory structure

myproj/bin/

buildoutparts/develop-eggs/buildout.cfg


buildout.cfg

Buildout does very little by itself[buildout] parts =

A buildout is made of parts. Parts are instances of recipes. Recipes do the work.

[buildout] parts = py

[py] recipe = zc.recipe.egg interpreter = py eggs = hello


Recipes – 185+ on pypi


Running buildout

$ bin/buildoutInstalling py.Generated interpreter 'myproj/bin/py'.

$ bin/py>>> from hello import helloworld

>>> helloworld() Hello World


Buildout - Scripts

You can generate scripts

[buildout]parts = buildout1 buildout2

[buildout1] recipe = zc.recipe.egg eggs = zc.buildout==1.2.1scripts = buildout=buildout1

[buildout2]recipe = zc.recipe.eggeggs = zc.buildout<=1.2.0scripts = buildout=buildout2


Buildout - Versions

You can pin versions

[buildout]parts = buildout1 buildout2versions = versions

[buildout1] recipe = zc.recipe.egg eggs = zc.buildoutscripts = buildout=buildout1

[versions]zc.buildout = 1.2.1


Buildout - Scripts are wrappers$ cat bin/buildout1

#!/bin/python

import syssys.path[0:0] = [ '/download-cache/eggs/zc.buildout-1.2.1-py2.4.egg', '/download-cache/eggs/setuptools-0.6c9-py2.4.egg', ]

import zc.buildout.buildout

if __name__ == '__main__': zc.buildout.buildout.main()


Buildout – manages installation$ bin/buildout -vUninstalling py.Installing buildout1.Installing 'zc.buildout==1.2.1'.We have the distribution that satisfies 'zc.buildout==1.2.1'.Adding required 'setuptools' required by zc.buildout 1.2.1.Picked: setuptools = 0.6c9Generated script 'myproj/bin/buildout1'.Installing buildout2.Installing 'zc.buildout==1.2.0'.We have the distribution that satisfies 'zc.buildout==1.2.0'.Adding required 'setuptools' required by zc.buildout 1.2.0.Picked: setuptools = 0.6c9Generated script 'myproj/bin/buildout2'.


Buildout – variables and dependencies and distutils

[buildout]parts = py

[py] recipe = zc.recipe.egg Interpreter = pyextra-paths=

${reportlab:output}

[reportlab]recipe = collective.recipe.distutilsurl = http://www.reportlab.org/ftp/ReportLab_2_3.tar.gzoutput = ${buildout:directory}/parts/site-packages/


Helloworld in Plonesrc/hello/hello/hello.py

from Products.Five.browser import BrowserView

class HelloWorld(BrowserView):

def __call__(self): return "Hello World"

src/hello/hello/configure.zcml

<configurexmlns:browser="http://namespaces.zope.org/browser"> <browser:page name="hello" class=".hello.HelloWorld" permission="zope2.Public" /></configure>


Helloworld in Plone buildoutbase.cfg

[buildout]parts = instancedevelop = src/hello

[zope2]recipe = plone.recipe.zope2installurl = http://www.zope.org/Products/Zope/2.10.8/Zope-2.10.8-final.tgz

[instance]recipe = plone.recipe.zope2instancezope2-location = ${zope2:location}user = dylan:jayeggs = hello Plonezcml = hello


Run Helloworld

$ bin/buildout -c base.cfgDevelop: 'myproj/src/hello'Updating zope2.Updating fake eggsInstalling instance.

$ bin/instance fg2009-06-08 20:24:54 INFO ZServer HTTP server started at Mon Jun 8 20:24:54 2009

Hostname: localhostPort: 8080

...2009-06-08 20:25:44 INFO Zope Ready to handle requests


Production buildout extends baseprod.cfg

[buildout]extends = base.cfgparts += supervisor

[instance]zeo-client = Truezeo-address = ${zeo:zeo-address}

[zeo]recipe = plone.recipe.zope2zeoserverzope2-location = ${zope2:location}Zeo-address = 127.0.0.1:9000

[varnish]recipe = plone.recipe.varnish:instancebind = 127.0.0.1:80mode = foregroundbackends = ${instance:http-address}daemon = ${varnish-build:location}/sbin/varnishd

[varnish-build]recipe = zc.recipe.cmmiurl = http://waix.dl.sourceforge.net/sourceforge/varnish/varnish-2.0.3.tar.gz

[supervisor]recipe = collective.recipe.supervisorprograms = 10 zeo ${zeo:location}/bin/runzeo 20 instance1 ${instance:location}/bin/runzope 70 cache ${buildout:bin-directory}/varnish ${varnish:location} true


Hosting a buildout

First get a server with root access

~$20USD p/m for 256mb

Slicehost, Amazon ec2, etc


collective.hostout

Aims to to make a hosted python app :- as easy to install as a cPanel app as easy to deploy as google app engine keep control of code


Put collective.hostout in buildout.cfg

[buildout]extends = base.cfgparts += hostout

[hostout]recipe = collective.hostouthost=myproj.slicehost.combuildout = prod.cfgstart_cmd = ${buildout:bin-directory}/supervisordstop_cmd = ${buildout:bin-directory}/supervisorctl shutdown


Run hostout$ bin/buildout Develop: 'myproj/src/hello'Updating zope2.Updating fake eggsUpdating instance.Installing hostout.

$ bin/hostout running setup script 'src/hello/setup.py'.…creating deployment file: versions.cfg prod.cfg base.cfg hello1.0dev_12...6-py2.4.egglogging into the following hosts as root: myproj.slicehost.comPassword:Installing pythonBootstrapping buildout in /var/local/buildoutUnpacking deployment fileRunning buildoutRunning start command


Hostout – with source control

Doesn't need access to svn or git on the server


Hostout TODO

Multi serverTransfer databasesIntegrate with source controlRollback

http://plone.org/products/collective.hostout

Technology

How to host an app for $20 in 20min using buildout and hostout