Patterns for building large Pyramid applications

Carlos de la Guardia

Pyramid is a very flexible framework

A la carte, use-only-what-you-need features. You can start small and grow your application as needed.

No persistence back end enforced.

Ability to use multiple templating languages.

Many pluggable points and extensibility mechanisms.

We love that and are proud about this.

But for new users, sometimes flexibility means too many options.

Please tell me what to do!

Decisions, decisions, decisions

Pyramid tries to help you decide in some ways:Scaffolds make some decisions for you.

Everything is documented so you can make informed choices.

Some third party packages make yet more decisions.

The scope of your project can also be a factor that influences how you layout your project and what tools you use.

Knowing how others do things can also help decide.

Some suggestions

Project layout.

Tips for choosing a persistence mechanism.

Assembling your toolkit.

What is traversal and do I need it?

View configuration.

Deployment.

Extensible applications.

KARL

Some of these suggestions come from my experience working as a member of KARL's development team.

KARL is an open source web system for collaboration, organizational intranets, and knowledge management.

Used by many international organizations, such as OXFAM GB, OSF, and more.

5000+ users (no anonymous access).

More than 75,000 Pages of content.

Nearly 100,000 lines of code

http://karlproject.org

Project layout

Pyramid does not enforce any layout.

Small or 'one-of' applications can simply use a single file or a few files on the same directory.

For small to medium size applications, all Pyramid scaffolds suggest the same layout.

Scaffold layout

MyProject/|-- CHANGES.txt|-- development.ini|-- MANIFEST.in|-- myproject| |-- __init__.py| |-- static| | |-- favicon.ico| | |-- logo.png| | `-- pylons.css| |-- templates| | `-- mytemplate.pt| |-- tests.py| `-- views.py|-- production.ini|-- README.txt|-- setup.cfg`-- setup.py

Larger project layout

MyLargerProject/|-- CHANGES.txt, etc.|-- mylargerproject| |-- __init__.py| |-- static| | `-- project.css| |-- templates| | `-- mytemplate.pt| |-- models| | |-- somemodel.py| | `-- othermodel.py| |-- someotherpackage| | `-- somestuff.py| |-- views| | |-- someview.py| | `-- otherview.py| |-- tests| | `-- test_something.py`-- setup.py, etc.

Do I have to use python packaging?

Love it or hate it, it's what we have.

You probably want your distributions on PyPi.

Plus, packaging tools are being improved right now by people at this conference.

Assembling your toolkit

Persistence back end:SQLAlchemy.

Other ORMs.

Direct db driver connection.

MongoDB (pyramid_mongodb).

ZODB (Pyramid_zodbconn).

Multiple transactional backends? Use pyramid_tm.

What tools are you familiar with?

The most important question is how does your data look like?

Assembling your toolkit

Templating language:Chameleon.

Mako.

Jinja2 (pyramid_jinja2).

Other.

Forms:pyramid_deform.

pyramid_formalchemy.

pyramid_formish.

pyramid_simpleform.

Assembling your toolkit

Authentication:Write your own with Pyramid's out of the box facilites.

pyramid-openid.

pyramid_basicauth.

pyramid_who.

Velruse.

Development:pyramid_debugtoolbar (a must).

buildout.

URLDispatch and traversal

Pyramid offers two mechanisms for finding views associated with your data.

URLDispatch is the classic, route-based mechanism. URLDispatch is great for table-like data.

Traversal allows you to create a tree of resources, each of which can be addressed by one or more URLs. It's great for tree-like data (think filesystem).

It's also a perfect fit for arbitrarily extendable sites and sites that require very granular security.

Like with relational backends, the key to choosing is how your data will look like.

View configuration

Use declarative configuration.Lets framework configuration statements live next to the code they configure.

Pyramid's view configuration decorators are inert. They don't change the view's input or output.

They have to be activated from the main configuration with scan().

Use renderers. Easy to change implementation.

Facilitates testing.

View configuration

Use predicates:In Pyramid, a single route can correspond to one or more views. Predicates allow the view finding mechanism to find the correct view.

route_name='home'

request_method=POST

xhr=True

Lots of others, plus custom predicates.

Easy HTTP caching:@view_config(http_cache=3600) # 60 minutes

Pyramid will add appropriate Cache-Control and Expires headers to responses generated when this view is invoked.

View configuration

Many applications have similar view data that needs to be passed to the renderers every time:You can generate that information on every view.

You can have a method or class that generates a dictionary with the common values.

You can make your views methods of a single class with a common initialization.

If you don't have alergic reactions to any mention of a particular forbidden 'Z' word, there are other ways.

View configuration

Class method views example:

from pyramid.response import Response
from pyramid.view import view_config

class Aview(object):
def __init__(self, request):
self.request = request

@view_config(route_name='view_one')
def view_one(self):
return Response('one')

@view_config(route_name='view_two')
def view_two(self):
return Response('two')

Extensibility

Requirements depend on kind of project.

Pyramid provides some easy to use mechanisms that allow you to compose or extend applications.Strong, extensible configuration mechanism with conflict detection and the ability to add directives.

Structured includes.

Asset specifications.

Tweens (Pyramid specific middleware).

ZCA-based overriding mechanisms.

Extensibility

Asset specification example:my.package:static/baz.css

identifies the file named baz.css in the static subdirectory of the my.package Python package.

Using asset overriding:

config.override_asset(
to_override='some.package:templates/mytemplate.pt',
override_with='another.package:othertemplates/anothertemplate.pt')

config.override_asset(to_override='some.package',
override_with='another.package')

Test, test, test

What do you call 100% coverage? A good start.

Unit tests. Make sure that nothing breaks.

Functional tests. Make sure it does what it's supposed to do.

Tools:Nose

Coverage

WebTest

Selenium

Use whatever you want, just test.

Document

If possible, use documentation to guide application design.

Write or sketch how things should work before implementing them.

Share with the team to get input, understanding.

Tools:Class or method doc strings.

Sphinx.

Balsamiq.

Deployment with buildout

Buildout is a Python system for assembling applications from multiple parts in a repeatable manner.

Recipes are used to define what each part of the buildout will install and/or setup. There are many available recipes on PyPI.

A buildout can have different configurations. For example, deployment and production.

Buildout can be used to setup a Pyramid application and it's dependencies easily.

Deployment with buildout

Sample buildout:

[buildout]
parts =
myapp
mkrelease

Develop = src/mypackage

index = http://example.github.com/myapp/staging/index/

[myapp]
recipe = zc.recipe.egg
eggs =
pyramid_debugtoolbar
nose
mypackage
interpreter = py

Deployment with supervisor

Supervisor is a client/server system that allows its users to monitor and control a number of processes on UNIX-like operating systems.

We can use supervisor to control our pyramid applications, either alone or together with other applications or services.

A very easy way to configure it is to simply add the supervisor egg to our buildout and include a configuration file

Deployment with supervisor

[inet_http_server]port=127.0.0.1:9001

[supervisord]logfile=%(here)s/var/supervisord.loglogfile_maxbytes=50MBlogfile_backups=10loglevel=infopidfile=%(here)s/var/supervisord.pid

[rpcinterface:supervisor]supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]serverurl=http://127.0.0.1:9001

[program:core]command = %(here)s/bin/paster serve %(here)s/src/inav2_core/development.iniredirect_stderr = true

Deployment with nginx

More information at: http://docs.pylonsproject.org/projects/pyramid_cookbook/dev/deployment/nginx.html

upstream myapp-site { server 127.0.0.1:5000; server 127.0.0.1:5001;}server { server_name example.com; access_log /home/example/env/access.log; location / { proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_connect_timeout 60s; proxy_send_timeout 90s; proxy_read_timeout 90s; proxy_buffering off; proxy_pass http://myapp-site; proxy_redirect off; }}

Deployment with mod_wsgi

We need to create an app that will call our application with configuration. Then we set up apache to call this app.

More information at:http://docs.pylonsproject.org/projects/pyramid/1.2/tutorials/modwsgi/index.html

#pyramid.wsgifrom pyramid.paster import get_appapplication = get_app('/mydir/modwsgi/env/myapp/production.ini', 'main')

#apache.confWSGIApplicationGroup %{GLOBAL}WSGIPassAuthorization OnWSGIDaemonProcess pyramid user=cguardia group=staff threads=4 \ python-path=/mydir/modwsgi/env/lib/python2.6/site-packagesWSGIScriptAlias /myapp /mydir/modwsgi/env/pyramid.wsgi

WSGIProcessGroup pyramid Order allow,deny Allow from all

Thanks!

@cguardia on Twitter and IRC.

Join us at the #pyramid IRC channel for encouragement and support.

There's also the Pyramid mailing lists at Google Groups:

http://groups.google.com/group/pylons-devel

Muokkaa otsikon tekstimuotoa napsauttamalla

Muokkaa jsennyksen tekstimuotoa napsauttamallaToinen jsennystasoKolmas jsennystasoNeljs jsennystasoViides jsennystasoKuudes jsennystasoSeitsems jsennystasoKahdeksas jsennystasoYhdekss jsennystaso