Extending and customising Plone 3Martin Aspeli

A small aside

Agenda

Love the buildout

Embrace the egg

Policy: Repeatable repeatable repeatable!

Here a dependency, there a dependency

You love GenericSetup

Thou shalt write tests!

Visual customisation: A cacophony of layers and skins

Next steps …

Introduction

Zope 2 is great

But we want more!

Smaller, isolated packages are better

But we need better tools to manage them

Development and deployment are getting more complex

Plan to keep control from the beginning

Through-the-web development is not cool anymore

Some terminology

Product – A Python package for Zope 2. Often lives in Products/ and is subject to certain magic when Zope starts up.

Package – A more general term, but in the Zope world is typically refers to something that is not in the Products.* name space.

Egg – A way to distribute packages. Supports automatic dependency management: dependencies can be downloaded from PyPI/The Cheese Shop.

Buildout – A way to manage projects, with tools for building Zope instances, downloading Plone, installing eggs and so on (these are known as “recipes”).

GenericSetup – A way to configure a Plone site using declarative XML files.

Policy product – A pattern whereby a project has a single product which, upon installation, installs all dependencies and customisations in one step.

A buildout

First, get easy_install, ZopeSkel and paster

These can go in the global Python (most of the time)

Watch out for where they get installed$ wget http://peak.telecommunity.com/dist/ez_setup.py$ easy_install -U ZopeSkel$ paster create --list-templates

A buildout

We will use the plone3_buildout template from ZopeSkel

$ paster create -t plone3_buildout pc07

Answer the questions

the only one you have to answer is password

Setting debug mode to on is usually a good idea

A buildout

Let’s look at what we got:

buildout.cfg is the main configuration file

products/ can be used for old-style Products

src/ is used for custom eggs

var/ is used for Data.fs and logs

A buildout

Buildout will generate other directories:

bin/ contains executables

parts/ contains files managed by buildout

eggs/ contains downloaded eggs

develop-eggs/ contains egg-links to development eggs

A buildout

Let’s build it$ cd pc07$ python bootstrap.py$ ./bin/buildout

If you are on Windows

I’m sorry to hear that

But you can still get it to work - just read the generated README.txt file and set up mingw32.

If Python 2.4 is not your default Python

Read README.txt

A buildout

Believe it or not, you now have a fully working Zope and Plone installation $ ./bin/instance fg

More importantly, we can manage other dependencies and packages through buildout.cfg

We can also set up ZEO, Varnish, Deliverance, Apache and so on using standard or custom recipes

To learn more about buildout, see http://plone.org/documentation/tutorial/buildout

Subversion

Idiot developers do not use source control

We are not idiot developers

You don’t need a remote server to run Subversion

$ svnadmin create /repo$ svn mkdir file:///repo/pc07/{,/trunk}$ svn co file:///repo/pc07/trunk pc07 # our project

Of course, if you have a server, use that instead!

In this case, you can use svn+ssh or Apache and mod_dav

Subversion

Check in only source code (not generated files or data)$ svn ps svn:ignore '> eggs> develop-eggs> bin> var> parts> .installed.cfg' .$ svn add products bootstrap.py buildout.cfg src README.txt$ svn commit

Other developers can now use the same buildout

You can move it to a staging or live server

A policy product

This is a package that performs all the customisations and installs all the dependencies to turn vanilla Plone site into your application or site

Create a new egg

$ cd src/$ paster create -t plone example.policy

Namespace is example, package is policy, Zope 2 product is True, Zip-safe is False

A policy product

This package is an egg!

setup.py declares dependencies, metadata

example/policy/ contains actual code

Let’s add to svn, but ignore the generated egg-info

$ svn add -N example.policy$ cd example.policy$ svn ps svn:ignore '*.egg-info' .$ svn add setup.py docs example README.txt setup.cfg$ svn commit

A policy product

We need to tell buildout about the package.

Declare it as a development egg (so buildout doesn’t search the Cheese Shop for it!)

Add it to the egg “working set” for the Zope 2 instance

Install a ZCML slug so that Zope finds it at startup

A policy product

Edit buildout.cfg:

[buildout]...eggs = elementtree example.policydevelop = src/example.policy

[instance]...zcml = example.policy

A policy product

Re-run ./bin/buildout

$ ./bin/buildout -oDevelop: '/Users/optilude/tmp/eggs/pc07/src/example.policy'...

We can run buildout in offline mode (-o) to make it a bit faster

The egg is now ready. We can run the generated tests just to check.

$ ./bin/instance test -s example.policy

Installing dependencies

Most sites use third party products

As a developer, you may want to use a particular Python library (not necessarily a Zope product)

We would like to make the process of obtaining and installing these as automated as possible

Installing dependencies

If it is an old-school product without a release, add it to the top-level products/ directory (note lowercase “p”)

Do not add anything inside parts/ as buildout may delete your changes

To track a package in Subversion, use svn:externals:

$ cd products$ svn ps svn:externals 'RichDocument https://svn.plone.org/svn/collective/RichDocument/trunk' .$ svn up$ svn commit

Installing dependenciesIf it is an old-school product with a release, you can have buildout download and configure it for you.

Edit buildout.cfg:[productdistros]...urls = http://plone.org/products/richdocument/releases/3.0.1/RichDocument-3.0.1.tar.gznested-packages =version-suffix-packages =

Then re-run ./bin/buildout

See http://plone.org/documentation/tutorial/buildout for more about the other options

Installing dependenciesIf your dependency is an egg in the Cheese Shop, add it to setup.py in the policy product and re-run buildout: install_requires=[ 'setuptools', 'plone.browserlayer>=1.0rc1,<1.1dev',],

Re-run ./bin/buildout to get the new package.

Be careful: Some of the newer Zope 3 eggs declare parts of Zope 3.4 as transitive dependencies. This may break Zope 2.10. For now, you must find a version without copious dependencies.

To learn more about how setuptools handles versions: http://peak.telecommunity.com/DevCenter/setuptools

Installing dependencies

If there is no release, put the egg in the src/ directory

Add it to the develop option in buildout.cfg[buildout]...develop = src/example.policy src/some.package

You can still use the policy product’s setup.py to declare it as a dependency install_requires=[ 'setuptools', 'some.package', ],

Installing dependencies

For a package outside Products.*, we must include it in ZCML processing explicitly

Edit src/example.policy/example/policy/configure.zcml:

<configure xmlns="http://namespaces.zope.org/zope" xmlns:five="http://namespaces.zope.org/five" i18n_domain="example.policy"> <include package="plone.browserlayer" />

...</configure>

Installing dependenciesWe should ensure that installable Zope 2 products are installed when the policy product itself is installed

Create src/example.policy/example/policy/Extensions/Install.py and add this boilerplate:import transactionfrom Products.CMFCore.utils import getToolByNamePRODUCT_DEPENDENCIES = ('RichDocument', 'plone.browerlayer') # Edit this list to add new dependencies EXTENSION_PROFILES = () # ('example.policy:default',) # Our profile - we’ll activate this in a momentdef install(self, reinstall=False): portal_quickinstaller = getToolByName(self, 'portal_quickinstaller') portal_setup = getToolByName(self, 'portal_setup') for product in PRODUCT_DEPENDENCIES: if reinstall and portal_quickinstaller.isProductInstalled(product): portal_quickinstaller.reinstallProducts([product]) transaction.savepoint() elif not portal_quickinstaller.isProductInstalled(product): portal_quickinstaller.installProduct(product) transaction.savepoint() for extension_id in EXTENSION_PROFILES: portal_setup.runAllImportStepsFromProfile('profile-%s' % extension_id, purge_old=False) product_name = extension_id.split(':')[0] portal_quickinstaller.notifyInstalled(product_name) transaction.savepoint()

Customisation with GS

GenericSetup lets us define customisations via XML Two types of profiles:

Base profile: A complete configuration. Plone is installed using one of these.

Extension profile: Bolts onto a base profile to amend or change configuration. This is the most useful kind for third party developers.

Managed via portal_setup, but portal_quickinstaller knows how to install them too

Customisation with GSFirst we must create and register a profile$ mkdir -p example.policy/example/policy/profiles/default

Edit configure.zcml and add:<configure ... xmlns:genericsetup="http://namespaces.zope.org/genericsetup"> <genericsetup:registerProfile name="default" title="Example policy product" directory="profiles/default" description="Install our example customisations" provides="Products.GenericSetup.interfaces.EXTENSION" />...

Customisation with GS

If we didn’t have Extensions/Install.py, the quick-installer would have found this profile and made it available for installation

In the future, we will be able to declare profile dependencies

Until then, we use Install.py to install dependencies and then trigger our own profile

Now we can un-comment this line in Install.py:EXTENSION_PROFILES = ('example.policy:default',)

Customisation with GSLook at parts/plone/CMFPlone/profiles/default

This is Plone’s base profile. Copy any of these files into your product’s extension profile and modify as appropriate.

For example, create profiles/default/propertiestool.xml:<object name="portal_properties"> <object name="navtree_properties"> <property name="metaTypesNotToList" type="lines" purge="false"> <element value="News Item"/> </property> </object> <object name="site_properties"> <property name="allowAnonymousViewAbout" type="boolean">False</property> </object></object>

When this is installed, portal_properties will be changed

Customisation with GSSome useful handlers:

properties.xml – sets properties at the root of the site

propertiestools.xml – manages portal_properties

rolemap.xml – Creates roles and sets permissions at the root of the site

actions.xml – Sets up actions in portal_actions

catalog.xml – Creates catalog indexes/metadata

workflows.xml – Maps and creates workflow definitions (kept in the workflows/ folder)

Customisation with GS

It is not terribly hard to write your own GenericSetup import/export handler

However, sometimes we just need to write some Python code

For this, we can use the “import-various” hack

This registers a new handler, but instead of parsing an XML file we simply execute some custom code

Customisation with GS

Create profiles/default/example.policy_various.txt. This is a “flag” - it can be empty

Create profiles/default/import_steps.xml:<import-steps> <import-step id="example.policy_various" version="20071012-01" handler="example.policy.setuphandlers.importVarious" title="Various Example Settings"> <dependency step="catalog"/> <dependency step="propertiestool"/> </import-step></import-steps>

Customisation with GS

Create setuphandlers.py:def importVarious(context): if context.readDataFile('example.policy_various.txt') \ is None: return

site = context.getSite()

# Now do something useful

We need to check for the flag - otherwise, the handler could run for other profiles as well

Try to avoid using an import-various step if you can

Installation tests

Idiot developers don’t write tests

Remember - we’re not idiot developers

The most basic tests just check that our customisations are in effect

There was a tests.py generated. That’s nice, but it’s a bit simple for our needs. Get rid of it.

Installation testsCreate src/example.policy/example/policy/tests/, and within it, __init__.py and then base.py:from Products.Five import zcmlfrom Products.Five import fiveconfigurefrom Testing import ZopeTestCase as ztcfrom Products.PloneTestCase import PloneTestCase as ptcfrom Products.PloneTestCase.layer import onsetup

ztc.installProduct('SimpleAttachment')ztc.installProduct('RichDocument')

@onsetupdef setup_package(): import example.policy zcml.load_config('configure.zcml', example.policy) ztc.installPackage('plone.browserlayer') ztc.installPackage('example.policy')

setup_package()ptc.setupPloneSite(products=['example.policy'])

class ExamplePolicyTestCase(ptc.PloneTestCase): """Common test base class """

This sets up a Plone site for testing with our example

Installation testsNow create tests/test_setup.py:import unittestfrom example.policy.tests.base import ExamplePolicyTestCasefrom Products.CMFCore.utils import getToolByName

class TestSetup(ExamplePolicyTestCase): def afterSetUp(self): self.properties = getToolByName(self.portal, 'portal_properties') self.types = getToolByName(self.portal, 'portal_types') def test_metaTypesNotToList_set(self): navtree_props = self.properties.navtree_properties mtntl = navtree_props.getProperty('metaTypesNotToList') self.failUnless('News Item' in mtntl) def test_allowAnonymousViewAbout_set(self): site_props = self.properties.site_properties allow = site_props.getProperty('allowAnonymousViewAbout') self.assertEquals(False, allow) def test_rich_document_installed(self): self.failUnless('RichDocument' in self.types.objectIds())

def test_suite(): suite = unittest.TestSuite() suite.addTest(unittest.makeSuite(TestSetup)) return suite

This tests that we got the basic setup right

Installation tests

To run the tests, go to the root of the buildout and do:

$ ./bin/instance test -s example.policy

You should also try it through-the-web!

Visual customisationVisual components include

Resources in layers found in portal_skins

Zope 3-style browser views

Zope 3-style viewlets

Zope 3-style browser resources (images, stylesheets)

Zope 3 resources are customised with the ‘layer’ ZCML attribute

Resources in portal_skins are customised in ‘skin layers’ - these have nothing to do with each other!

Visual customisation

In portal_skins in the ZMI, on the Properties tab, find your skin

This lists a number of layers, which refer to folders inside portal_skins itself

Bar custom, these folders are actually files on the file system

Layers higher up take precedence when looking for a template, image or other resource

Visual customisation

Edit example.policy/example/policy/__init__.py:

from Products.CMFCore.DirectoryView import registerDirectoryGLOBALS = globals()registerDirectory('skins', GLOBALS)

Add the folder skins/example_policy

Copy from parts/plone/CMFPlone/skins/* into here and change as necessary

If the file has an associated .metadata file, get this too!

Visual customisation

To install the new layer, create profiles/default/skins.xml:

<object name="portal_skins"> <object name="example_policy" meta_type="Filesystem Directory View" directory="example.policy:skins/example_policy"/> <skin-path name="*"> <layer name="example_policy" insert-after="custom"/> </skin-path></object>

Install or re-install in Plone and look at portal_skins

Visual customisationZope 3 also has the concept of a layer

A Zope 3 layer is just an interface

All the browser ZCML directives can take a layer

If given, the view/viewlet/resource is only available when this layer is in effect - this can also be used to override a particular resource (by name)

With plone.theme, we can associated a layer with a theme

With plone.browserlayer, we can install a layer for a particular product.

Visual customisationCreate an interface in interfaces.py:

from zope.interface import Interfaceclass IExamplePolicy(Interface): """This marker is used for Zope 3 layers """

Then create profiles/default/browserlayer.xml:<layers> <layer name="example.policy.layer" interface="example.policy.interfaces.IExamplePolicy" /></layers>

With this, the IExamplePolicy layer is applied (as a marker interface on the request) on each request in this Plone site after the product has been installed.

Visual customisation

Let’s create a viewlet that is installed with this package

A viewlet is a small snippet of HTML that is inserted in the page, inside a viewlet manager

Create browser/ and browser/__init__.py

In configure.zcml add:

<include package=".browser" />

Visual customisation

Create browser/configure.zcml:

<configure xmlns="http://namespaces.zope.org/zope" xmlns:browser="http://namespaces.zope.org/browser" i18n_domain="example.policy"> <browser:viewlet name="example.footer" manager="plone.app.layout.viewlets.interfaces.IPortalFooter" class=".footer.FooterViewlet" permission="zope2.View" layer="..interfaces.IExamplePolicy" /></configure>

Look at plone.app.layout.viewlets to find standard viewlet managers and viewlets

This is in eggs/plone.app.layout-{version}.egg/...

Visual customisation

Create browser/footer.py:

from plone.app.layout.viewlets.common import ViewletBasefrom Products.Five.browser.pagetemplatefile import \ ViewPageTemplateFileclass FooterViewlet(ViewletBase): """Create an extra footer """ template = ViewPageTemplateFile('footer.pt')

def update(self): super(FooterViewlet, self).update() # calculate stuff here def render(self): return self.template()

Visual customisation

Create browser/footer.pt:

<div> We hope you liked <a tal:attributes="href view/portal_url">our site</a></div>

Now restart Plone, install the product, and you should see the viewlet

Use the /@@mange-viewlets view to move things around

Then make that permanent with viewlets.xml

Visual customisationOur viewlet had a unique name (and layer)

We can also override a browser resource by context type (for attribute) or layer – using the same name

Here, we override a browser view

For simplicity, we specify a template only, and re-use the same class

Some views have templates only

For a class-only view, the __call__() method is called – it can render a template (or do nothing)

Visual customisation

Add this to browser/configure.zcml: <include package="plone.app.portlets" /> <browser:page for="Products.CMFCore.interfaces.ISiteRoot" name="dashboard" permission="plone.app.portlets.ManageOwnPortlets" class="plone.app.layout.dashboard.dashboard.DashboardView" template="dashboard.pt" layer="..interfaces.IExamplePolicy" />

We copied this from plone.app.layout.dashboard

We include plone.app.portlets first to ensure we have the plone.app.portlets.ManageOwnPortlets permission

Visual customisation

Copy plone.app.layout.dashboard’s dashboard.pt to browser/dashboard.pt.

Make a change, e.g. change title to: <h1 class="documentFirstHeading" i18n:translate="heading_dashboard"> <span tal:replace="name" i18n:name="user_name" />'s really cool dashboard</h1>

Next steps

Archetypes content types

Forms and formlib

Themes

New workflows

Portlets

Other components

Deployment buildouts