Default

Intro to Testing
in Zope, Plone

Andriy Mylenkyy
Quintagroup, 2008

What we'll talk about

Testing intro

unittest

doctest

zope.testing

Zope Test Cases

Plone Test Cases

Testing intro

Problem

Types of test

Unit tests

Integration tests

Functional tests

System tests

Testing and documentation

unittest

Author Steve Purcell

Actually is PyUnit, which is part of the Python 2.1 standard library

based on Erich Gamma's JUnit and Kent Beck's Smalltalk testing framework.

main = TestProgram(TP)(ready to use tests runner class)

unittest.py

TCTCTC

TestSuite(TS)TestCase(TC)TestResult(TR)

TestLoader(TL)[test case/suite finder]TestRunner(TR)

TestSuite TestCase

Output results

unittest: TestCase

testMethodName

result

fail*/assert*

(failUnless|assert)Raises

run/debug

__call__

run(result)

result.startTest()result.stopTest()testMethod()setUp()tearDown()

debug

unittest: module members

TestSuite

_test (addTest, addTests)

run/debug (__call__)

TestLoader

define test method prefix ['test']

sort test method function [cmp]

Methods:

loadTestsFrom* [TestCase, Module, Name, Names]

getTestCaseNames -> use test method prefix for find test methods -> used for build TestCase instances

unittest: module members (cont.)

TextTestRunner

TestProgram

FunctionTestCase

TestResult

Help functions:

makeSuite,

findTestCase,

getTestCaseNames

unittest: TestSuite / TestCase

import unittest as ut

class WidgetTC(ut.TestCase): def setUp(self): ... def tearDown(self): ... def testDefaultSize(self): assertEqual(...) def testResize(self): self.failIf(...) def otherTest(self): self.assertRaises(...)

if __name__ == '__main__': # 1 # ut.main() #1b# ut.main(defaultTest='WidgetTC') #2,3# suite = ut.TestSuite() # 2 # suite.addTest(ut.makeSuite(WidgetTC)) # 3 # suite.addTest('WidgetTC('otherTest')) #2,3# runner = ut.TextTestRunner() #2,3# runner.run(sute)

TestSuite

_tests

WidgetTC("testDefaultSize")WidgetTC("testResize")unittest.makeSuite(WidgetTC)

unittest: command line

mytest.py----------------------------------------------import unittestclass WidgetTC(unittest.TestCase): ...if __name__ == '__main__': unittest.main()-----------------------------------------------$ python mytest.py..OK-------------------------------------------------$ python mytest.py WidgetTC.testDefaultSize WidgetTC.testResize..OK-------------------------------------------------$ python mytest.py -vtestDefaultSize (__main__.WidgetTC) ... oktestResize (__main__.WidgetTC) ... ok...OK

doctest

Authors: Jim Fulton, Edward Loper

part of the Python standard library since version 2.1.

import types, mymodule, doctestdef hasInt(*args): """ Test is integer value present in args. >>> hasInt("the string") False

>>> hasInt("the string", 1) True """ args = list(args) while args: if type(args.pop()) == types.IntType: return True return False

__test__ = {'My module test' : mymodule, 'My string test' : >>>1==1True}if __name__ == '__main__': doctest.testmod()

doctest: bases

Python shell with explanations

Docstrings

Separate file

__test__ in module

doctest: testmod/testfile

testmod()

m=None

globs=None

optionflags=0

verbose=None, report=True, raise_on_error=False

testfile(filename)

module_relative=True, package=None

parser=DocTestParser()

same to testmod

run_docstring_examples(f, globs)

compileflags=None

doctest: Option flags

COMPARISON_FLAGS

DONT_ACCEPT_TRUE_FOR_1, DONT_ACCEPT_BLANKLINE, NORMALIZE_WHITESPACE, ELLIPSIS, IGNORE_EXCEPTION_DETAIL

REPORTING_FLAGS

REPORT_ONLY_FIRST_FAILURE, REPORT_*DIFF

`want` strings markers

BLANKLINE_MARKER='', ELLIPSIS_MARKER = '...'

Test ELLIPSIS option >>> range(100) #doctest:+ELLIPSIS [0, ..., 99]

>>> range(100) [0, ..., 99]

import doctest>>> range(100) [0, ..., 99]doctest.testmod( optionflags=doctest.ELLIPSIS)

doctest: Unittest Support

DocTestSuite

Common options: module, globs, extraglobs, test_finder

**options: setUp, tearDown, globs, optionflags

DocFileTest(path)

module_relative=True, package=None

set_unittest_reportflags. Only reporting

>>> range(100)\n[0, ..., 99]import doctest, unittestif __name__ == __main__:suite = unittest.TestSuite() testrunner = unittest.TextTestRunner() testrunner.run(doctest.DocTestSuite(optionflags=doctest.ELLIPSIS)) #testrunner.run(doctest.DocTestSuite())

doctest: Internal structure

objectDocTest

ExampleExampleExampleresultDocTestFinderDocTestRunner

zope.testing

testrunner for run automated tests

primary feature - it finds tests in directory trees

highly customizable

Test filtering

Organization of tests into levels and layers

Reporting

Analysis of test execution

setUp

tearDownsetUp

tearDown

TestCaseTestCaseTestCasezope.testing: Lyaers

Minimize the number of invocations

TestCase/TestSuite associate with layer

Classes with setUp and tearDown class methods

Nested Layers

testSetUp/testTearDown

-l --layer=

Unit layer (-u) / not unit (-f)

zope.testing: Lyaers (cont)

class BaseLayer: @classmethod def setUp(cls): Layer setup @classmethod def tearDown(cls): Layer teardown @classmethod def testSetUp(cls): Before each test setup @classmethod def testTearDown(cls): After each test teardown

class MyTestCase(TestCase): layer = NestedLayer def setUp(self): pass

def tearDown(self): pass

class NestedLayer(BaseLayer): @classmethod def setUp(cls): Layer setup @classmethod def tearDown(cls): Layer teardown

testrunner.py --path', directory_with_tests --layer 112 --layer Unit'.split

zope.testing: Levels

Additional filter alow run tests for specific level

TestCase/TestInteger attribute 'level' integer >0

Test runner can be configured to only run tests at a specific level or below by default.

Additional filter alow run tests for specific level

TestCase/TestInteger attribute 'level' integer >0

Test runner can be configured to only run tests at a specific level or below by default.

If no level -> assumed level 1

by default, only tests at level 1

--at-level (-a) / --all

TestCaseTestCaseTestCaseTestCaseTestCaseTestCase

Level 1

Level 2

TestCaseTestCaseTestCaseLevel - 3

zope.testing: Test selection

Search for

Package (-s, --package, --dir)

Module (-m, --module)

Test within the module (-t, --test )

layer (-l, --level) & level (-a, --at-level / --all )

--tests-pattern (--tests-pattern ^tests$ )

--test-suite ('test_suite' by default)

Module & Test can be re.expr, positional args

zope.testing: options (cont)

Where search for tests

--path

--test-path / --package-path --> not add to sys.path

--progress (-p). -v (with test descriptions)

--coverage= - where put coverage reports

--profile

--repeat

--verbose (-v) / --quiet (-q)

zope.testing: test.py script

Wrapper around zope.testing.testrunner

Extends testrunner with options:

--config-file

--nowarnings

Prepare default options list for call with testrunner:

--path = INSTANCE_HOME/python/lib

--package-path products Products

call testrunner.run

zope.testing: examples

$ZH/bin/test.py --config-file=$IH/etc/zope.conf \

--path=~/Plone/lib/python \

--package-path ~/Plone/Products Products \

-s Products.CMFQuickInstallerTool

$INSTANCE_HOME/bin/zopectl test \

-s Products.CMFQuickInstallerTool

ZopeTestCase(ZTC)

Author : Stefan H. Holek

ZTC Goals

Simplify distribute tests with Zope products.

Be as close to real Zope as possible.

Be as simple to use as unittest.

Decisions

Base ZTC on Testing and improve on it

Provide a batteries included Zope test sandbox

Support for testing Zope security.

ZTC: framework.py vs test.py

framework.py now depricated

bin/test.py config-file=.../zope.conf ...

BUT:

Supports custom_zodb.py in the tests directory

Allows to connect to a running ZEO server

import osfrom ZODB.DemoStorage import DemoStoragefrom ZODB.FileStorage import FileStoragedb = os.path.join('..', '..', '..', 'var', 'Data.fs')db = os.path.abspath(db)Storage = DemoStorage(base=FileStorage(db, read_only=1))

ZTC: ZopeTestCase.py

ZopeTestCase class

ZopeTestCase sets up a default fixture

Handles opening and closing of ZODB con.

Handles transactions

Provides a REQUEST object

API to modify the test users credentials

ZTC: Entrypoints

setUp(self)try: self.beforeSetUp() self.app = self._app() self._setup() self.afterSetUp()except: self._clear() raise

tearDown(self)try: self.beforeTearDown() self._clear(1)except: self._clear() raise

Zope.APP

Opens a ZODB connection Request.__of__(app)

Before ALL

ZTC Folder, UserFolder, User, login

After Fixture created

ConnectionRegistry

Before delete folderfrom self.app

After folder deleteBefore close connection to ZODBbeforeClose()

afterClear()

After ALL

ZTC: Class diagram

ZTC: Writing Tests

Create a test case class derived from ZopeTestCase.ZopeTestCase

Override afterSetUp() / tearDown() to add the objects you want to test, clearing data

Write one or more tests exercising these objects

Define test_suite object, which return unittest.TestSuite object

ZTC: Example

import unittest

from Testing import ZopeTestCase

ZopeTestCase.installProduct(SomeProduct)

class MyTest(ZopeTestCase.ZopeTestCase):

def afterSetUp(self):

self.folder.addDTMLMethod(foo)

def testFoo(self):

assert hasattr(self.folder, foo)

def test_suite():

return unittest.TestSuite(

[unittest.makeSuite(MyTest),] )

PortalTestCase

Almost ZopeTestCase

portal object (self.portal)

user folder (UF) inside the portal

portal object must create in Subclasses

getPortal() returns a usable portal object

Defaults:

user with role 'Member' inside the UF

default user's memberarea (self.folder)

default user is logged in

PortalTestCase (cont)

portal = 'portal'

setUp(self)try: self.beforeSetUp() self.app = self._app() self.portal = self._portal() self._setup() self.afterSetUp()except: self._clear() raise

_portal(self)return self.getPortal()

getPortal(self)return getattr(self.app, poratl)

PloneTestCase(PTC)

PloneTestCase class

Usefull methods:

hasProduct. installProduct

hasPackage, installPackage

setupPloneSite(products, extension_profiles)

Constants:

portal_*, default_*

PLONE2.x, PLONE3.x, PLONE4.x

PTC: PloneTestCase (cont)

Plone installs in setup.py module

ZCMLLayer, PloneSiteLayer(ZCML) layers

onsetup, onteardown functions.

Extend PortalTestCase security with methods:

setGroups, loginAsPortalOwner

PTC: PloneTestCase example

import os, sysif __name__ == '__main__': execfile(os.path.join(sys.path[0], 'framework.py'))

from Products.PloneTestCase import PloneTestCase

PloneTestCase.setupPloneSite()

class TestDocument(PloneTestCase.PloneTestCase):

def afterSetUp(self): ...

def testAddDocument(self): self.failUnless(...)

def test_suite(): ... return suite

if __name__ == '__main__': framework()

Resources

Sources ;)

http://plone.org/documentation/tutorial/testing

http://www.zope.org/Members/shh/ZopeTestCaseWiki

http://pypi.python.org/pypi/zope.testing/3.6.0

http://agiletesting.blogspot.com

http://pyunit.sourceforge.net

http://svn.plone.org/svn/collective/examples/example.tests/trunk

Click to edit the title text format