Types of Tests

Unit tests

Testing done in as extremely minimal fashion as possible. Having some monstrous setup done before hand is not acceptable. These tests are done in a mind set by which the developer knows what the inner code looks like so he can test that certain inner code flags/values have been set after running the test.

Integration tests

Testing done very similar to unit tests (that is they're all code based and don't actually go through the browser interface) but require some integration to have been performed. In the case of Plone this often means having a complete plone site installed with all the trimmings. Basically these are unit tests on steriods. 95% of plone add-on developers write their tests these ways (the fastest way to see if a test is an integration test is to see if it uses PloneTestCase or ZopeTestCase ... if it does, it's an integration test). If a test requires a working portal instance, it's an integration test.

Functional tests

Testing from as close to the real environment as possible, in most casee this means using something like selenium or testbrowser (I tend to use testbrowser). These tests never touch actual code api's (other than to run the mock web browser).

An Example

Let's say we have browser.py. Anyone familiar with the newer Zope 3 style way of coding applications is familar with the browser module. In this case, as expected, browser.py is giving us view classes that ultimately will get accessed via a web browser. Here's how we would write different types of tests for that browser module.

But before we get into the actual tests, We need to begin by defining browser.py as follows:

from Products.Five.browser import BrowserView
from Products.CMFCore import utils as cmfutils

class SimpleView(BrowserView):
    """A simple view."""

    def nextval(self):
        portal = cmfutils.getToolByName(self.context, 'portal_url') \
                 .getPortalObject()
        current = getattr(portal, '_simpleview_count', 0)
        portal._simpleview_count = current + 1
        return portal._simpleview_count

    def __call__(self):
        return 'Retrieved %i' % self.nextval()

Also need configure.zcml:

<configure
  xmlns="http://namespaces.zope.org/zope"
  xmlns:browser="http://namespaces.zope.org/browser">

<browser:page
    name="simpleview"
    for="*"
    class=".browser.SimpleView"
    permission="zope.Public"
    />

</configure>

Unit test

In general no setup will be performed for this at all. Here the developer would simply import the browser module and instantiate the view classes using python code. And then with the view instance, test each of the methods. In general when any additional functionality is needed, it's done in the form of mock objects.

Here's what the test harness looks like (in this example, expected to live as tests/test_unit.py):

import unittest
from zope.testing import doctest

def test_suite():
    return unittest.TestSuite(doctest.DocFileSuite
                              ('unit-example.txt',
                               package='testingexample'))

Here's the test (in doctest-style) as is expected to live in unit-example.txt:

First some mock objects.

    >>> class Mock(object):
    ...     def __init__(self, **kwargs):
    ...         for k, v in kwargs.items(): setattr(self, k, v)

    >>> portal = Mock()
    >>> context = Mock(portal_url=Mock(getPortalObject=lambda: portal))

We don't bother with request since we know the innards of our code and
the fact that it doesn't use the request for anything.

    >>> from testingexample.browser import SimpleView
    >>> view = SimpleView(context, None)
    >>> view.nextval()
    1
    >>> portal._simpleview_count
    1

And adjusting the private var manually will work as expected.

    >>> portal._simpleview_count = 50
    >>> view.nextval()
    51
    >>> portal._simpleview_count
    51

And then testing the string output of ``__call__``.

    >>> view()
    'Retrieved 52'
    >>> portal._simpleview_count
    52

The only way to see this break is if someone corrupted the _simpleview_count
value (which we should have to account for anyhow).

    >>> portal._simpleview_count = 'foobar'
    >>> view.nextval()
    Traceback (most recent call last):
    TypeError: cannot concatenate 'str' and 'int' objects

Integration test

Use a setUp that sets up a simple plone site and installs any plone add-ons we need. Use code like view = somecontentobj.restrictedTraverse('@@someview') to look up a view that is being created with browser.py and interact with that view component on an api level.

Here's what the test harness looks like (in this example, expected to live as tests/test_integration.py):

import unittest
import testingexample
from Testing import ZopeTestCase
from Testing.ZopeTestCase.zopedoctest import ZopeDocFileSuite
from Products.PloneTestCase import PloneTestCase
from Products.PloneTestCase.layer import PloneSite
from Products.Five import zcml

PloneTestCase.setupPloneSite()

class MainTestCase(PloneTestCase.PloneTestCase):
    def afterSetUp(self):
        zcml.load_config('configure.zcml', testingexample)
        self.portal._simpleview_count = 0

def test_suite():
    suite = ZopeDocFileSuite('integration-example.txt',
                             package='testingexample',
                             test_class=MainTestCase)
    suite.layer = PloneSite

    return unittest.TestSuite((suite,))

And here's the actual tests:

In these tests we expect that the portal object has already been setup
(ala ``PloneTestCase``) and is available as simply ``portal``.

    >>> portal
    <PloneSite at /plone>

Our first integration test just checks to make sure that we can actually
lookup the view by traversing.

    >>> view = portal.restrictedTraverse('@@simpleview')
    >>> view is not None
    True

Our view instance is already expected to have a working *context* and
*request* so we can continue as expected.

    >>> view.nextval()
    1
    >>> portal._simpleview_count
    1

And adjusting the private var manually will work as expected.

    >>> portal._simpleview_count = 50
    >>> view.nextval()
    51
    >>> portal._simpleview_count
    51

And then testing the string output of ``__call__``.

    >>> view()
    'Retrieved 52'
    >>> portal._simpleview_count
    52

The only way to see this break is if someone corrupted the _simpleview_count
value (which we should have to account for anyhow).

    >>> portal._simpleview_count = 'foobar'
    >>> view.nextval()
    Traceback (most recent call last):
    TypeError: cannot concatenate 'str' and 'int' objects

Functional test

Use a setUp that sets up a simple plone site and installs any plone add-ons we need. Instantiate a test browser instance (via zope.testbrowser) and mimick browser actions to "log into" the site and access whatever views were produced by browser.py.

Here's what the test harness looks like (in this example, expected to live as tests/test_functional.py):

import unittest
import testingexample
from Testing import ZopeTestCase
from Testing.ZopeTestCase import FunctionalDocFileSuite
from Products.PloneTestCase import PloneTestCase
from Products.PloneTestCase.layer import PloneSite
from Products.Five import zcml

PloneTestCase.setupPloneSite()

class MainTestCase(PloneTestCase.PloneTestCase):
    def afterSetUp(self):
        zcml.load_config('configure.zcml', testingexample)
        self.portal._simpleview_count = 0

def test_suite():
    suite = FunctionalDocFileSuite('functional-example.txt',
                                   package='testingexample',
                                   test_class=MainTestCase)
    suite.layer = PloneSite

    return unittest.TestSuite((suite,))

And here's the actual tests:

These tests are all about seeing and testing what the browser sees.  We
make no assumptions on the innards of the code -- pretending we have indeed
never seen the code itself.

First we need to setup a browser instance.

    >>> from Products.Five.testbrowser import Browser
    >>> browser = Browser()

Now we can start checking things out.  Really all we can test here now
is that the output to the browser has an integer that increments each time.

    >>> browser.open(portal.absolute_url()+'/@@simpleview')
    >>> browser.contents
    'Retrieved 1'

    >>> browser.open(portal.absolute_url()+'/@@simpleview')
    >>> browser.contents
    'Retrieved 2'

Seeing It In Use

All of the example code here can be found in the svn collective as an actual python package. Read the included README.txt to figure out how to set it up in your own zope instance. The package is available at: http://svn.plone.org/svn/collective/examples/testingexample/tags/rocky-blog-post-20070919/

The tests are meant to be run with:

$ ./bin/zopectl test -s testingexample

But don't forget that when developing code you can save yourself a ton of time by maintaining 100% unit test coverage. Then you can run the unit tests as often as you want (at a very rapid speed) and only run the integration and/or functional tests at milestone intervals. To run them separately you would do:

$ ./bin/zopectl test -m testingexample.tests.test_unit
$ ./bin/zopectl test -m testingexample.tests.test_integration
$ ./bin/zopectl test -m testingexample.tests.test_functional

Conclusion

Testing is great. I'm not particularly advocating test driven development (it works for some people, but other people it does not). But it's important to understand the differences between the different types of tests. The author suggests maintaining 100% unit test coverage and some (client-derived) acceptable amount of functional tests. Integration tests aren't so important when you already have good unit and functional test coverage.

Viewlets

A viewlet's render() method needs to return unicode objects because the standard zope3 viewlet implementation concatenates the output of various viewlet render()'s. If the output is a str, then implicit conversion-to-unicode will happen using the default python character encoding which is most often ascii (causing a utf-8 encoded str with non-ascii chars to blow up).

Archetypes Schema Fields

Archetypes (as present in Plone 2.5 and 3.0, and possibly older) stores all StringField fields as utf-8 encoded str's irrelevant of what the Plone site encoding has been configured as. Safest way to handle this is to always feed StringField field mutators unicode objects instead of str's. And upon retrieval, be prepared to decode the str's using utf-8 (ie context.Title().decode('utf-8') => u'someval'). Remember, StringField accessors will always return utf-8 encoded str's and Zope 3 often makes assumptions that the string values it's dealing with are unicode objects.

Summing Up

Use unicode objects everywhere. If some Zope 2 / Plone API forces you to use str's or get str's convert them to unicode's before doing anything else.

Feedfeeder

I recently worked on a Zope 2 product for Zest Software called feedfeeder that would use the feedparser python library to create content items within a Plone site. The basic premise was that there was a folderish content type that could be assigned URL's to ATOM feeds. Then there was an action (a simple Zope 3 view) made available on the folder that when called would pull down the ATOM feeds and hand the content over to a Zope 3 utility. That utility would then create the appropriate content representing the ATOM entries.

This was all find and dandy for handling generic ATOM feeds. The problem is that the client had specific entry metadata they wanted to make available for each ATOM entry. Of course we didn't want to extend the feedfeeder product to take into account the client specific data so we had to figure out an acceptable manner of extending it.

<entry>
  <title>Bicycle service closed 17 July-7 August</title>
  <id>http://somecustomer.com/someuid</id>
  <content type="xhtml">
    <div xmlns="http://www.w3.org/1999/xhtml">
Here lies the body of the atom entry.
    </div>
 </content>
 <updated>2006-07-18T12:00:00Z</updated>
 <author><name>Joe Schmoe</name></author>
</entry>

<dl>
  <dt>somename</dt>
  <dd>somevalue</dd>
</dl>

<entry>
  <title>Bicycle service closed 17 July-7 August</title>
  <id>http://somecustomer.com/someuid</id>
  <content type="xhtml">
    <div xmlns="http://www.w3.org/1999/xhtml" class="colloquia">
      <dl>
        <dt>date</dt>
        <dd>2006-03-23</dd>
      </dl>
      <div>
Here lies the body of the atom entry.
      </div>
    </div>
 </content>
 <updated>2006-07-18T12:00:00Z</updated>
 <author><name>Joe Schmoe</name></author>
</entry>

Extending Feedfeeder

The question we're left to answer is:

How do we provide a replaceable mechanism in feedfeeder for handling the metadata?

And the answer is:

Abstract the content handling logic to lookup up an adapter to deal with the content.

So since we knew our microformat had to use the XHTML type, within the feedfeeder code for handling the content we would only do our thing if the type was XHTML.

1. Check the content type, is it XHTML?
2. If the answer is "yes", proceed to step #4
3. If the answer is "no", simply go about your default business
4. Since it's XML (XHTML), we use the standard python library package, xml.dom.minidom, to handle the body as a DOM fragment
5. Next we look up the class attribute on the toplevel DIV tag within the DOM fragment
6. Do a named adapter lookup providing IFeedItemContentHandler using the class attribute as the name
7. Did we find an adapter? If so, we call it's handling function passing in the DOM fragment
8. We didn't find an adapter? Then lookup the default unnamed adapter providing IFeedItemContentHandler and pass it the DOM fragement

Extending Client Product

So now we've given our feedfeeder product way of extending it's content handling without the feedfeeder ever needing to know the specific things that could be done by external specific code.

What next? We go inside the client product implementing specific client business logic and provide a named adapter that provides IFeedItemContentHandler. In this case the type of metadata we want to handle is referred to as Colloquia. Now we'll go ahead and make the name of the adapter be "colloquia". Here's an example of the ZCML used to register the adapter:

<adapter
    provides="Products.feedfeeder.interfaces.contenthandler.IFeedItemContentHandler"
    for="Products.feedfeeder.interfaces.item.IFeedItem"
    name="colloquia"
    factory=".colloquia.ColloquiaContentHandler"
    />

With the actual ColloquiaContentHandler class we simply provide the appropriate handling function that will now take the DOM fragment and parse the toplevel DL tag to determine what metadata is available.

# make sure the content item implements IAttributeAnnotatable,
# otherwise we won't be able to use annotations on it
if not IAttributeAnnotatable.providedBy(self.context):
    directly = zope.interface.directlyProvidedBy(self.context)
    zope.interface.directlyProvides(self.context,  
                                    directly + IAttributeAnnotatable)
annotations = IAnnotations(self.context)
metadata = annotations.get('colloquia', None)
if metadata is None:
    metadata = PersistentDict()
    annotations['colloquia'] = metadata

# and what we're left with is a simple dict referenced by the
# variable, metadata

for dl_el in contentNode.childNodes:
    if dl_el.nodeName != 'dl':
        continue
          
    term = None
    for el in dl_el.childNodes:
        if el.nodeName == 'dt':
            term = self._extractText(el)
        elif el.nodeName == 'dd':
            definition = self._extractText(el)
            metadata[term] = definition

Traditional Archetypes Schema's

By now the entire Plone and Plone add-on developer communities generally use the Archetypes framework to construct their content types. The Archetypes framework introduced its own concepts of schema's, widgets, and fields which suited most developers fine. The problem is that shortly thereafter (perhaps even simultaneously) the Zope 3 community was already building their own constructs of these same concepts. The result was zope.interface, zope.schema, and zope.app.form. Eventually came zope.formlib that would help glue more of this together in a web UI environment.

Zope 3 Schema's

A Zope 3 schema functions very closely to an Archetypes schema from a conceptual perspective. They both share the basic principles of defining the fields of a content type. One glaring difference is that is that a Zope 3 schema field has no direct association (within the schema) with any sort of UI logic. That is, it does not know what widget will be used to display itself on a web form. This help keeps content and UI concerns separate. Content, after all, should never need to know or care what it will actually look like in a user interface. It only cares about, well, data.

Our Example

For purposes of keeping things clean, we will break up some of our previous example code into more common python modules. This means moving our ISearch interface from the browser module into a new interfaces module.

class IExampleContent(interface.Interface):
    title = schema.TextLine(title=u'Title',
                            required=True)

    description = schema.Text(title=u'Description',
                              required=False)

    funny = schema.Bool(title=u'Am I Funny?',
                        default=False,
                        required=True)

    really_funny = schema.Bool(title=u'Am I Really and Truly Funny?',
                               default=False,
                               required=True)

def YesNoWidget(field, request, true=_('yes'), false=_('no')):
    vocabulary = schemavocab.SimpleVocabulary.fromItems(((true, True), 
                                                         (false, False)))
    return form_browser.RadioWidget(field, vocabulary, request)

example_content_fields = form.FormFields(interfaces.IExampleContent)
example_content_fields['really_funny'].custom_widget = YesNoWidget

class ExampleContentView(formbase.DisplayForm):
    form_fields = example_content_fields

    def __init__(self, *args, **kwargs):
        formbase.DisplayForm.__init__(self, *args, **kwargs)
    
        # a hack to make the content tab work
        self.template.getId = lambda: 'index.html'

class ExampleContentEditForm(formbase.EditForm):
    form_fields = example_content_fields

    def __init__(self, *args, **kwargs):
        formbase.EditForm.__init__(self, *args, **kwargs)
    
        # a hack to make the content tab work
        self.template.getId = lambda: 'edit.html'

<browser:page
    name="index.html"
    for=".interfaces.IExampleContent"
    class=".browser.ExampleContentView"
    permission="zope2.View"
    />

<browser:page
    name="edit.html"
    for=".interfaces.IExampleContent"
    class=".browser.ExampleContentEditForm"
    permission="cmf.ModifyPortalContent"
    />

class FormlibExampleContent(atapi.BaseContent):
    interface.implements(interfaces.IExampleContent)

    title = fieldproperty.FieldProperty(interfaces.IExampleContent['title'])
    description = fieldproperty.FieldProperty(interfaces.IExampleContent['description'])
    funny = fieldproperty.FieldProperty(interfaces.IExampleContent['funny'])
    really_funny = fieldproperty.FieldProperty(interfaces.IExampleContent['really_funny'])

def catalog_content(obj, event):
    obj.reindexObject()

<subscriber
    for=".interfaces.IExampleContent
         zope.app.event.interfaces.IObjectModifiedEvent"
    handler=".content.catalog_content"
    />

Conclusion

Zope 3 schema's in Plone has arrived. And thanks to zope.formlib we have auto-generated views and forms to boot. The widget and field sets to choose from in Zope 3 are a lot smaller than in the Archetypes world, but Zope 3 is quickly catching up.

The only main missing piece (from formlib's perspective) here is using an auto-generated add form. While we can build those, they can't easily be hooked into Plone as Plone feels it needs to create the content first before displaying any forms.

Understanding Formlib

zope.formlib is a Zope 3 package designed to ease the development of web-based forms in your Zope applications. In its simplest form you can compare what it does with the auto-generated displays Archetypes provides you for viewing a content type (base_view) and editing a content type (base_edit). For all practical sense formlib based components are really regular Zope view components with some convenient base classes for auto-generating output based on schema's and other configuration info.

Thankfully beginning with Zope 2.9.3 zope.formlib is now being distributed with Zope 2. Of course Five >= 1.4 is required to make use of this Zope 3 package.

Defining Our First Form

For purposes of this writing we will construct a very simple search form for searching Plone content. This form will be similar to Plone's built in advanced search form but much simpler.

You can view the working source code of these examples at the updated collective svn browser and updated collective svn repository locations.

from zope import interface, schema
from zope.formlib import form
from Products.CMFCore import utils as cmfutils
from Products.Five.browser import pagetemplatefile
from Products.Five.formlib import formbase

class ISearch(interface.Interface):
    text = schema.TextLine(title=u'Search Text',
                           description=u'The text to search for',
                           required=False)
    
    description = schema.TextLine(title=u'Description',
                                  required=False)

class SearchForm(formbase.PageForm):
    form_fields = form.FormFields(ISearch)
    result_template = pagetemplatefile.ZopeTwoPageTemplateFile('search-results.pt')

@form.action("search")
def action_search(self, action, data):
    catalog = cmfutils.getToolByName(self.context, 'portal_catalog')
    
    kwargs = {}
    if data['text']:
        kwargs['SearchableText'] = data['text']
    if data['description']:
        kwargs['description'] = data['description']
    
    self.search_results = catalog(**kwargs)
    self.search_results_count = len(self.search_results)
    return self.result_template()

<tal:block tal:repeat="single view/search_results">
<div class="single-result">
  <h4>
    <span tal:replace="repeat/single/number"></span>. 
    <a tal:content="single/Title" tal:attributes="href single/getURL" href=""></a>
  </h4>
  <p tal:content="single/Description"></p>
</div>
</tal:block>

<browser:page
    name="search.html"
    for="Products.CMFPlone.Portal.PloneSite"
    class=".browser.SearchForm"
    permission="zope.Public"
    />

Our First zope.formlib Example In Summary

The example demonstrated here shows the simplest form that could be created with formlib and how to hook in a simple action. It should be obvious from this example how you could use formlib to replace simple CMFFormController based logic. Of course formlib can do many other advanced things such as provide sub-form functionality and autogenerated add and edit forms for content classes.

The bottom line is that zope.formlib is ready for use inside Plone today. And since formlib is so easy to work with, the author recommends all Plone application developers give it a try.

Required Stack Components

The example built by this writing is designed to run on the following stack:

• Python >= 2.4.3
• Zope >= 2.9.3
• Plone >= 2.5
• Five >= 1.4

The example described in this writing may work on different versions but is untested by the author so your mileage may vary.

Some Things To Note Before Starting

We will begin by working with a product by the name of ploneexample.formlib. The finished code for this example can be browsed using the collective svn browser or checked out from Subversion using at the collective svn repository.

The first thing the keen Plone application developer will notice is the unusual naming convention of the product. Traditionally Plone products use CamelCase as the convention for naming products with no additional periods (ie PloneExampleFormlib). But as one of the goals of this writing, best practices will be demonstrated where we try to be as pythonic as possible which means using all lowercase for package names.

The Importance Of Pythonic

Many of you may be asking yourself, "but why do we really care about keeping this pythonic?" The single biggest reason for keeping things pythonic is to keep things as simple and easy to understand for people who are already familiar with Python. When developing standard Zope 2 products and placing them into the Products directory of your Zope 2 instance, Zope magically places them within the Products namespace package (ie so the full package path of CMFPlone actually becomes Products.CMFPlone). Living in PYTHONPATH like any other package and being reusable in general is A Good Thing TM.

Also, by keeping products pythonic, we can now use generic Python tools such as easy_install and setuptools (both great package management components) to work with these products.

Creating The Product

For the initial creation of the product we will use a small component of the Python Paste project. The rest of this writing will assume the reader has installed Phillip J. Eby's setuptools and easy_install products. As well as the necessary paster component along with the ZopeSkel template package. For instructions on how to setup paster and ZopeSkel, please see Daniel Nouri's excellent article on ZopeSkel.

So lets get started:

$ paster create -t plone_core ploneexample.formlib
Selected and implied templates:
  ZopeSkel#plone_core  A Plone Core project

Variables:
  package:  ploneexampleformlib
  project:  ploneexample.formlib
Creating template plone_core
Enter namespace_package (Namespace package) ['plone']: ploneexample
Enter package (The package contained namespace package (like i18n)) ['']: formlib
Enter pythonproducts (Are you making a productsless Zope 2 Product?) [False]: True
Enter version (Version) ['0.1']:
Enter description (One-line description of the package) ['']: A Plone product for demonstrating zope.formlib usage
Enter long_description (Multi-line description (in reST)) ['']:
Enter author (Author name) ['Plone Foundation']: Rocky Burt
Enter author_email (Author email) ['plone-developers@lists.sourceforge.net']: rocky@serverzen.com
Enter keywords (Space-separated keywords/tags) ['']:
Enter url (URL of homepage) ['http://svn.plone.org/svn/plone/plone.i18n']: http://dev.plone.org/collective/browser/examples/ploneexample.formlib
Enter license_name (License name) ['GPL']:
Enter zip_safe (True/False: if the package can be distributed as a .zip file) [False]:
  Recursing into +namespace_package+
    Creating ./ploneexample.formlib/ploneexample/
    Recursing into +package+
      Creating ./ploneexample.formlib/ploneexample/formlib/
      Copying HISTORY.txt_tmpl to ./ploneexample.formlib/ploneexample/formlib/HISTORY.txt
      Copying LICENSE.GPL to ./ploneexample.formlib/ploneexample/formlib/LICENSE.GPL
      Copying LICENSE.txt_tmpl to ./ploneexample.formlib/ploneexample/formlib/LICENSE.txt
      Copying README.txt_tmpl to ./ploneexample.formlib/ploneexample/formlib/README.txt
      Copying __init__.py_tmpl to ./ploneexample.formlib/ploneexample/formlib/__init__.py
      Copying configure.zcml to ./ploneexample.formlib/ploneexample/formlib/configure.zcml
      Copying version.txt_tmpl to ./ploneexample.formlib/ploneexample/formlib/version.txt
    Copying __init__.py_tmpl to ./ploneexample.formlib/ploneexample/__init__.py
  Copying setup.cfg to ./ploneexample.formlib/setup.cfg
  Copying setup.py_tmpl to ./ploneexample.formlib/setup.py
Running /usr/bin/python2.4 setup.py egg_info

Setting Up Development

After this has completed running you should now have a ploneexample.formlib directory in your current working directory. For purposes of development we will now use setuptools to setup the ploneexample.formlib project on the PYTHONPATH in order to make it available to Zope. So make sure you make the ploneexample.formlib directory your current directory and do the following (remember to run this with sudo if you're in a UNIX environment and don't have permission to write to the system python site-packages directory):

$ python2.4 setup.py develop

This will yield some output similar to the following:

[snip output...]
Installed /home/rocky/Documents/developing/projects/ploneexample.formlib
Processing dependencies for ploneexample.formlib==0.1dev

Hooking Up The New Product Into Zope 2

Now that we have a fully runnable Zope 2 product we will proceed with hooking this up to an existing Zope 2 instance. Until Zope and CMF/Plone catch up (Zope 2.10 already includes the necessary changes) we will need to use the pythonproducts product to enable Zope 2 products to exist outside of the instance Products directory.

So be sure to download and install the pythonproducts product into your Zope 2 instance by following the install instructions provided by pythonproducts. For those in a hurry, setting up pythonproducts is as simple as downloading the pythonproducts tarball, extracting the contents into some temporary directory, and running python setup.py install --home $INSTANCE_HOME.

Now you need to tell your Zope 2 instance to "activate" the new ploneexample.formlib package as a Zope 2 product. You do this by going to the etc/package-includes directory in your Zope 2 instance and create a new file there called ploneexample.formlib-configure.zcml with its only contents being:

<include package="ploneexample.formlib" />

You should now test your Zope 2 instance to confirm that the new ploneexample.formlib product is available. You can do this by starting your Zope 2 instance as you normally would and going to the Products section of the Control Panel via the ZMI. Towards the bottom of the list you should see:

ploneexample.formlib (Installed product ploneexample.formlib)

Stay tuned for the next part of this series!

The Theory Behind XM

The big idea behind xm is to break up tasks into organized components that can be managed in a sane manner and then to ensure these components are worked on together by both the customer and the project team. And of course provide organization on a per-project basis.

The three types of components are:

1. Tasks
2. Stories
3. Iterations

Business Impact

Random Non-Plone Impacts

From the perspective of a traditional Zope 2 developer deciding whether a new project should be developed and deployed on Zope 2 VS Zope 3 there are two different types of people:

1. Zope 2 developer who is accustomed and familiar with Zope 2 development techniques with no interest to move outside of the Zope 2 development model.
2. Zope 2 developer who is interested in Zope 3 development techniques but due to missing functionality (Zope 2 has obviously been around a lot longer than Zope 3) in Zope 3 has to develop and deploy on Zope 2.

Obviously the move to Zope 3 development techniques for #1 Zope developer doesn't make any sense. But developer #2 would be very interested in getting equivalent (in capability) functionality from Zope 2 into Zope 3. And once enough of the required Zope 2 functionality has found a capable equivalent in Zope 3, #2 developers will have a path to fully migrate to developing and deploying on Zope 3.

In order for a full set of Plone components to deploy and run on Zope 3 a lot of this traditional Zope 2 functionality will have to have equivalents on Zope 3. So in the process of providing this functionality for Plone on Zope 3, Zope 2 developers will have much of their wanted/needed functionality migrated as well. This means non-Plone Zope 2 developers will have a higher likelihood of being able to deploy on Zope 3.

Conclusion

It is the recommendation of this author that moving to Zope 3 development techniques is a required manner of evolution for Plone and the Plone software stack (which is in agreement with the Goldegg effort of course). And related to this, the development process and attitude should be to build Zope 3 components on Zope 3 and build them so they (for the some finite time period) can be deployed on Zope 2 until the convergence of Zope 2 and Zope 3 is complete.

CMFonFive Functionality

It used to be that CMFonFive provided a fair amount of functionality when building Zope 3 / Five applications with CMF. These days most of that functionality has been integrated directly into CMF. As of CMF 1.5.2, CMF includes both the interface work and standard macros pieces that CMFonFive used to provide. At this point this means the only functionality that is left specific to CMFonFive is the bridge between the Zope 3 menu and CMF actions facilities.

Zope 3 Menu Facilities

A menu in Zope 3 is analogous to menu's in standard desktop applications. Such menu's are generally built to provide categorized actions available to the running application and are often context sensitive (ie the menu will change shape, deactivate items, etc when the application changes what data is being viewed).

In Zope 3, menu's are registered as utilities. There are a few specific core Zope 3 menu's that are registered on startup including (but not limited to) zmi_views and zmi_actions. These specific menu's are used to display actions available in the Zope 3 ZMI [*]. Its easy enough to add your own actions to these menu's by defining a browser menu item declaration such as:

<browser:menuItem
    for="some.interface.IHere"
    menu="zmi_views"
    title="Preview"
    action="preview.html"
    permission="zope.ManageContent"
    />

CMF and Menu's

Since Zope 2 (by default) doesn't provide any real notion of categorized actions its left up to CMF to pick up the slack. CMF has the portal_actions tool and related API which brings a concept of categorized actions to Zope 2. Some of the categories that CMF provides by default are object, user, and folder. The standard way to add new actions to those categories is to either go to the ZMI and add new actions to one of the tools there (filling in the appropriate 'Category' field) or by (in code) creating a new CMF tool that provides its own set of actions.

CMFonFive Bridges CMF Actions and Zope 3 Menu's

CMFonFive has its own CMF tool that implements the CMF actions interfaces and translates Zope 3 defined menu's and menu items into CMF action categories and actions. So, to get a new "user" action to show up in the CMF UI, you would do the following in ZCML:

<browser:menu
  title="user"
  id="user"
  />
  
<browser:menuItem
  for="*"
  menu="user"
  title="Manage Hours"
  action="${portal_url}/multiplehours.html"
  permission="zope2.View"
  />

This ends up being a lot less work than the standard way of defining CMF actions in a standard CMF based application. No CMF tools required; no manual ZMI configuration required.

Note: Since Plone is based on CMF, all of the previous text applies to Plone equally.

Skeletor

In a recent posting to the plone-users mailing list Alan Runyan says the following:

so while I agree with this helpful suggestion - I dont think its something that most people on plone.users will understand or benefit from tremendously. What we really need is something like Skeletor that will pre-generate some code harness. I would see it something like this:

python skeletor.py -id "MyProject"

Personally I agree with the need for something like this and think Skeletor is on the right path. But, there shouldn't be as much of a need for a product like Skeletor as there is currently. The problem is that there is too much boilerplate code right now. It shouldn't be 10 near identical steps to create a new content type. Also there needs to be less configuration by python code and more configuration by external means (in Zope 3 this would be done by ZCML).

Zope 3 minimizes boilerplate code by quite a bit (this may be due to the fact there are currently less layers requiring their own boilerplates compared to the common Plone/CMF/Archetypes/Zope2 combo). But there is certainly less work (and more importantly, less code) involved in setting up a new content type.

So lets minimize the work needed to be done with all of these frameworks to make an application work, shall we?

Zope3 Is In Zope2, Huh?

I've heard on many occasions that today the latest Zope2 releases contain Zope3 but it's taken a while to truly realize what that means. A quick inspection of SOFTWARE_HOME/lib/python shows the usual python packages we're used to dealing with in Zope2. OFS, App, zLOG, all of these get used quite often. At the very end of this list (if you're folder listing sorts alphabetically) is a little unused package called "zope". Hmm... as a Zope2 developer, I've never used this package.

One of the nice things about Zope3 is that it's more or less just a packaging of Python packages that basically all start from the base "zope" package. Click (that was the sound of a light bulb coming on, btw). That "zope" package contained in my Zope2 SOFTWARE_HOME/lib/python was in fact the toplevel package for all Zope3 source code. As of Zope 2.8, this means Zope 2.8 contains Zope X3 3.0 in its entirety. Beginning with Zope 2.9, Zope 3.2 will be included.

So, now that we have Zope3 at our finger tips in Zope2-world, how do we use it?

Enter Five

Zope2 and Zope3 developers talked and talked (I'm imagining) about what would be possible regarding using Zope3 technologies in Zope2 given that their api's aren't exactly compatible. And with a little bit of nudging and a little bit of funding, Martijn Faassen took upon himself the task of making these zope.* packages usable in Zope2. (Of course this isn't the full story but its enough for my story-telling purposes) These days Martijn Faassen and Philipp von Weitershausen consume many daily cycles in an effort to maintain Five and bring into Five even more pieces for working with the Zope3 techniques in Zope2.

Zope3-Based Development Using Five

Probably one of the most often used development terms used Plone development is the term, "Five views". This is a bit of a misnomer as the views are actually a Zope3 concept. Its just that Five enables use to use Zope3 views in Zope2.

Views

Views are an improved way of separating view logic from page templates. This could be described as a MVC (Model View Controller) approach with the model being the zodb, view being the page template, and the controller being a separate class called the view class.

Interfaces

Interfaces are one common object-oriented aspect that Python lacks natively and so has been provided by Zope. Zope2 had interfaces in a simple (ugly?) manner but Zope3 really shines with its interfaces implementation.

Adapters

Need one interface to fit (api-wise) against another interface? Use an adapter.

Utilities

Often uses as a CMF tool replacement mechanism. Provides a nice way to access useful functionality without having to stuff content classes with inappropriate logic.