zc.buildout vs. "plain" setuptools
A brief proposal on how and when one might prefer to use "standard" setuptools features, rather than relying on zc.bulidout.
Created by tseaver. Last modified 2008-04-23 20:32:58.
The Zope development community has largely adopted
zc.buildout as a
de facto standard for how it builds and maintains Zope-related
software. This post attempts to outline some reasons why
may not be a "sufficient" tool for all cases, and proposes some additional
practices which may help alleviate the issues described.
allows for creating a repeatable environment using one or more
"recipes" (which serve as plug-ins for the fraemwork) and a
configuration file, which defines policies for a given environment.
There are recipes available for a wide variety of tasks, ranging from
building and testing a single Python package, to building a Python
application using any number of such packages, to building arbitrary
applications from source (e.g., using "configure && make && make install").
After bootstrapping, a developer can ensure that her environment is
in a known state by running (or re-running) the
created by the bootstrap script. This command parses the
file, and uses it to build a number of "parts", each configured via
a section in the configuration file, and driven by a recipe.
As the environment grows more complex, the find-grained control
zc.buildout becomes much more helpful.
At the simpler end of the spectrum, however (building and testing a single, pure-Python package), the advantages are not so clear. In particular:
buildout.cfgfile is redundant to the equivalent information expressed in the
buildout.cfgare noticeable as "clutter" in the checkout / tarball, which provide no interesting benefit to developers who mean only to "consume" the package.
zc.buildoutmay be put off by its "foreign" paradigm, which violates their expectation that a package should be manageable using the standard commands of `distuilts' (a de jure standard) or
setuptools(a de facto standard).
For instance, runing the tests for a package in the
world involves merely unpacking the tarball and running 'setup.py
zc.buildout, the equivalent steps are unpacking,
running the bootstrap script, then runnning an arbitrarily-named
test runner script. Documenting this paradigm in each package is
extra effort for the maintainer, as well.
Rather than advocating that the Zope community abandon
for the "simple package" case, I would like to propose some practices
which mitigate some of the downsides.
Resist the temptation to exploit the additional expressiveness of
the configuration language provided by
zc.buildout when packaging
setup.pyas declarative as possible.
Avoid "tricky" things inside the script: use the "standard"
mechanisms as much as possible. For instance, use
pkg_resources.find_packages, rather than roll your own
In particular, move the various kinds dependency information out
into globals, if the
zc.buildout framework cannot introspect it
For instance, because
zc.buildout does not support the
tests_require keyword directly, we should perhaps adopt a
conventional name (e.g.,
TESTS_REQUIRE) at "module scope" in the
setup.py script, which contains the testing dependencies to be
zc.buildout could then use such a name to
configure its test runner scripts.
Before releasing the package to the wider Python world, ensure that
setuptools commands (
install, 'test') work with the package in an
environment which does not have
provides an easy means of creating such environments.
Unless your package has C extensions, you should plan to
distribute only the source distribution (the one created
setup.py sdist). This tarball will contain all the files
in your package, including tests and documentation, as opposed to
the various binary formats. The binary formats often have subtle
portability issues, as well some of which can't be resolved (e.g.,
32-bit vs. 64-bit, Python version, UCS2 vs. UCS4, etc.).
Windows developers should be able to install pure Python
distributions, as well, even without the compiler required to
build C extensions.
If your package does include C extensions, then you should plan,
if possible, to build and distribute the
bdist_win package, for
the benefit of Windows users who can't compile the extension.
You might also distribute a
bdist_egg built for Windows (some
users prefer the installer version, and others the egg). Generally,
it would be best to avoid building other binary distributions at all:
on any platform where Python or Zope is likely to run, the extension
should be trivial to build from the
 One issue with the
setup.py test command is that any
dependencies which are not already in the current
environment will be installed as eggs into the unpacked
tarball's top-level directory. These eggs will interfere
setup.py commands, if not removed.