Categorizing Packages: Clarifying Terms

tl;dr

Some of the friction which comes up on the zope-dev list, apparently due to different goals, might in fact be due to some confusion in the terms we use to talk about our goals, and about the shared software we manage toward those goals. In the interests of reducing the friction, I would like to sketch out how I am using those terms.

Disclaimers

First, let me note that I am elaborating on concepts I outlined a year ago in the weeks before the Plone Strategic Planning Summit. With that in mind, here are the terms as I mean to be using them.

Second, I'm sure that the examples listed under each section below are woefully incomplete: I'm mostly picking the packages I know well, and don't mean to be making value judgements about any I haven't listed.

Library

A reusable Python module or package which supplies functionality to other components. The normal mode for using a library is via a Python 'import' statement.

Examples:

Framework

A component or package which provides services on one side (like a library), and defines slots / plug-in interfaces on the other which are intended to be filled with objects supplying application-specific implementations or policies. The key concept of a framework is that it is both rigid and configurable: it defines some parts of its functionality as "unchanging", (e.g., the order in which it processes inputs and calls into its plugins), with other points where plugins supply the remainder of the functionality.

Note that some frameworks are designed to be usable (as libraries) even in the absence of explicitly configured plugins: in these cases, the framework supplies an "overridable default" implementation of the plugin API.

Examples:

Plugins

Plugins are components / packages which implement the API defined by a given framework.

Examples:

Configuration

Having frameworks and plugins requires some way to spell which plugins get configured into the slots provided by the framework. This kind of configuration is distinct from the kind which spells out declarative policy choices (e.g., port numbers, database connection strings, etc.), but this distinction can get muddied by the way various frameworks "spell" their configuration.

For example: in Zope-based applications, while the zope.component machinery is configured through calls to registerAdapter, registerUtility, etc., most people think of that configuration as being done via ZCML, as implemented by the XML dialect of zope.configuration. That dialect itself is both a mini-application (the interpreter which maps XML elements onto the corresponding 'register*' actions) and it is a framework: it is designed to be extended by plugins defining new handlers.

Examples:

  • ZCML files

  • Paste configuration files

  • Buildout configuration files

  • Django's middleware.py

Stack

A "stack" is a set of software components, divided into separate "layers". Each layer functions consumes APIs / services defined in the layers beneath it, and provides APIs to the layers above it. Stacks are often illustrated as "layer cakes," although there is often more than one "underlying" layer used by a single layer.

Managing a stack involves keeping the layering "clean": there should be no dependency cycles or uses of undocumented APIs between the layers. The "higher" layers tend to be more like "applications", including specific policies, while the lower layers tend to be more like "libraries."

Examples:

  • Plone -> {CMF | Archetypes | ...} -> Zope2 -> Zope3 libraries -> Python

  • Grok -> Zope3 libraries -> Python

Platforms

The term "platform" indicates a software configuration which forms the basis for building applications. Platforms are often organized as "stacks", at least internally.

Examples:

"Mega-framework"

This term, originated by the TurboGears folks, originally meant to signal that TurboGears was designed as a loosely-collaborating arrangement of "best-of-breed" frameworks (e.g., one for templating, one for request-response processing, one for RDBMS connectivity, etc.): it was meant to be used to differentiate TurboGears from "monolithic" platforms such as Django and Zope. We should probably avoid using it to refer to our own platforms.

"Pluggable Applications"

The term, "pluggable application", refers to a kind of platform which is organized as a web application, designed to "pluggable" in the sense of a framework.

Examples:

  • Zope2 ZMI

  • Zope3 ZMI

  • Grok admin UI (a somewhat reduced case)

Applications

Applications are normally deployed as websites in our world (they have URLs). They are normally built on a given "web framework ("stack" / "platform"), according to one a couple of patterns:

  • One common mode of building an application is to mix application- specific glue code with one or more components designed as "plugins" to the platform, along with various declarative configurations / policies. In a traditional Zope2 application, the glue code is defined TTW, via external methods / templates / PythonScripts, with the plugins supplied via Zope2 "products", and configuration stored in persistent objects ("tools" and / or "properties").

    Because managing TTW glue code is hard, we often migrate it down to filesystem-based "productcs", somewhat muddying the distinction.

  • The model of framework as "pluggable application" is not universal among the various web frameworks: some platforms aren't really designed for pluggability, preferring instead to generate code for the application.

    These frameworks tend to trade away the notion of running multiple applications, or instances of applications, within a given process, gaining instead some conceptual clarity: the application is in the genrated code, which uses the services supplied by the framework in library-like fashion. Any "configuration" of the framework tends to be done in Python code.

Products

Products are applications which are noremally installed and run in their "out-of-the box" form, with minimal configuration and without requiring any customization / glue code.

The "acid test" for determining whether a bit of code is a "product" or a "platform" is whether anybody who is not the developer deploys it in production without customizing it through code. It seems interesting that none of the Python-based web applications widely-installed in this fashion use one of the frameworks we have been discussing: I'm not sure what that means, unless it is that using one of the "frameworks" is a predictor of "one-off" applications.

Examples:

  • Trac

  • Roundup

  • Moinmoin

  • Venus