We’ve created many web sites at The Washington Times, many were just speculative, and most are not around today. In order to prototype a web site quickly, we developed a set of habits that make it easier. I call it “app-centric Django development”.

Projects and Apps

Django projects and apps are codependent. As a result, most developers write monolithic projects that contain all their apps. It’s easy to do and to understand, but it is not easy for reusing your code.

I had a 500-in-1 Electronics set from Radio Shack when I was a kid. I loved it because I could easily create a new device or test different modifications easily. That’s what a Django project is, to me: tt is a place for wiring all the components together. Apps are the individual components; the diodes, transistors, integrated circuits and light bulbs.

Our prototyping framework, Calloway, is the direct result of this methodology. The project consists of templates, settings and configuring how different applications work together, and the apps are completely separate. A project may have a few very specific apps that just don’t make sense outside of the web site’s scope, but the vast majority of the apps are installed using pip.

Isn’t it more work?

Not really, if you do it right. The key is having a good boilerplate application skeleton. I have one on github that uses a script to build the custom application with an example project for testing, setup.py script for packaging and boilerplate documentation.

Benefits

Modular code

By isolating the application from everything, you reveal the necessary and optional dependencies and will also try to minimize the dependencies. What you end up with is an application you can insert into a variety of projects cleanly.

Isolation for testing

I had an issue where a standalone application seemed to be buggy as it didn’t work correctly in a project. The tests ran fine in the standalone environment. It made it easy to determine that it was the forked version of one of the dependencies that was the culprit.

Easy deployment and retraction of code updates

Things like deploying code updates and rolling back a flawed update are easy if you are a solo developer. Once you have several people, with several sites, on several servers, things get a bit more … complicated.

Back in the days when we had a monolithic project and all the apps where in it, deployment of code was carefully scrutinized and more than once a stray, unrelated commit screwed up the deployment.

Now, we simply use pip (and fabric) to install a specific version of a specific app. If it has unforeseen consequences, we simply use pip to install the previous version. The fact that another developer is working on a different feature on a different app doesn’t affect the process.

Rapid prototyping of new projects

When someone in your company, or your team, or one of the voices in your head comes up with the “Next Big Thing,” you want it fail (or succeed) fast. The quicker you can see a working prototype, the quicker you can decide if this idea has legs. By keeping your code in configurable, standalone applications, prototyping a new idea will take less time and effort.

Your apps get better with use

When we redesigned our open source blog, we used the blogging engine that we created and use in two other sites. Unfortunately the application was created to handle multiple blogs for multiple users and insert the name of the blog in the URLs. We wanted just one blog, and why put it in the URL?

We modified our existing application and added a configuration to allow a default blog. That setting affects the URLs and url configurations, but the default is for multiple blogs. Our blogging engine is now better than it was. If we find a bug on one site, the fix is deployed on all sites with the application.

Drawbacks

Nothing is all sunshine and roses. What problems exist in this process?

Increased complexity

Your projects aren’t just a checkout or export from a repository anymore. Getting code on a machine, even a development machine takes a bit more thought.

What we do. A requirements file with specific version numbers is a must. Then a developer can create a virtualenv and use pip to install all the requirements.

Application version handling

As your project progresses, the versions of each application used will change. Keeping the requirements file up-to-date can be a problem.

What we do. We haven’t completely conquered this, but we’re making process (and we don’t do many virgin deploys). To help keep our requirements file up-to-date, the fabric script that installs a package on our servers (a new package or a different version) also updates the requirements file on the developer’s machine. They just need to remember to check it into the repository.

We also use a script that is part of Calloway called “check_for_updates” that will review every application installed (via pip freeze) against multiple PyPI repositories and tell you which packages have newer versions (and which PyPI they’re on)

Incorrectly packaged apps

Packaging in python isn’t easy, and many (most) Django applications are simply a repository on github or bitbucket. Others are incorrectly packaged, leading to a failed install.

What we do. We have our own PyPI where our forked, repackaged or private applications reside. We have a forked version of the ChiShop that implements the XML-RPC protocol. We customize the version number of the package to keep it separate.

A bugfix isn’t a simple commit

Each bug fix in an application is more than a commit; it is also a repackaging of the application. With an immature app, this can get a bit tiresome.

What we do. Even though it is a bit more work to fix a simple bug in an application, the benefits make it worthwhile (see above).

Summary

I obviously feel that the benefits outweigh the drawbacks in using this methodology. It is a new way of thinking and I’m sure with time, handling the few drawbacks will become easier.