Since it comes up a lot, I thought I’d spend a bit of time writing up my thoughts on what django.contrib really is, and what including a package in it really means.
The following is just my personal opinion — really; that’s why this is posted here instead of over in the official Django documentation. However, most of the core team discussed this topic at length at DjangoCon, so I’m fairly sure there’s consensus over the rough outline.
In a nutshell, django.contrib contains optional, de facto standard implementations of common patterns. That is:
Optional: contrib packages should be removable. You should be able to rm -rf django/contrib and Django shouldn’t break.
Astute readers will note that django.contrib.contenttypes violates this rule. This should be considered a bug, I think.
Along these lines, django.contrib packages ideally shouldn’t get “special access” to brittle internals — anything a contrib package does ought to be possible in an external app. Again, astute readers will notice that we’ve broken these rules in a few places, and, again, that should be considered a bug.
De facto standard: anything in contrib needs to be generally accepted as the Right Way to do something for a large majority of users.
django.contrib.sessions is one example of this. There are may ways of doing sessions, but the way Django does it — an opaque session key in the cookie, with all the data stored on the backend — is generally accepted as a best practice. Yes, there are other ways of implementing sessions — signed cookies comes to mind — but the approach used by django.contrib.sessions generally works for most users.
Common patterns: contrib packages should solve problems encountered frequently by real-world web developers. So I’d argue against including something like django-reversion. It’s incredibly cool, and I use it myself, but tracking of versioned data at such granularity is not needed by a large majority of web sites.
Good contrib packages tackle issues that that are not trivial, are bikeshed prone, and are difficult to get right for the common case. We want to prevent folks from needing to decide among seventeen different session frameworks, for example.
Hence django.contrib.gis: it’s not exactly a common pattern exactly, but if you’re dealing with geographic data, building on existing GIS tools like PostGIS and GDAL is both the common pattern and the best practice. So you could say that django.contrib.gis implements a common pattern for a smaller class of users, I suppose. More importantly, though, it’s a tricky thing to get right, so there’s a great value in having it Just Work.
Of course, there’s a marketing value of having something in contrib. It certainly helps PR to saying that “Django ships with GIS support.” It’s fashionable to look down on PR in the open source world, but it’d be disingenuous to claim that I don’t use the power of contrib as a marketing tool. I certainly do, and I have no qualms about doing so.
However, there’s a danger in bringing something into the core: it stifles future innovation. As soon as we “bless” a contrib package, we drastically reduce impetus to write competing libraries. So, a good contrib package should have general consensus, and should be fairly mature. This is why a schema evolution library won’t be in Django 1.1 despite at least three or four viable alternatives. None of these options are clearly better, and the entire space is evolving fairly quickly. We should wait to “bless” one library until we’re sure we’ve got the right one.
Another downside to contrib inclusion is the coupling of the package’s features to Django’s release cycle. A small package like django-tagging can add features quickly; limiting releases to every six-nine months would retard its spread as users waited for new versions of Django to be able to upgrade.
Including a package in contrib is also a promise of future support. Nothing would be worse than having to remove a contrib package in a future release — we want to add features, not remove ‘em! However, adding new contrib packages means more work for committers, so a new contrib package really needs to come along with a new contributor. A group of them, ideally — a contrib package with a bus factor of one isn’t a particularly good idea.
Comments:
Personally I don't get the argument that something being in contrib kills innovation. People are often quick to cite the example of comments being an application that was sub-par, however comments is also the perfect example of something being in contrib *not* stifling innovation. Despite comments blessed status Eric Florenzano still wrote threaded-comments which was still widely used, and not just because it added threading. Ultimately I think the impetus for people is to have an application that does what they need, if a give contrib application doesn't do what they need they will write their own, writing their own isn't *more* work because someone wrote something else.
Good points all in this post. Given your point on being able to rm -rf * in contrib, why is contenttypes not simply part of Django core, since it is so tightly integrated?
I'm working on a very large Django project right now with a similar issue (on a smaller scale), and one I think all large Django applications run into eventually (not a bad issue to have) - you have custom apps specific to your project (in the case Django proper) and then you have reusable apps (in this case Django-Tagging works). Do you fork the reusable app and wire them into the project or do you let them live and hopefully thrive externally on their own.
The strongest argument I see for something being in Django contrib is that there's a guarantee it will be supported. You don't have that same guarantee with a reusable app you find on Google Code or GitHub.
So what's the strongest argument against placing something in django.contrib? I think there are two: 1) Developer feels obligated/safer(?) to use the contrib application even though it may not fit their needs - the original django.comments for example (great app but definitely didn't cover all the bases) even though you saw it on almost everyone's Django blog in the book you know you were missing that URL/Website field every time you filled out a comment form :) And that's really a mistake of the developer but I have a feeling it happens more than we'd like to think. 2) Coupling to Django releases. As long as the reusable app runs on the current version (usually a stable version) of Django you won't hear a client saying is django-xyz on it's version 2.0 ... the question/demand is the version of Django. As long as the contrib/reusable app runs on that particular version of Django then all is well in the world.
I believe innovation just happens, regardless of something given a "blessing". Innovators will innovate. So I agree with Alex on his point.
As far as GeoDjango goes, even though it's not in contrib I know from my own recent experiences that GeoDjango, the fact that it exists, is a great marketing tool for Django overall. So having it in contrib isn't the "sell" ... the "sell" is that a) it exists with an incredibly smart and active community b) brands like the NYT are employing it and talking about it publicly.
This all being said, I'm not anxious to know what *does* make the next cut into django.contrib!
Excellent post Jacob. You covered all the concerns and issue and did so in an objective way. I can't really disagree with any of it. I'm glad to see that Django Core's are really thinking this kind of stuff through.
Thanks Jacob for this wonderful post. @Alex_Gaynor, @Kevin_Fricovsky: If a third-party app is included in contrib it needs to be backward-compatible atleast till the next major release of django whereas if it was independent, it could have developed rapidly and broken backward-compatibility with (or without) public consensus. At the same time, inclusion in contrib gives an assurance to people that it will be supported for a long time. My personal opinion: I wouldn't be scared even if a third-party app got abandoned because I have the source to make the changes but I can't live without those cool features that would have not have gotten in otherwise. Also having lesser contrib apps, gives Django core developers more time to improve the core framework which is what we all want!!! If people still believe in the contrib inclusion, why not have something on the lines of stable and unstable repos in Debian. What say?
Thejaswi if you look at the backwards compatibility docs they actually say that stuff in contrib doesn't have the same rules about it as the rest of Django does.
http://docs.djangoproject.c... mentions supporting two versions of django if there is a backward incompatibility change in a contrib app. God forbid such a change...ever!!!
It’s fashionable to look down on PR in the open source world, but it’d be disingenuous to claim that I don’t use the power of contrib as a marketing tool. I certainly do, and I have no qualms about doing so....
I completely agree, marketing is a very important factor in django's success, however one of my colleagues (who's wife is a doctor) just pointed out that in the medical world PR == per rectum.
Made me laugh :-)
Well, actually no.
"django.contrib contains optional [...]" is not true. They aren't optional after a while.
I mean something like django.contrib.auth. Whenever you need to reference a specific user, you need to choose what to ForeignKey to. And people choose django.contrib.auth.models.User, making any other authentication app more or less useless.
I think this actually is a disease plaguing Django, I often bump into issues like the one above. :-(
Oh, and your Preview button seems to actually post comments. (If this comment gets through, it does. In Safari 4.)
Leave a comment: