Dynamic form generation
I had the pleasure of being on a forms panel at PyCon 2010 chaired by Brandon Craig Rhodes. To get a stable baseline, Brandon asked each of us to provide code showing how each forms toolkit might tackle a problem:
Imagine that someone has already written a form with your forms library. The form looks something like this:
New username: __________ Password: __________ Repeat password: __________ [Submit]
Now, someone from Marketing comes along and announces that the developers must add some additional questions to the form - and the number of extra questions is not determined until runtime! They give you a
get_questions(request)
function that looks up a profile they cook up for each person browsing the site, and returns a list of strings like one of these:['Where did you hear about this site?'] ['What college did you attend?', 'What year did you graduate?'] ['What is the velocity of a swallow?', 'African or European?']
They explain that they cannot limit how many questions might be returned; for simple users who are bumpkins, it might just be one or two, but could be many if the user sounds like a very interesting one. Your form will, then, look something like this (in the case of the third datum above):
New username: __________ Password: __________ Repeat password: __________ What is the velocity of a swallow? _________ African or European? _________ [Submit]
When you get the answers, you can save each one of them simply by calling save_answer(request, question_text, answer_text).
But each question must be answered; if the user fails to fill in one of the questions, then the form should be re-displayed with all of the data in place but with a note next to each un-answered question noting that it is a required field. (Yes: you can assume that
get_questions(request)
for a particular session returns the same list of questions over and over again, so that’s not a value you have to stash away to make the form appear consistent from one page load to the next!)Your form, then, will have to go from being static, and driven by a simple schema, to having a series of fields that are variable, and driven by runtime logic, rather than static and driven by a schema that the programmer can enter as a constant.
How can your forms library best be used to present and validate the above form?"
I had fun with the problem, so here’s my writeup on how Django would solve this problem.
Before we start, let’s talk – real briefly – about the parts you need to display and process a form in Django. You’ll need:
- A form class. This’ll be a subclass of
django.forms.Form
, and will contain all the display and validation logic for the form.- A view function. This’ll be responsible for connecting the form to the submitted data, rendering the form into a template, etc.
- (Usually) a template used to render the form and the rest of the page.
Part 1
The first part of this assignment – username / password / repeat password – is very easily solved: just install James Bennett’s django-registration and go out for beer.
Snark aside, this illustrates one of the best aspects of Django: there’s an incredibly active community producing reusable apps, so when you’re faced with a problem in Django there’s a good chance that “there’s an app for that.” In the real world, in fact, I’d use django-registration for this entire round.
However, to keep things simple, I’ll stick to what’s built into Django.
The form
Very simple:
from django.contrib.auth.forms import UserCreationForm
That’s right: Django ships with a form that handles this common user/password/confirm task perfectly. For pedagogical purposes, though, I’ll show how you’d actually define this form by hand if you cared to:
from django import forms
class UserCreationForm(forms.Form):
username = forms.CharField(max_length=30)
password1 = forms.CharField(widget=forms.PasswordInput)
password2 = forms.CharField(widget=forms.PasswordInput)
def clean_password2(self):
password1 = self.cleaned_data.get("password1", "")
password2 = self.cleaned_data["password2"]
if password1 != password2:
raise forms.ValidationError("The two password fields didn't match."))
return password2
The salient features here:
The form is a subclass of
forms.Form
.A form includes a number of fields. Django’s got a bunch of built-in fields for common types (characters, dates/times, file uploads, etc.), and it’s easy to create custom field types.
Each field has an associated “widget” which defines how to render the form field. Django’s HTML-focused, and as such ships with built-in widgets for all the HTML input types. Again, it’s easy to write your own widgets, and since there’s nothing HTML-specific until you reach the widget layer it’d be easy to use the forms framework on top of something else like XForms.
Here, all the fields use
CharField
, but the password fields get rendered as<input type=password>
by using aPasswordInput
.Fields generally know how to validate and clean (convert from strings to proper types) themselves. So an
IntegerField
will automatically coerece strings on the wire into integers.Here, we need some extra validation – we want to check that
password2
matchespassword1
. For the purpose, we can define a custom clean method. We provide a clean method by defining a method on the form calledclean_<fieldname>
.By the time our
clean_password
method gets called, the fields' clean methods have already been called, which has put cleaned and validated data intoself.cleaned_data
.However, we need to guard against
password1
not having been provided: fields are required by default, butcleaned_data
only contains data that’s passed validation, so ifpassword1
wasn’t given, it won’t be incleaned_data
. For similar reasons,clean_password2
will only be called if something’s been entered in the field.Clean functions do two things:
- Validate data, raising
forms.ValidationError
if the data’s not valid.- Clean – coerce, modify, or otherwise munge – the data.
They’ll pull the old data out of
self.cleaned_data
, and return the new, cleaned and validated data.There are also various hooks for form-wide validation which aren’t needed here.
The view
Now that we’ve got a form, we need a view function to present that form. Views and how they’re wired up are out of scope for this discussion, so I won’t cover them in detail. Instead, I’ll just look at how they apply to form processing.
So here’s our view:
from django.shortcuts import redirect, render_to_response
from myapp.forms import UserCreationForm
def create_user(request):
if request.method == 'POST':
form = UserCreationForm(request.POST)
if form.is_valid():
do_something_with(form.cleaned_data)
return redirect("create_user_success")
else:
form = UserCreationForm()
return render_to_response("signup/form.html", {'form': form})
Salient details:
To be a good HTTP citizen, we only process the form if the request was submitted with
POST
.If the request was a
POST
, we’ll create a bound form – a form bound to some associated data – by passing inrequest.POST
.
request.POST
is essentially adict
, and forms will operate on anydict
-like structure. I mention this to point out that though forms are usually used with HTML/HTTP, they can also be used with lots of other data sources.For example, Piston, a REST API framework, uses Django’s forms to validate data submitted via an API in JSON, YAML, XML, etc.
Calling
form.is_valid()
kicks off all the data cleaning and validation.If it returns
True
then all the validation succeeded. At this point we can do something with our submitted data – usually by readingform.cleaned_data
.Again, to be a good Web citizen, we’ll follow the common Post/Redirect/Get pattern upon success.
If the form wasn’t valid, we’ll fall through returning a response. The form instance, however, will have an
errors
attribute containing all the errors that prevented it from being valid.If the request wasn’t a
POST
, we’ll construct an unbound form. This form will have no associated data, and hence will have no errors.Finally, we’ll render a template –
signup/form.html
– and pass in the form instance to be rendered.
There’s actually a slightly shorter way (one that’s rapidly becoming a Django idiom) to write this view:
def create_user(request):
form = UserCreationForm(request.POST or None)
if form.is_valid():
do_something_with(form.cleaned_data)
return redirect("create_user_success")
return render_to_response("signup/form.html", {'form': form})
Exactly how and why this works is left as an exercise to the reader.
Hint: recall that request.POST
will only have POST
-ed data in it…
The template
Again, templates are out of scope, so I’ll just look at how they’d touch on form rendering.
At its simplest, a the signup/form.html
template would be very simple
indeed:
<form method="POST" action=".">
{{ form }}
<input type=submit>
</form>
Forms know how to render themselves into HTML with relatively simple markup. This does quite a lot, though: it’ll render the form, filling in any pre-existing data if the form was bound, and will also render associated error messages.
Notice that the form doesn’t render the surrounding <form>
tag, nor
does it render the <ipnut type=submit>
. This is by design: you could,
for example, compose multiple forms into a single <form>
, or you could
be rendering this form into something that’s not HTML.
There are a couple of other shortcuts to rendering forms in a few common
ways: {{ form.as_p }}
, {{ form.as_table }}
, {{ form.as_ul }}
. For
maximum control, you can render each field separately, or even write the
form completely by hand. For details, consult the form
documentation
Part 2
Okay, let’s kick it up a notch. The second part of this problem involves adding custom registration questions for each person. We’ll need to integrate these custom question into the form.
Note that the template wouldn’t need to be changed at all, so we’ll skip it entirely in this section.
The view
We’ll actually start by looking at the view, since it’ll need a couple of minor tweaks to handle the new form:
def create_user(request):
extra_questions = get_questions(request)
form = UserCreationForm(request.POST or None, extra=extra_questions)
if form.is_valid():
for (question, answer) in form.extra_answers():
save_answer(request, question, answer)
return redirect("create_user_success")
return render_to_response("signup/form.html", {'form': form})
You’ll notice that the general structure of the form processing is intact. This is typical: form views almost always follow this idiomatic style. So what’s different here?
- We’re gathering a list of extra questions from this hypothetical
get_questions(request)
function, and then feeding them into the form class.- If the form is valid, we’re calling
form.extra_answers()
and using the returned data to save those answers.
I’ll look at how to make the associated changes to the form in a second, but first a brief philosophical digression. In essence, we’ve pushed the handling of the questions and the validation of their answers down into the form class itself, but we’ve left the parts of the problem that deal with the request object in the view. This is the idiomatic way of tackling the problem: if we handed the request object to the form directly we’d be cutting against the grain of Django’s form library, which doesn’t want to be tied to HTTP directly.
The form
After writing the view, it’s clear we need to modify our form in a couple of ways:
- We’ll need to change the
__init__
signature to accept this list of extra questions, and we’ll need to modify the form’s fields accordingly.- We’ll need to add an extra
extra_answers()
function that returns(question, answer)
pairs.
Let’s look at __init__
first:
class UserCreationForm(forms.Form):
username = forms.CharField(max_length=30)
password1 = forms.CharField(widget=forms.PasswordInput)
password2 = forms.CharField(widget=forms.PasswordInput)
def __init__(self, *args, **kwargs):
extra = kwargs.pop('extra')
super(UserCreationForm, self).__init__(*args, **kwargs)
for i, question in enumerate(extra):
self.fields['custom_%s' % i] = forms.CharField(label=question)
So what’d we do here?
- We pulled the
extra
keyword argument out of**kwargs
and then called the superclass initializer. In practice, we’d probably want some error checking to make sureextra
was given.- We then looped over the extra questions and added them to
self.fields
.self.fields
is a dict mapping field identifiers – input names, essentially – toField
instances. Normally, we define these declaratively (as we still do forusername
,password1
, andpassword2
). Since we’re dealing with dynamic data here, though we can’t define them declaratively, so we can simple add new fields toself.fields
.- We’re arbitrarily naming the fields
custom_0
,custom_1
, etc. since we’ve been informed thatget_questions
is stable. If it wasn’t, we might need to do something like hash the question text.- By default, the label for the field will be the same as the field’s identifier. Since “Custom 1: _____” doesn’t make for very good UX, we’ve passed the question text itself in as the field’s
label
argument, which will render by default as an associated<label>
tag.- Fields are required by default, so we fulfill that part of the assignment without any custom validation. If they were to be optional we could pass
required=False
in to the field’s constructor.
That was the tricky part; the extra_answers()
function is easy:
class UserCreationForm(forms.Form):
...
def extra_answers(self):
for name, value in self.cleaned_data.items():
if name.startswith('custom_'):
yield (self.fields[name].label, value)
We simply loop over self.cleaned_data
, yielding any pairs for fields
starting with custom_
. The only slightly tricky bit is that we need to
pull the original question back out of the label
value by accessing
self.fields[name].label
.
Thanks for the challenge, Brandon; it was fun!