Empty Thoughts

JSON Generic Serializer

Recently I needed to implement some Ajax functionality into a Django app I’m building. Although Django has full support for working with Ajax and the Ajax library of your choice, out of the box it doesn’t steer you in any particular direction in this regard. Because of this I had been a bit hesitant about attacking this particular problem, fearing I would end up losing days of productivity trying to get something going. Luckily my fear was unfounded. It turns out that it’s fairly easy to implement, and quite simple to get your head around it.

One of the things I really needed was an easy way to serialize objects of data as an Ajax response back to my client. I searched around and through a combination of various forum postings and IRC, I was able to come up with the following generic json serializer. Like Generic Views, it allows me to easily make requests without a bunch of extra view code for standard cases. The implementation that I ended up with looks like this:

from django.http import HttpResponse 

def serialize(queryset, root_name=None): 
    if not root_name: 
        root_name = queryset.model._meta.verbose_name_plural 
    return '{"total": %s, "%s": %s}' % (queryset.count(), root_name, json_encode(queryset))  

def json_generic_query_serialize(request, queryset, criteria=None, parameter='query', root_name=None): 
    if criteria:
        qs = queryset.filter(**{'%s' % criteria: request[parameter]})
    else:
        qs = queryset
    return HttpResponse(serialize(qs, root_name=root_name), mimetype='text/javascript')

The only item you’re not likely to have is the method json_encode by Wolfram Kriesing. Believe it or not I found this tidbit on a dpaste post and it was exactly what I needed. Later I found out that Wolfram blogged about this item on his website. I encourage you to read his reasons for writing this nice tidbit.

The reason the default json serializer (django.core.serialize) doesn’t work well for my purposes is because the default implementation is built around the idea that the objects that are serializable need to be deserializable as full objects as well. I’m using the as my Ajax library and YUI will not work well with the default json structure provided by Django.

Just in case the json_encode method gets removed from dpaste I’m reprinting it here. I want to stress again that this is the work of Wolfram Kriesing, not mine. I included this method in the same file as my serializer methods posted above.

import types
from django.db import models
from django.utils import simplejson as json
from django.core.serializers.json import DateTimeAwareJSONEncoder
from decimal import *

def json_encode(data):
    """ 
    The main issues with django's default json serializer is that properties that
    had been added to a object dynamically are being ignored (and it also has 
    problems with some models).
    """ 

    def _any(data):
        ret = None
        if type(data) is types.ListType:
            ret = _list(data)
        elif type(data) is types.DictType:
            ret = _dict(data)
        elif isinstance(data, Decimal):
            # json.dumps() cant handle Decimal
            ret = str(data)
        elif isinstance(data, models.query.QuerySet):
            # Actually its the same as a list ...
            ret = _list(data)
        elif isinstance(data, models.Model):
            ret = _model(data)
        else:
            ret = data
        return ret

    def _model(data):
        ret = {}
        # If we only have a model, we only want to encode the fields.
        for f in data._meta.fields:
            ret[f.attname] = _any(getattr(data, f.attname))
        # And additionally encode arbitrary properties that had been added.
        fields = dir(data.__class__) + ret.keys()
        add_ons = [k for k in dir(data) if k not in fields]
        for k in add_ons:
            ret[k] = _any(getattr(data, k))
        return ret

    def _list(data):
        ret = []
        for v in data:
            ret.append(_any(v))
        return ret

    def _dict(data):
        ret = {}
        for k,v in data.items():
            ret[k] = _any(v)
        return ret

    ret = _any(data)

    return json.dumps(ret, cls=DateTimeAwareJSONEncoder)

Once the generic serializer is in place, it becomes easy to use. We need an entry in our urls.py file that routes the Ajax request to our generic serializer. A typical implementation will look like this:

categories_dict = { 
    'queryset': Category.objects.all(), 
    'root_name': 'categories',
    'criteria': 'name__icontains',  
} 

urlpatterns += patterns('', 
    (r'json/categories/', json_generic_query_serialize, categories_dict), 
)

Notice how flexible this implementation is. We can pass any criteria we want for finding our items. If we wanted to simply return all items, we would only need the ‘queryset’ item.

So what does this look like in the Form definition in our Forms.py file? It depends on what we want to accomplish. As an example, I’m implementing an AutoComplete field from the Yahoo! User Interface Library. My implementation looks like this:

category = forms.CharField(max_length=255, 
                           label=u'Category', 
                           widget=AutoCompleteField(source='/blog/json/categories/', 
                           schema=["categories", "name", "id"], 
                           options={ 'animSpeed': 0.1, 'autoHighlight': True, 'forceSelection': True, 'typeAhead': True }))

In the above sample I’m using a custom AutoCompleteField widget that I created that gives me all of the auto complete goodness that I need. I’ll write more about the implementation of that custom widget field in another blog post.

As you can see, solving the problem of easily getting serialized data in json format makes the whole Ajax process much simpler. Although Ruby on Rails bakes this functionality into the framework at times it becomes a two edged sword. It sure makes it easy for new users to start using Ajax immediately with little or no knowledge, but it also makes it quite painful to use any Ajax library that has not been “anointed.” I have no person preference of one implementation over the other, it’s just important to recognize the advantages and disadvantages of differing approaches.

---