David Cramer's Blog

Python, Django, and Scale.

Presenting Django-devserver, a Better Runserver.

Today I’m going to talk (rant) about runserver, a great inclusion of the Django software package. It’s designed to help you easily run a Django project in a local (development) environment. Well, like many people, I have Rails in my background, and I miss certain things from it. One of those things, is the shiny SQL output in your terminal while running Mongrel.

Since I don’t have enough open source projects to maintain, I decided today that I needed this. It turned out to be extraordinarily easy even. I just ripped out my code from django-debug-toolbar (SQL and Cache tracking), flipped up a generic framework, and voila, django-devserver was born.

Now let’s get down to it. django-devserver provides a simple drop-in runserver replacement. It allows you to run a command, python manage.py rundevserver, and to get some additional information. As of writing, that additional information includes real-time SQL logging (aka mass query spam in your terminal), and a summary of cache calls.

Let’s take a look at some of our sample output:

While the syntax coloring still has a lot to be desired, it’s definitely a nice usable output. The SQL module, at the moment, simply outputs the query pre-execution, and post-execution outputs the time taken. The cache module outputs some generic stats, such as total time taken, # of calls, and hits vs misses.

The beautiful thing about the solution is how extensible it came out, for example, our cache module is only a few lines of code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
from django.template.loader import render_to_string
from django.shortcuts import render_to_response
from django.utils import simplejson
from django.core.cache import cache

from devserver.modules import DevServerModule

class CacheSummaryModule(DevServerModule):
"""
Outputs a summary of cache events once a response is ready.
"""

logger_name = 'cache'

attrs_to_track = ['set', 'get', 'delete', 'add', 'get_many']

def process_init(self):
from devserver.utils.stats import track

# save our current attributes
self.old = dict((k, getattr(cache, k)) for k in self.attrs_to_track)

for k in self.attrs_to_track:
setattr(cache, k, track(getattr(cache, k), 'cache'))

def process_complete(self):
from devserver.utils.stats import stats

self.logger.info('total time %(time)s - %(calls)s calls; %(hits)s hits; %(misses)s misses' % dict(
calls = stats.get_total_calls('cache'),
time = stats.get_total_time('cache'),
hits = stats.get_total_hits('cache'),
misses = stats.get_total_misses_for_function('cache', cache.get) + stats.get_total_misses_for_function('cache', cache.get_many),
gets = stats.get_total_calls_for_function('cache', cache.get),
sets = stats.get_total_calls_for_function('cache', cache.set),
get_many = stats.get_total_calls_for_function('cache', cache.get_many),
deletes = stats.get_total_calls_for_function('cache', cache.delete),
#cache_calls_list = [(c['time'], c['func'].__name__, c['args'], c['kwargs'], simplejson.dumps(c['stack'])) for c in stats.get_calls('cache')],
))

<p># set our attributes back to their defaults
for k, v in self.old.iteritems():
setattr(cache, k, v)

The entire thing supports all of the typical middleware processing, as well as the two additional methods: process_init() and process_complete().

I’m hoping to pull all of my usable work on the django-debug-toolbar into the devserver, but right now the two main additions I have planned are an SQLSummaryModule, and a ProcessTimeModule.

Hope you all enjoy, and get busy forking over at GitHub.

Comments