Derrick Petzold

Open source enthusiast and developer


Django request logging and json

Views: 6,689 Comments: 0

Posted January 15, 2012 by derrick


I wrote a decorator to make the request available to logging by using a LoggerAdpater and RequestInfo make the request object dict-like.

class RequestInfo(object):

    def __init__(self, request):
        self.request = request

    def __getitem__(self, name):

        if name == 'request.host':
            return socket.gethostname()

        if name.startswith('request.meta.'):
            val = name.split('.')[2]
            try:
                return self.request.META[val.upper()]
            except KeyError as e:
                return None
        return eval('self.%s' % (name))

    def _get_attrs(self, obj):
        attrs = []
        for attr in dir(obj):
            try:
                if not attr.startswith('_') and \
                        not callable(getattr(obj, attr)):
                    attrs.append(attr)
            except AttributeError:
                pass
        return attrs

    def __iter__(self):
        keys = ['request.host']
        keys.extend(['request.%s' % (a) for a in
                self._get_attrs(self.request)])
        keys.extend(['request.session.%s' % (a) for a in
            self._get_attrs(self.request.session)])
        keys.extend(['request.user.%s' % (a) for a in
            self._get_attrs(self.request.user)])
        keys.extend(['request.meta.%s' % (a.lower()) for a in
            self.request.META.keys()])
        return keys.__iter__()

def logger(name):
    def wrap(func):
        def caller(*args, **kwargs):
            request = None
            for arg in args:
                if isinstance(arg, HttpRequest):
                    request = arg
            if 'logger' not in kwargs:
                if request is not None:
                    kwargs['logger'] = logging.LoggerAdapter(
                            logging.getLogger(name), RequestInfo(request))
                else:
                    kwargs['logger'] = logging.getLogger(name)
            return func(*args, **kwargs)
        return caller
    return wrap
its meant to be used with the views like this
@log.logger(__name__)
def home(request, logger=None):
    logger.info('user performed some action')
update your logging.yaml to use the new format. For more info on that please see my previous post.
request-json:
    (): log.CustomJsonFormatter
    format: "'created', 'request.host', 'module', 'funcName', 'lineno', 'levelname', 'request.meta.remote_addr',  'request.meta.http_user_agent', 'request.user.username', 'request.path', 'message'"
and the resulting log message should look something like this
{"host": "fastcgi22", "request.meta.http_user_agent": "Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.15 (KHTML, like Gecko) Ubuntu/11.04 Chromium/18.0.996.0 Chrome/18.0.996.0 Safari/535.15", "request.path": "/", "created": 1326706577.275584, "module": "log", "funcName": "caller", "request.user.username": "derrick", "lineno": 114, "request.meta.remote_addr": "76.93.216.138", "message": "user performed some action", "levelname": "INFO"}
Now django requests are fully integrated with logging. Happy Trails.
Download the source.

Python function runtime logging decorator

Views: 2,521 Comments: 0
Tags:

Posted July 15, 2010 by derrick


When profiling it can be useful to log the amount of time that is spent in a function. With Python that is super easy to do with decorators.

logtime.py

#!/usr/bin/python

import time
import syslog

def logtime(func):
    def caller(*args, **kwargs):
        stime = time.time()
        ret = func(*args, **kwargs)
        syslog.syslog(
            syslog.LOG_LOCAL2 | syslog.LOG_INFO,
            '%s=%s\n' % (func.__name__, time.time() - stime))
        return ret
    return caller

@logtime
def test_func(arg1, arg2=None):
    print arg1, arg2
    time.sleep(1)

if __name__ == '__main__':
    test_func(1, 2)
logtime will log the time spent in the function to syslog.
Jul 14 15:05:01 olomai python: test_func=1.00114893913
Download the source.

derrickpetzold.com

My home on the web. Uses the Django web framework, uwsgi as the WSGI server, nginx as the media server and load balancer, pygments for the syntax highlighting. Author and designer.

crowdtube.tv

crowdtube.tv
CrowdTube.tv streams trending videos to your browser like a never ending tv show. This was Cory Shaw's awesome idea. I was responsible for the backend development.

dmusic.bz

dMusic.bz is Pandora clone written using Django and JavaScript. It uses last.fm to find the music relationships. Author and designer.

ilovephotos.com

ilovephotos.com
ilovephotos.com is a photo sharing and tagging website. Facial detection was run on the photos so the bounding boxes were already drawn around the faces eliminating a step in a tedious process. I authored views and the process to run facial detection on the photos and upload them to S3.

kindfish.com

kindfish.com
Kindfish.com was BlueLava's first photo site. It was event based and slideshows would be created from the albums. I authored the views and process to generate the slideshows from the photos on EC2.

Below is a snippet of my resume. Click here to view the full version in pdf, its proper format.

captcha