In this post I will show you how to configure logging so your application servers can send info in nicely parseable json format to a centralized server for analysis.

So first things first. Lets begin by moving the logging out of settings.py and into it own config. Why because its easier to read and edit in yaml than in python/json and it shows up great in vim/emacs. So let's replace this:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'filters': {
        'require_debug_false': {
            '()': 'django.utils.log.RequireDebugFalse',
        }
    },
    'handlers': {
        'mail_admins': {
            'level': 'ERROR',
            'filters': ['require_debug_false'],
            'class': 'django.utils.log.AdminEmailHandler'
        }
    },
    'loggers': {
        'django.request': {
            'handlers': ['mail_admins'],
            'level': 'ERROR',
            'propagate': True,
        },
    }
}
with this
import yaml
SITE_ROOT = os.path.dirname(os.path.realpath(__file__))
LOGGING = yaml.load(open(SITE_ROOT + '/logging.yaml', 'r'))
and put this into logging.yaml
version: 1
disable_existing_loggers: True
        
filters:
    require_debug_false:
        (): django.utils.log.RequireDebugFalse

handlers: 
    mail_admins:
        level: ERROR
        filters: [require_debug_false]
        class: django.utils.log.AdminEmailHandler

loggers:
    django.request:
        handlers: [mail_admins]
        propagate: True
        level: ERROR
Yay. Now we have a logging config that's easy to read and edit. To make the logging as painless as possible use Madzak's excellent module python-json-logger. I went the distance and defined my own _fmt parser.
class CustomJsonFormatter(jsonlogger.JsonFormatter):
    def parse(self):
        return eval(self._fmt)
That allows for the format to be defined as python string. Append the json formatter, syslog handler and the app logger to their respective locations in logging.yaml.
formatters:
    (): log.CustomJsonFormatter
    json:
        format: "'created', 'module', 'funcName', 'lineno', 'levelname', 'message'"

handlers:
    syslog:
        level: DEBUG
        class: logging.handlers.SysLogHandler
        facility: local0
        formatter: json
        address: [loghost, 514]

loggers:
    app_name:
        handlers: [syslog]
        propagate: True
        level: DEBUG
Next on to your loghost. I use my db server but any server with space for the logs will do. We need to configure it to accept remote requests and enable the local0 facility. I don't know why the local? are not enabled by default but no matter. Ubuntu's default syslogd is rsyslog and if that is what you got put the following in /etc/rsyslog.d/local0.conf
$ModLoad imudp
$UDPServerRun 514
$template msg,"{'%msg%\n"
local0.*        /var/log/local0;msg
Notice the "{" in the template. That should certainly not be there but for some reason rsyslog strips some of the message with just logging the %msg% like that. Why I have no idea and I wasn't going to go to the source to find out (syslog-ng did something similar so I am really at a loss). Now be sure to create the file and restart rsyslog.
sudo touch /var/log/local0
sudo chown syslog.adm /var/log/local0
sudo restart rsyslog
logger -p local0.info test test

Okay now its time to test. Logging messages from your app server should now be going to the central log host but you would be the luckiest person alive if that all worked without a hitch. If it doesn't be sure to check firewall rules, security groups, stuff like that.