Logging

Logging plays an essential part of software applications. With well-written logs, it’s easy to retrieve event history, and speed up troubleshooting.

Most devs tend to keep logs in the source code directory, which is fine if they are set to rotate off eventually. Otherwise, the source code directory keeps growing with the application and eventually it may fill up the disk quota.

Another popular option is to send logs to a log server (Splunk, ELK, Graylog etc). But it shouldn’t fully replace the local logs in the filesystem. If the log services are down, you might still want the application to live. You can send the logs via UDP (i.e. it does not care whether the log has reached the log server, or not), or, you can send the local logs with Filebeats.

The combo of both strategies have worked out well for me. i.e. the local filesystem (where the application is) keeps the recent logs. The log server retains more historical logs.

Below is an example of Django settings to log locally, and to Graylog (via GELF UDP).

LOGGING = {
  'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
            'formatter': 'verbose',
        },
        'timedRotatingFileHandler': {
            'class': 'logging.handlers.TimedRotatingFileHandler',
            'formatter': 'verbose',
            'filename': os.path.join(BASE_DIR, "logs", "myapp.log"),
            'when': 'd',
            'interval' : 1,
            'backupCount': 5
        },
        'gelfUDAPHandler': {
            'class': 'graypy.GELFUDAPHandler',
            'formatter': 'simple',
            'host': 'mygraylogserver',
            'port': <the port assigned>
        },
    'formatters': {
        'verbose': {
            'format': '{asctime} {levelname} {pathname}:{funcName}[line:{lineno}] [{process:d}:{thread:d}] - {message}',
            'style': '{',
        },
        'simple': {
            'format': '{message}',
            'style': '{',
        },
    },
   'loggers': {
        '': {
            'handlers': ['console', 'timedRotatingFileHandler', 'gelfUDPHandler'],
            'level': os.getenv('DJANGO_LOG_LEVEL', 'DEBUG'),
            'propagate': False,
        },
    },
}

This chunk means the local logs are rotated every 1 day, with 5 backup copies.

'when': 'd',
'interval' : 1,
'backupCount': 5

Refer to

logging.handlers - Logging handlers - Python 3.10.0 documentation