How can I log all SQL queries that my django application performed?

I want to log everything, including SQLs from admin site. I saw this question and a FAQ answer but I still can't figure out where should I put

from django.db import connection
connection.queries

to log everything to one file?

So my question is - what should I do to have a file (say all-sql.log) where all SQL statements are logged?

Solution 1

Merge the following snippet with the LOGGING field in your settings.py:

LOGGING = {
    'version': 1,
    'filters': {
        'require_debug_true': {
            '()': 'django.utils.log.RequireDebugTrue',
        }
    },
    'handlers': {
        'console': {
            'level': 'DEBUG',
            'filters': ['require_debug_true'],
            'class': 'logging.StreamHandler',
        }
    },
    'loggers': {
        'django.db.backends': {
            'level': 'DEBUG',
            'handlers': ['console'],
        }
    }
}

Tweaked from @acardenas89 answer

Solution 2

Add the following bold statements in settings.py


if DEBUG:
    import logging
    l = logging.getLogger('django.db.backends')
    l.setLevel(logging.DEBUG)
    l.addHandler(logging.StreamHandler())


LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'filters': {
        'require_debug_false': {
            '()': 'django.utils.log.RequireDebugFalse'
        }
    },
    'handlers': {
        'mail_admins': {
            'level': 'ERROR',
            'filters': ['require_debug_false'],
            'class': 'django.utils.log.AdminEmailHandler'
        },'console': {
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',
        },
    },
    'loggers': {
        'django.request': {
            'handlers': ['mail_admins'],
            'level': 'ERROR',
            'propagate': True,
        },'django.db.backends.sqlite3': {
            'level': 'DEBUG',
            'handlers': ['console'],
        },
    }
}
  

Resource/Credit

Solution 3

Maybe check out https://github.com/django-debug-toolbar/django-debug-toolbar

It'll let you see all the queries generated by a given page. As well as stacktraces of where they occur etc.

EDIT: to log all SQL queries to a file etc, then you will want to create some middleware. Middleware gets run on every request. There are several Django snippets out there for this sort of thing:

Those are concerned with printing to the terminal, but it wouldn't be hard to adapt them to use python's logging library.

Solution 4

To log SQL queries during testing, you need two things:

  1. django.db.backends logger enabled and
  2. @override_settings(DEBUG=True) decorator.

Test runner will set DEBUG=False by default, ignoring what you may have set in DJANGO_SETTINGS_MODULE.

The minimum settings:

# https://docs.djangoproject.com/en/dev/ref/settings/#logging
LOGGING = {
    'version': 1,
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
        },
    },
    'loggers': {
        'django.db.backends': {
            'level': 'DEBUG',
        },
    },
    'root': {
        'handlers': ['console'],
    }
}

The example test case:

from django.contrib.auth.models import User
from django.test import TestCase, override_settings


class UserTests(TestCase):

    # To log queries in tests you need to manually override DEBUG setting
    # because testing sets DEBUG=False by default

    @override_settings(DEBUG=True)
    def test_create_user(self):
        User.objects.create()

Solution 5

Django 1.3 logs all SQL statements to django.db.backends logger:

https://docs.djangoproject.com/en/dev/topics/logging/#django-db-backends

Solution 6

You only need:

@override_settings(DEBUG=True)

if you already have SQL debug statements being printed in runserver.

Add the decorator to your class TestA(TestCase) or test_function:

@override_settings(DEBUG=True)
class TestA(TestCase):
...

    @override_settings(DEBUG=True)
    def test_function(self):
    ...

Credits to @Janusz Skonieczny's answer!

Solution 7

You need to put this in a middleware package. The middleware sits between the webserver/django core and all your views. It can do preprocessing before the request, and postprocessing after the request completed. For example, save the queries to a file.