Django ====== The package ``dockerflow.django`` package implements various tools to support Django projects that want to follow the Dockerflow specs: - A Python logging formatter following the `mozlog`_ format to be used in the ``LOGGING`` setting. - A middleware to emit `request.summary`_ log records based on request specific data. - Views for health monitoring: - ``/__version__`` - Serves a ``version.json`` file - ``/__heartbeat__`` - Run Django checks as configured in the ``DOCKERFLOW_CHECKS`` setting - ``/__lbheartbeat__`` - Retuns a HTTP 200 response - Signals for passed and failed heartbeats. .. _`mozlog`: https://github.com/mozilla-services/Dockerflow/blob/main/docs/mozlog.md .. _`request.summary`: https://github.com/mozilla-services/Dockerflow/blob/main/docs/mozlog.md#application-request-summary-type-requestsummary .. seealso:: For more information see the :doc:`API documentation ` for the ``dockerflow.django`` module. Setup ----- To install ``python-dockerflow``'s Django support please follow these steps: #. Add ``dockerflow.django`` to your ``INSTALLED_APPS`` setting #. Define a ``BASE_DIR`` setting that is the root path of your Django project. This will be used to locate the ``version.json`` file that is generated by CircleCI or another process during deployment. .. seealso:: :ref:`django-versions` for more information #. Add the ``DockerflowMiddleware`` to your ``MIDDLEWARE_CLASSES`` or ``MIDDLEWARE`` setting:: MIDDLEWARE_CLASSES = ( # ... 'dockerflow.django.middleware.DockerflowMiddleware', # ... ) #. :ref:`Configure logging ` to use the :class:`~dockerflow.logging.JsonLogFormatter` logging formatter for the ``request.summary`` logger (you may have to extend your existing logging configuration!). .. _django-config: Configuration ------------- .. epigraph:: Accept its configuration through environment variables. There are several options to handle configuration values through environment variables, e.g. as shown in the `configuration grid`_ on djangopackages.com. ``os.environ`` ~~~~~~~~~~~~~~ The simplest is to use Python's ``os.environ`` object to access environment variables for settings and other variables, e.g.:: MY_SETTING = os.environ.get('DJANGO_MY_SETTING', 'default value') The downside of that is that it nicely works only for string based variables, since that's what ``os.environ`` returns. python-decouple ~~~~~~~~~~~~~~~ A good replacement is python-decouple_ as it's agnostic to the framework in use and offers casting the returned value to the type wanted, e.g.:: from decouple import config MY_SETTING = config('DJANGO_MY_SETTING', default='default value') DEBUG = config('DJANGO_DEBUG', default=False, cast=bool) As you can see the ``DEBUG`` setting would be populated from the ``DJANGO_DEBUG`` environment variable but also be cast as a boolean (while considering the string values ``'1'``, ``'yes'``, ``'true'`` and ``'on'`` as truthy values, and similar for falsey values). django-environ ~~~~~~~~~~~~~~ Django-environ_ follows similar patterns as python-decouple but implements specific casters for typical Django settings. E.g.:: import environ env = environ.Env() MY_SETTING = env.str('DJANGO_MY_SETTING', default='default value') DEBUG = env.bool('DJANGO_DEBUG', default=False) DATABASES = { 'default': env.db(), # automatically looks for DATABASE_URL } django-configurations ~~~~~~~~~~~~~~~~~~~~~ If you're interested in even more complex scenarios there are tools like django-configurations_ which allows loading different sets of settings depending on an additional environment variable ``DJANGO_CONFIGURATION`` to separate settings by environment (e.g. dev, stage, prod). It also ships with ``Value`` classes that implement configuration parsing from environment variable and casting, e.g.:: from configurations import Configuration, values class Dev(Configuration): SESSION_COOKIE_SECURE = False DEBUG = values.BooleanValue(default=False) class Prod(Dev): SESSION_COOKIE_SECURE = True In that example the configuration class that is given in the ``DJANGO_CONFIGURATION`` environment variable would be used as the base for Django's settings. .. _configuration grid: https://djangopackages.org/grids/g/configuration/ .. _python-decouple: https://pypi.python.org/pypi/python-decouple .. _Django-environ: https://django-environ.readthedocs.io/ .. _Django-configurations: https://django-configurations.readthedocs.io/ .. _django-serving: ``PORT`` -------- .. epigraph:: Listen on environment variable ``$PORT`` for HTTP requests. Depending on which WSGI server you are using to run your Python application there are different ways to accept the :envvar:`PORT` as the port to launch your application with. It's recommended to use port ``8000`` by default. Gunicorn ~~~~~~~~ Gunicorn automatically will bind to the hostname:port combination of ``0.0.0.0:$PORT`` if it find the :envvar:`PORT` environment variable. That means running gunicorn is as simple as using this:: gunicorn myproject.wsgi:application --workers 4 --access-logfile - .. seealso:: The `full gunicorn documentation `_ for more details. uWSGI ~~~~~ For uWSGI all you have to do is to bind on the :envvar:`PORT` when you define the ``uwsgi.ini``, e.g.: .. code-block:: ini [uwsgi] http-socket = :$(PORT) master = true processes = 4 module = myproject.wsgi:application chdir = /app enable-threads = True .. seealso:: The `full uWSGI documentation `_ for more details. .. _django-versions: Versions -------- .. epigraph:: Must have a JSON version object at /app/version.json. Dockerflow requires writing a `version object`_ to the file ``/app/version.json`` as seen from the docker container to be served under the URL path ``/__version__``. To facilitate this python-dockerflow contains a Django view to read the file under path ``BASE_DIR + 'version.json'`` where ``BASE_DIR`` is required to be defined in the Django project settings, e.g.: .. code-block:: python import os BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) Assuming that the ``settings.py`` file is contained in the project folder That means the ``BASE_DIR`` setting will be the one where the ``manage.py`` file is located in the below example directory tree: .. code-block:: text :emphasize-lines: 14,22 . ├── .dockerignore ├── .gitignore ├── Dockerfile ├── README.rst ├── circle.yml ├── manage.py ├── requirements.txt ├── staticfiles │   └── .. ├── tests │   └── .. ├── version.json ├── myproject │   ├── app1 │   │   ├── .. │   │   └── .. │   ├── app2 │   │   ├── .. │   │   └── .. │   ├── settings.py │   └── urls.py └── .. .. _version object: https://github.com/mozilla-services/Dockerflow/blob/main/docs/version_object.md .. _django-health: Health monitoring ----------------- Health monitoring happens via three different views following the Dockerflow_ spec: .. http:get:: /__version__ The view that serves the :ref:`version information `. **Example request**: .. sourcecode:: http GET /__version__ HTTP/1.1 Host: example.com **Example response**: .. sourcecode:: http HTTP/1.1 200 OK Vary: Accept-Encoding Content-Type: application/json { "commit": "52ce614fbf99540a1bf6228e36be6cef63b4d73b", "version": "2017.11.0", "source": "https://github.com/mozilla/telemetry-analysis-service", "build": "https://circleci.com/gh/mozilla/telemetry-analysis-service/2223" } :statuscode 200: no error :statuscode 404: a version.json wasn't found .. http:get:: /__heartbeat__ The heartbeat view will go through the list of configured Dockerflow checks in the :ref:`DOCKERFLOW_CHECKS` setting, run each check, and, if `settings.DEBUG` is `True`, add their results to a JSON response. The view will return HTTP responses with either a status code of 200 if all checks ran successfully or 500 if there was one or more warnings or errors returned by the checks. **Custom Dockerflow checks:** To write your own custom Dockerflow checks, please follow the documentation about :mod:`Django's system check framework ` and particularly the section **"Writing your own checks"**. .. note:: Don't forget to add the check additionally to the :ref:`DOCKERFLOW_CHECKS` setting once you've added it to your code. **Example request**: .. sourcecode:: http GET /__heartbeat__ HTTP/1.1 Host: example.com **Example response**: .. sourcecode:: http HTTP/1.1 500 Internal Server Error Vary: Accept-Encoding Content-Type: application/json { "status": "warning", "checks": { "check_debug": "ok", "check_sts_preload": "warning" }, "details": { "check_sts_preload": { "status": "warning", "level": 30, "messages": { "security.W021": "You have not set the SECURE_HSTS_PRELOAD setting to True. Without this, your site cannot be submitted to the browser preload list." } } } } :statuscode 200: no error :statuscode 500: there was a warning or error .. http:get:: /__lbheartbeat__ The view that simply returns a successful HTTP response so that a load balancer in front of the application can check that the web application has started up. **Example request**: .. sourcecode:: http GET /__lbheartbeat__ HTTP/1.1 Host: example.com **Example response**: .. sourcecode:: http HTTP/1.1 200 OK Vary: Accept-Encoding Content-Type: application/json :statuscode 200: no error .. _Dockerflow: https://github.com/mozilla-services/Dockerflow .. _django-logging: Logging ------- Dockerflow provides a :class:`~dockerflow.logging.JsonLogFormatter` Python logging formatter class. To use it, put something like this in your Django ``settings`` file and configure **at least** the ``request.summary`` logger that way:: LOGGING = { 'version': 1, 'formatters': { 'json': { '()': 'dockerflow.logging.JsonLogFormatter', 'logger_name': 'myproject' } }, 'handlers': { 'console': { 'level': 'DEBUG', 'class': 'logging.StreamHandler', 'formatter': 'json' }, }, 'loggers': { 'request.summary': { 'handlers': ['console'], 'level': 'DEBUG', }, } } .. _django-static: Static content -------------- To properly serve static content it's recommended to use `Whitenoise`_. It contains a middleware that is able to serve files that were built by Django's collectstatic management command (e.g. including bundle files built by django-pipeline) with **far-future headers** and proper response headers for the AWS CDN to work. To enable Whitenoise, please install it from PyPI and then enable it in your Django projet: #. Set your ``STATIC_ROOT`` setting:: STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles') #. Add the middleware to your ``MIDDLEWARE`` (or ``MIDDLEWARE_CLASSES``) setting:: MIDDLEWARE_CLASSES = [ # 'django.middleware.security.SecurityMiddleware', 'whitenoise.middleware.WhiteNoiseMiddleware', # ... ] Make sure to follow the SecurityMiddleware. #. Enable the staticfiles_ storage that is able to compress files during collection and ship them with far-future headers:: STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage' .. _Whitenoise: https://whitenoise.readthedocs.io/ #. Install brotlipy_ so the storage can generate compressed files of your static files in the brotli_ format. For more configuration options and details how to use Whitenoise see the section about `Using WhiteNoise with Django`_ in its documentation. .. _staticfiles: https://docs.djangoproject.com/en/stable/howto/static-files/ .. _brotli: https://en.wikipedia.org/wiki/Brotli .. _brotlipy: http://brotlipy.readthedocs.org/en/latest/ .. _`Using WhiteNoise with Django`: https://whitenoise.readthedocs.io/en/stable/django.html Settings -------- .. _DOCKERFLOW_VERSION_CALLBACK: ``DOCKERFLOW_VERSION_CALLBACK`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The dotted import path for the callable that returns the content to return under ``/__version__``. Defaults to ``'dockerflow.version.get_version'`` which will be passed the ``BASE_DIR`` setting by default. .. _DOCKERFLOW_CHECKS: ``DOCKERFLOW_CHECKS`` ~~~~~~~~~~~~~~~~~~~~~ A list of dotted import paths to register during Django setup, to be used in the rendering of the ``/__heartbeat__`` view. Defaults to: .. code-block:: python DOCKERFLOW_CHECKS = [ 'dockerflow.django.checks.check_database_connected', 'dockerflow.django.checks.check_migrations_applied', ]