Unattended docker container updates

To keep your container infrastructure up to date and therefore secure, there are two primary objectives that you need to achieve:

1. Keep the host‘s operating system up to date
2. Keep the content of your containers up to date

Update the operating system

Updating your operating system is quite straight forward. Just use the well known package managers to do the job for you. E.g. for Debian this would be as easy as running apt-get update && apt-get upgrade on a regular basis.

To automate running these two commands, there are several approaches available. Either you configure a cronjob to run these commands on a regular basis, or you go for a higher sophisticated solution.

For apt there is a package called unattended-upgrades available. The package takes care to install all security upgrades on da daily basis. It is fully configurable via /etc/apt/apt.conf.d/50-unattended-Upgrades.

Update containers

For updating docker containers automatically, you not only need to consider the upgrade process itself.

The risk of getting incompatible

There are many containers out there which do not automatically ensure that the data (e.g. databases or configurations) are also updated. There might be even a newer version of your service, that is incompatible with the database running in your sidecar container. If you just upgrade one of them, you might end up with completely unusable services.

To mitigate this risk there are several things you can do:

• Use fixed container versions in your docker-compose.yaml. If you do not specify any image tag, docker implicitly uses latest as image tag. That means on an unattended upgrade, you would always get the latest and greatest version. Even if this version might introduce changes that break the api to one of your other services. Just have a look at the available tags for your image. Often there is a tag that allows you to restrict the major version of the image only. E.g. nginx:2 would allow only upgrades to other versions starting with 2, which hopefully doesn’t include any breaking changes in APIs.
• Make sure you have automated data backups in place. For example by using the cloud as your panic room. If you pull a new version of an image, there is always the risk that your data is corrupted. That is true even for images that migrate your data. Migration processes can fail!
• Have monitoring with alerting in place. In the rarely case that an upgrade fails, you need to know immediately. Monitoring systems such as Prometheus provide an easy way to know when a container is down, or a webservice is not reachable anymore.

Unattended container updates via cron

For achieving unattended container upgrades, you may use the standard docker commands you already know. Just put the script below in /etc/cron.daily/ and you are already good to go:

#!/bin/bash

cd /my-docker-compose-project
docker-compose pull
docker-compose up -d --remove-orphans 

This will daily pull the newest image versions referenced in your docker-compose project, restart the containers with updated images, and remove old containers not contained in the project anymore.

Unattended container updates via watchtower

There is a marvelous open source project called Watchtower which is basically a docker container on its own. It will monitor the registries of all images running on your docker daemon. If an image updates in the registry, it will pull the image and just restart your service with the old configuration. Genius, isn‘t it?

How to include watchtower in a docker-compose project

Add another service configuration for watchtower to your docker compose file.

services:
    watchtower:
        restart: always
        image: containrrr/watchtower
        container_name: watchtower
        volumes:
            - /var/run/docker.sock:/var/run/docker.sock

This will run watchtower in its default configuration, which monitors all containers in the system. There are many configuration options available. Just checkout their Documentation.