How to Manage Persistent Data in Docker Containers

In this guide, we will discuss how to manage persistent data in Docker containers.

Docker is one of the most popular container technology. It allows running applications in an isolated environment. Nowadays, container technologies are deployed in production environments as well.

Data management in Docker is significantly different than traditional server-based computing. In the case of Docker, all files are created on a writeable layer by default. It means:

Data doesn’t persist when the container is removed
Data cannot be shared with other containers running on the same host as the writable layer is unique per container.

This seems a problematic situation. However, Docker provides a way to handle such scenarios.

Also Read: How to Install Docker on Ubuntu 22.04 / 20.04 (Step by Step)

Use Cases for Data Persistence

Docker provides the following two options for data persistence:

Bind mounts
Volumes

We need data persistence in stateful applications when they are running inside containers. Following are the common use case of data persistence:

Persist logs generated by the application
Persist database records created by the application

Data Persistence Using Bind Mounts

Bind mount feature is available since the early days of Docker. With the help of this feature, we can mount the host directory into a container.

We can use either –mount or –volume options to create the bind-mount. However, the –mount option is recommended as it is more explicit and verbose.

1) Use a Bind Mount in Read-Write Mode

Bind mount allows data sharing between the containers and the host machine. It means the host can see the data generated by the containers and vice-versa.

By default, bind mount is created in read-write mode. Let’s understand this with an example.

First, create a directory and file on the host machine:

$ mkdir /tmp/read-write-mount
$ cd /tmp/read-write-mount
$ echo "Read-write bind mount example" > file-1.txt

Now, let’s mount this directory into the container using the bind mount:

$ docker container run --rm -d -it --name web-server-01 --mount type=bind,source="$(pwd)",target=/rw-mount nginx:alpine

In the above example, the mount option accepts comma-separated key-value pairs. In this command:

type – represents the type of the mount. We have set it to bind
source – represents a directory on the host machine. We have set it to the current working directory using the pwd command
target – represents the path in the container. We have set it to /rw-mount

Let’s inspect the container and verify that the bind mount was created correctly:

$ docker inspect web-server-01 --format '{{ json .Mounts }}' | python3 -m json.tool
[
    {
        "Type": "bind",
        "Source": "/tmp/read-write-mount",
        "Destination": "/rw-mount",
        "Mode": "",
        "RW": true,
        "Propagation": "rprivate"
    }
]

In the above output, “RW”: true indicates that the bind mount was created in a read-write mode.

Now, let’s exec to the container and verify that the container can access the host’s directory and its contents:

$ docker exec -it web-server-01 sh
# ls /rw-mount
file-1.txt
# cat /rw-mount/file-1.txt

In the above output, we can see that container is able to access host’s directory that is mounted on /rw-mount path.

Additionally, we can also verify that the host can access the data created by the container. To do that, create a new file in the container and exit from it:

# echo "This file is created by the container" > /rw-mount/file-2.txt
# ls /rw-mount
file-1.txt  file-2.txt
# exit

Now, let’s verify that the newly created file is present on the host machine:

$ ls /tmp/read-write-mount/
file-1.txt  file-2.txt
$ cat /tmp/read-write-mount/file-2.txt
This file is created by the container

In the above output, we can see that host is able to access the file created by the container.

2) Use a Bind Mount in Read-Only Mode

Sometimes, the container needs read-only access to the host’s data. In such cases, we can create a bind mount in a read-only mode. We can achieve this by providing a readonly flag. Let’s understand this with an example.

First, create a new directory and file on a host machine:

$ mkdir /tmp/read-only-mount
$ cd /tmp/read-only-mount/
$ echo "Read-only bind mount example" > file-3.txt

Now, let’s mount this directory in a read-only mode into the container using the below command:

$ docker container run --rm -d -it --name web-server-02 --mount type=bind,source="$(pwd)",target=/ro-mount,readonly nginx:alpine

Please note that, in the above command we have used readonly flag after the target field.

Let’s verify that the read-only bind mount was created successfully using the below command:

$ docker inspect web-server-02 --format '{{ json .Mounts }}' | python3 -m json.tool
[
    {
        "Type": "bind",
        "Source": "/tmp/read-only-mount",
        "Destination": "/ro-mount",
        "Mode": "",
        "RW": false,
        "Propagation": "rprivate"
    }
]

In the above output, “RW”: false indicates that the bind mount was created in a read-only mode.

To verify this, let’s exec to the container and try to create a file in a /ro-mount directory:

$ docker exec -it web-server-02 sh
# echo "This should generate error" > /ro-mount/file-4.txt
sh: can't create /ro-mount/file-4.txt: Read-only file system

As expected, the file creation operation failed due to a read-only file system.

Data Persistence Using Volumes

Similar to bind mounts, we can use the volumes to persist data. The bind mounts are tightly coupled with the directory structure and OS of the host machine. Thus they are less portable.

In Docker, volumes are preferred for data management as they are managed by the Docker. They provide portability across multiple OS platforms.

1) Use Volume in Read-Write Mode

Like bind mounts, volumes enable data sharing and persistence between the containers and host machine.

By default, volume is attached in read-write mode to the container. Let’s understand this with an example.

First, create a volume:

$ docker volume create read-write-vol
read-write-vol

Now, let’s attach this volume to the container:

$ docker container run --rm -d -it --name web-server-03 --mount source=read-write-vol,target=/rw-mount nginx:alpine

Please note that in the above command we have used volume name – read-write-vol as a parameter with the source field.

Let’s verify that volume was attached correctly using the below command:

$ docker inspect web-server-03 --format '{{ json .Mounts }}' | python3 -m json.tool
[
    {
        "Type": "volume",
        "Name": "read-write-vol",
        "Source": "/var/lib/docker/volumes/read-write-vol/_data",
        "Destination": "/rw-mount",
        "Driver": "local",
        "Mode": "z",
        "RW": true,
        "Propagation": ""
    }
]

In the above output, “RW”: true indicates that the volume is attached in a read-write mode.

We can also see that, read-write-vol volume is using /var/lib/docker/volumes/read-write-vol/_data directory from the host machine. The Source field from the above output indicates this.

Let’s display the content of that directory:

$ sudo -s ls -l /var/lib/docker/volumes/read-write-vol/_data
total 0

Here, we can see that the directory is empty. Let’s create a new file inside that directory:

$ echo "Read-write volume example" | sudo -s tee /var/lib/docker/volumes/read-write-vol/_data/file-5.txt
$ sudo ls /var/lib/docker/volumes/read-write-vol/_data/
file-5.txt

Now, let’s exec to the container and verify that the container can access the file created by the host:

$ docker exec -it web-server-03 sh
# ls /rw-mount
file-5.txt
# cat /rw-mount/file-5.txt

Here, we can see that container is able to access the host’s directory that is mounted on /rw-mount path.

In addition to this, we can also verify that the host can access the data created by the container. To do that, create a new file in the container and exit from it:

# echo "This file is created by the container" > /rw-mount/file-6.txt
# exit

Now, let’s verify that the file is present on the host’s machine:

$ sudo ls /var/lib/docker/volumes/read-write-vol/_data/
file-5.txt  file-6.txt
$ sudo cat /var/lib/docker/volumes/read-write-vol/_data/file-6.txt
This file is created by the container

Here, we can see that host is able to access the file created by the container.

2) Use Volume in Read-Only Mode

In certain scenarios, the container needs read-only access to the host’s file system. In such cases, we can attach volume in a read-only mode. We can achieve this by adding readonly flag to the command. Let’s understand this with an example.

First, create a volume:

$ docker volume create read-only-vol
read-only-vol

Now, let’s start the container with this volume:

$ docker container run --rm -d -it --name web-server-04 --mount source=read-only-vol,target=/ro-mount,readonly nginx:alpine

Please note that, in the above command we have used readonly flag after the target field.

Let’s verify that volume was attached correctly using the below command:

$ docker inspect web-server-04 --format '{{ json .Mounts }}' | python3 -m json.tool
[
    {
        "Type": "volume",
        "Name": "read-only-vol",
        "Source": "/var/lib/docker/volumes/read-only-vol/_data",
        "Destination": "/ro-mount",
        "Driver": "local",
        "Mode": "z",
        "RW": false,
        "Propagation": ""
    }
]

In the above output, “RW”: false indicates that the volume was attached in a read-only mode.

Now, let’s exec to the container and try to create a file in a /ro-mount directory:

$ docker exec -it web-server-04 sh
# echo "This should generate error" > /ro-mount/file-7.txt
sh: can't create /ro-mount/file-7.txt: Read-only file system