This tutorial addresses issues with Docker and how nesting docker applications inside of Incus/LXD containers is a best practice. I introduce a couple of helpful utilities along the way.
Docker applications accumulate orphaned stopped containers, networks, volumes, images and build caches. Although “docker system prune -a” theoretically removes these “dangling” data structures, the “/var/lib/docker/vfs/dir” folder contains all of the layers used by running containers. This data repository is critical to docker, but becomes unruly over time.
You can examine an application created with “docker compose”:
docker compose ls
You can drill down to the indivdual containers:
docker ps
“docu” is the docker usage utility. To install docu, start by creating a folder on your docker host or Incus/LXD container hosting a docker application. Normally I suggest creating a folder in which to host the components of each docker application and placing the components of an application inside of it.
mkdir docu
cd docu
Edit the docker compose file for docu:
nano docker-compose.yml
Insert the following into the file.
version: '3.3'
services:
doku:
image: amerkurev/doku
container_name: doku
restart: always
ports:
- '9090:9090'
volumes:
- '/var/run/docker.sock:/var/run/docker.sock:ro'
- '/:/hostroot:ro'
Save the file with a CTRL O and Enter and CTRL X to exit the nano editor.
Execute “docu”:
docker compose up -d
Find out the address of the container by examining eth0:
ip a
Go to your browser at the address that you found at port 9090 to launch “docu”:
The various tabs describe the docker space utilization. Each tab also identifies the docker command to “prune” dangling or unused docker constructs for docker data:
Likewise, the container tab:
You can also examine volumes:
There is a general command to remove unused docker infrastructure.
docker system prune -a
This command is helpful. However not in the above example. By shutting down the “docu” application, you can see the prune at work:
Here’s an example of a command that can remove the dangling or unused images on a docker host:
docker rmi $(docker images -f "dangling=true" -q)
A really useful command to examine disk utilization in general and docker infrastructure that is unused, but not managed by the above commands is “Ncurses Disk Usage” or ncdu.
Install ncdu:
sudo apt install ncdu
Run the disk usage utility.
sudo ncdu /
In my example, ncdu tells me that “/var” has 9GB of data used.
If we drill down in ncdu, we find that the bulk of docker storage is located in the folder below:
/var/lib/docker/vfs/dir
This is where docker stores the “segments” for various containers that have been loaded when you perform “docker compose” or “docker run” commands.
The problem with this area is that if a docker application crashes or a docker host is shut down abnormally, these files may be left behind and can use significant storage.
You can’t just arbitrarily delete the files in this folder or you WILL corrupt your docker applications.
There is a “cleanup docker volumes” application that can delete this orphaned data. My advice is to backup your data before using this, because it can be potentially dangerous.
I warned you! So here’s the command:
docker run -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker:/var/lib/docker --rm martin/docker-cleanup-volumes
As this utility runs, it deletes the “docker vfs” elements that it sees as unused. Very useful, but be sure to do a backup before running this utility. Watch my video “Incus Container Snapshots”.
You will note, that “docker-cleanup-volumes” does not delete volumes in use.
After running “ncdu” again, I notice that my initial 9GB of system data has been reduced to 7.5GB and now the docker vfs storage is 6.8GB.
As always, perform snapshots/backups before performing these techniques.