UP | HOME
Impaktor

Impaktor

Inject Emacs intravenously

On docker — running, monitoring, administering, and cleaning images and containers
Published on Sep 10, 2020 by Impaktor.

The greatest thing in the world is to know how to belong to oneself

Michel de Montaigne, The Complete Essays

Table of Contents

Introduction

These are my notes on Docker, compiled from a tutorial, arch wiki, Jérôme Petazzoni’s PyCon 2016 talk and possibly other places.

Definition: “Images” & “Containers”

One can have multiple containers from a single image

  • images are conceptually similar to classes
  • layers are conceptually similar to inheritance
  • containers are conceptually similar to instances

Getting started

Installing docker

start it manually for now:

sudo systemctl start docker.service

…or add it for each startup:

sudo systemctl enable docker.service

check your groups

groups

add yourself to docker group

gpasswd -a <user> docker

test that you can run docker as normal user:

docker info
docker version

Basic docker commands

Images are by default stored in: /var/lib/docker

docker search <image name>

e.g. install an arch docker:

docker pull archlinux/base

Run interactive (-i) and in terminal (-t):

docker run -i -t ubuntu

installs the docker image and drops me in the docker image (exit to leave)

In general:

docker run [image name] [command to run]

shut down running docker image

docker stop [container ID]

or more brütal:

docker kill
docker attach [container ID]

remove docker image

docker rm [container ID]

Monitoring

Show running docker processes:

docker ps

Show last, show only id (useful for scripting)

docker ps -l -q

Show also (all) non-running:

docker ps -a

Show status (CPU/RAM, etc) on running images

docker stats
docker stats --no-stream
docker stats <list container IDs>

Or also include stopped containers

docker stats --all

Common usage for “logs” (shows output of container, e.g. if running daemonized with run -d, or to publish all ports exposed in the image: run -P)

docker logs --tail 10 --follow <container ID>

There are three networks by default:

docker network ls

e.g.

docker network inspect bridge

“The bridge network corresponds to the docker0 network, which is present in all Docker installations. The none network doesn’t have any access to the external network, but it can be used for running batch jobs. Finally, the host network adds a container on the host’s network stack without any isolation between the host machine and the container.”

From linuxhint

Often one might develop a docker image to run on a VM. Then we need to know how much RAM to configure the VM for. To monitor RAM usage, to file, of a running docker image, for plotting later on:

while true; do docker stats --no-stream --format "{{.Container}}:\t{{.CPUPerc}}\t{{.MemUsage}}" | tee --append stats-training.txt; sleep 1; done

Gives output:

ab5fade5cf45:	0.05%	547.5MiB / 15.67GiB
ab5fade5cf45:	0.01%	547.5MiB / 15.67GiB
ab5fade5cf45:	0.01%	547.5MiB / 15.67GiB
ab5fade5cf45:	264.85%	1.023GiB / 15.67GiB
ab5fade5cf45:	104.34%	3.01GiB / 15.67GiB
ab5fade5cf45:	100.48%	3.075GiB / 15.67GiB
ab5fade5cf45:	111.40%	2.187GiB / 15.67GiB

To extract RAM, e.g. for plotting in gnuplot, I use this:

awk '{m=substr($3, 0, length($3)-3); u=substr($3, length($3)-2); if("MiB"==u) M=m; else if("GiB"==u) M=1024*m; print M, m, u}' stats-training.txt > out

or this to include CPU usage:

awk '{cpu=substr($2,0,length($2)-1); m=substr($3, 0, length($3)-3); u=substr($3, length($3)-2); if("MiB"==u) M=m; else if("GiB"==u) M=1024*m; print cpu, M, m, u}' stats-training.txt > out

which gives easy to plot results:

547. 547. MiB
547. 547. MiB
547. 547. MiB
1044.48 1.02 GiB
3072 3.0 GiB
3143.68 3.07 GiB
2232.32 2.18 GiB

e.g. in gnuplot:

plot 'out' u 0:2 title "RAM", '' u 0:1 title "CPU"
set xlabel 'time (s)'
set ylabel 'docker RAM (MiB)'

Setting up a docker image

Start a docker image:

docker run -it ubuntu

now in said docker image:

apt-get update
apt-get install <needed packages>
exit

show diff from base image:

docker diff <container ID>

commit container:

docker commit <container ID>
docker commit <container ID> <desired name of image>

which returns an sha256 hexadecimal string. NOTE: commit only saves state of file system, not running processes.

Launch it, and check that it has the installed files

docker run -ti <returned image ID>

or tag it to alias:

docker tag <returned image ID> <desired name of image>

Instead of using commit to create a docker container, we can use the build command, which takes a docker-file that automates what we did manually above. Create file Dockerfile (default name it looks for) in folder (optional name) with content:

FROM ubuntu
RUN apt-get update
RUN apt-get install figlet

execute, new image with tag “Figlet”

docker build -t Figlet <path to folder with dockerfile>

good to have:

docker history Figlet

run figlet command from figlet container:

docker run figlet figlet hello

can add the command directly in the container file:

FROM ubuntu
RUN apt-get update
RUN apt-get install figlet
CMD figlet hello

and will only then need:

docker run figlet

if I still want to get into the container:

docker run -ti figlet bash

Using ENTRYPOINT allows passing arguments to docker, so the container behaves like a binary file, or script that can take arguments. If using ENTRYPOINT and CMD together, both must use json syntax

(Taken from https://www.youtube.com/watch?v=ZVaRK10HBjo&t=1h45m)

Naming containers

Docker gives names by default, see docker ps -a or we can give a name

docker run --name ticktock python

(or docker rename) and then use:

docker logs ticktock
docker stop ticktock
docker inspect ticktock
docker inspect --format '{{ .State.Running}}' ticktock

(This was taken from yt: Introduction to Docker and containers - PyCon 2016)

Example dockerfile

FROM python:3.7.3

## every run-command creates a new layer, i.e. use "&&" to execute multiple commands as single line/layer
RUN apt-get update && apt-get install foobar

# NOTE: this should be done before copying source code, or else, any
# change in source code will invalidate the cache, and all subsequent
# steps are re-done, so it would re-install all dependencies after
# each change in code or any other file
COPY ./requirements.txt /tmp/requirements.txt
RUN pip install -qr /tmp/requirements.txt

# now copy everything in (buildcontext) folder recursively, to /src:
# (not possible to reference paths outside of passed build folder)...
COPY . /src
# ...thus this is the same as above:
# COPY / /src

# this is where we do stuff, rel. CMD/RUN and other tings.
WORKDIR /src

# like COPY, but can get remote files,
ADD http://foo.bar.txt /opt/
#  and will auto-unzip zip and tar local files
ADD foo.zip /opt/

CMD python /myfile.py

# setting variable (don't use "RUN export..." as that will only be for
# that build container and then be discarded)
ENV MY_VAR

# e.g. common way to change configuration parameters for containers:
# docker run -e MY_VAR=8000

Update docker when changes are made

See this stackoverflow post

Clear disk space — Go nuclear!

Running docker eats a lot of disk space, especially when experimenting / starting out.

When docker images run out of disk space:

sudo rm -f /var/lib/docker
sudo systemctl stop docker.service
sudo systemctl start docker.service

Read: https://www.digitalocean.com/community/tutorials/how-to-remove-docker-images-containers-and-volumes

docker system prune -a
docker images -a
docker rmi <Image> <Image>
docker images purge
docker volume prune

Links / external resources