This is part of the Semicolon&Sons Code Diary - consisting of lessons learned on the job. You're in the containerization category.

Last Updated: 2024-12-03

Warning: This are WIP notes. Don't take this document in its current form too seriously

I started working on an existing php project which was fully dockerized locally. Unfortunately it was a black box to me, so this was my process of figuring out how Docker worked (generally and in this project)

Telltale files in a dockerized project

There are often two Docker related files in the root directory:

docker-compose.yml
Dockerfile

Docker CLI

To see if what Docker processes are running $ docker ps. But TBH when debugging, you probably want $ docker ps -a since this also lists non-running containers.
To see what containers are available $ docker container ls. This also shows the container IDs.
To stop a container $ docker stop b5b82dc2f650 - last bit is a container ID. You can just fill in the first few chars and it will do the rest (like git and SHA1s)
To remove all stopped containers: $ docker system prune -a
To spin up a container and shut it down after running the program, use $ docker container run IMAGE_NAME CMD e.g. $ docker container run alpine ls -l
Run a command within a given, already running, docker container with: $ docker exec my_container ping google.com - thie pings google within my_container (which is already running)
Get a shell inside a running container: $ docker exec -it app bash, where app is the container name. The -it bit is for "interactive" and "attach TTY"
In the above command you could pass in the numerical container ID. Careful: Image IDs (after build) != container IDs (a container is an instance of image - an image which is running)
And to get a shell for the root user $ docker exec -u root -it 750af8a bash (specify user with -u username)

Container

If you multiple docker container run alpine command commands (i.e. multiple commands on same image), they will not effect one another. Each has own filesystem etc.
If you modify the filesystem (e.g. create hello.txt) during a once-off $ docker run, you can actually get access to that modified system state by finding the container ID with $ docker container ls -a then running $ docker container start CONTAINER_ID and the running docker container exec CONTAINER_ID xyz

Images

Pull one locally with $ docker image pull NAME e.g. $ docker image pull alpine
Alpine is light (like 5mb) so popular
You can roll back to previous layer in an image (they are immutable after all)
The best way to create images is with a Dockerfile E.g. you could write a bunch of commands there, then run (in the same folder as this Dockerfile) $ docker image build -t DESIRED_IMAGE_NAME . e.g. $ docker image build -t hello:v0.1 . (for name hello:v0.1)
Each command in the docker file creates a layer, as seen with $ docker image history CONTAINER_ID. This allows it to use cache.
See available images with: $ docker image ls

Deleting stuff for blank-slate tests and debugging

docker rm -vf $(docker ps -aq) # containers
docker rmi -f $(docker images -aq) # images

How to create your own image

Let's say you did a lot of manual tweaking inside a docker container. From outside this container you can run docker commit your_container_id and then your system will be persisted exactly as is. This is not considered a best practice, but can be very handy for quick and dirty deploys (e.g. prototypes).

Later you can boot this up with docker run your_new_image_id.

Interactivity

Let's say you want to run a shell:

$ docker container run alpine /bin/sh

This version just exists immediately. Instead you have to add interactivity and attach a tty:

$ docker container run -it alpine /bin/sh

You will now be in the shell of the container, as you can confirm with uname -a

See history of what you've run

Containers that finished executing are hidden when you type docker container ls but they show up if you write docker container ls -a

Volumes

Help you persist and share data with other containers
By default, if you create a file in a container, even with $ docker container run the results can be seen by looking at the GraphDriver entry of docker container inspect CONTAINER_ID. See the UpperDir entry, specifically. Now run ls on that directory on your host system, and the data will be present. However, if you run docker container rm CONTAINER_ID that folder will be removed.
Contrast to where you add an entry to the Dockerfile: VOLUME ["/mydata"]. This time the data will be at the folder listed under the Mounts/Source entry of $ docker container inspect
If you specify the volume in the Dockerfile as /mydata then a folder called /mydata is created in the container. Only data stored there is persisted.
bind-mounting (where both are folders) is v helpful for sharing code in development - e.g. to map /tmp locally to /data in the container: $ docker container run -ti -v /tmp:/data alpine sh
Warning: By using bind mounts, you make it possible to change the host filesystem through processes running in a container, including creating, modifying, or deleting important system files or directories. This is a powerful ability with security implications, and could impact non-Docker processes on the host system. Use bind mounts with care.
It's possible to create a volume outside a dockerfile (e.g. command line docker volume create --name html) or with docker compose. This named volume can then by assigned to a particular folder in the container: html:/usr/share/nginx/html
with bind-mounting, consider what happens if the folder given in the container already contains data - e.g. the mapping /tmp:/bin/ - this clobbers the container's /bin folder and breaks everything.
if you have say a wordpress container that installs wordpress without touching your host system and a separate container wordpress_cli that expects wordpress to be present, you'll need to share a volume from the wordpress container to the wordpress_cli container

How to use current folder within docker container

docker run -it --rm -v $(pwd):/usr/src/project tensorflow/tensorflow bash

Now when you visit /usr/src/project it will have everything you need from the current folder.

Networking

From the host machine, you can ping the ip address of a running container and it will work. Get the ip address with docker network inspect bridge.
By default, a container can talk to the outside world (e.g. ping github.com works).
From within a docker container, how do you connect to the host computer, e.g. to a database connection that is available at localhost:8001 on the host machine? On the host run ifconfig Look for inet. It will have an ip address of format 10.x or 192.y That is the address that you use instead of localhost when you want to talk to the host machine. However this port will likely be firewalled, so you'll need to open it on the host. This may not be worth it. Thanks to: https://nickjanetakis.com/blog/docker-tip-35-connect-to-a-database-running-on-your-docker-host
Within a container, localhost differs: curl host.docker.internal
What is the effect to the port mapping here? $ docker run --name web1 -d -p 8080:80 nginx? It means traffic to port 8080 on the host machine is sent to 80 on the docker container. You can prove this by curling curl 127.0.0.1:8080. Since nginx was running with this command, we should get an nginx output. If you are running docker on a publicly accessibe server, e.g. one you SSH'ed into, then, assuming no firewalls, from any arbitrary machine you could curl HOSTIP:8080 and it would work.
Sometimes it makes sense to map to the same port - e.g. -p 80:80. But watch out: this can lead to trip ups, e.g. if someone has a local postgres database running that interferes with or is confused for the docker database.
A container's name (e.g. docker run --name=apperson) is the hostname. Thus, for example, the database host for a laravel application might go from being set to 127.0.0.1 on a non-dockerized codebase to being mysql in a dockerized one (if that was the name I gave to my mysql container). You can confirm this with getent hosts localhost (this command getent hosts, available on macos and linux resolves hosts to ip addresses - e.g. localhost is resolved to 127.0.0.1 on my mac)

Advanced Networking Details

Get a container to use a network with docker run ... -network=mynetwork ...
If you run docker network ls, you'll see a column scope. By default, this says local for each network, meaning that the network exists only on this Dockerhost.
By default, 3 networks are created - bridge, host, none
The bridge driver is designed for single-host networking. it is based on a Linux Bridge (aka virtual switch). You can verify this on the host machine by install bridge and then running brctrl show - it'll list docker0 if you have docker running. By default, however, no interfaces will be connected to it.
If you run ip a on the host system, you'll see info about the docker0 bridge. The subnet details (172.17.0.0/16 in my case) match what you'll see with $ docker network inspect bridge
By default, each container uses the bridge network to connect. If you run an ongoing container (e.g. a sleep daemon) then run brctl show, you'll see an interface was added, corresponding to the container.
Also if you do $ docker network create NAME it uses bridge.

Titbit on how Docker workers:

Text to terminal? Daemon creates a container from an image and then runs an executable there. Output from this executable is fed to Docker client, which sends it to the host machine terminal.
VM? It's a hardware abstraction. Part of the CPU and RAM on the host is divided across smaller virtual machines. The VM software usually has no real knowledge of any OS running inside the VM.
Container? By contrast to VM, it's an application abstraction. Focus is on OS and application. This means commands like docker container run, which spin up an image, can run much faster than if they also needed to boot up a VM.
A container shares the kernel with other containers, and runs as an isolated process in user space on the host OS. It namespaces devices, etc.

Docker Compose

Tool for defining and running multi-container Docker applications. With docker compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration.
i.e. it's an alternative to running multiple docker container run commands
Recreate a modified configuration with docker-compose build
To get docker compose to run your app, type $ docker-compose up. Or to detach from terminal: $ docker-compose up -d
(In the PHP world): Not to be confused with composer a PHP package manager.
Bring down the whole lot with $ docker-compose down
If you want to remove volumes (e.g. to reset a DB), docker-compose down is not enough. You you need to do docker-compose down --volumes

Here's how to interpret part of the docker-composer.yml file:

 #MySQL Service
  db:
    image: mysql:5.7.22
    container_name: db
   restart: unless-stopped
   tty: true
   ports:
     - "4306:3306"
   environment:
     MYSQL_DATABASE: laravel
     MYSQL_ROOT_PASSWORD: your_mysql_root_password
   volumes:
     - dbdata:/var/lib/mysql/
     - ./mysql/my.cnf:/etc/mysql/my.cnf
   networks:
     - app-network

  # App service
  app:
    build:
      context: .
      dockerfile: Dockerfile

Big picture: There are two containers: app and db. These are both automatically created. The container variables are essentially like command line arguments to docker run for each individual entry E.g.$ docker run -d --name app --restart unless-stopped app
image pulls a mysql image with v 5.7.22 from Dockerhub (or equivalent)
environment defines variables available on this image
restart: unless-stopped - on-failure policy. This one restarts a container if it fails... with the exception of when you deliberately stop a container.
tty- (NOT SURE) A tty is essentially a text input output environment aka shell. Setting this to true enables better interactivity via your local shell.
ports - syntax is "HOST:CONTAINER" - i.e. here it matches port 4606 on your local machine to port 3606 on the container.
volumes -
- The first is a named docker volume. We know this because it is not a path (dbdata). These volumes contents exist outside the lifetime of a container.
- The second is in "bind mount" format - we know this because the part before the ":" is a path. This means ./mysql/my.cnf on the host machine ('./' so currency directory) will contain the same files as /etc/mysql/my.cnf in the container. When the container is removed, the directory is unaffected on the host machine.
networks - here we're only setting a name to the network rather than configuring anything special
build - specifies location of DockerFile used to build container. Used instead of image. Here it's used because there is a custom DockerFile
Pro tip to use locally, if you have multiple docker projects on your computer: give your service and container names differernt names in each project (otherwise they clash). E.g. I had mailhog in projects and mailhog in projectp. They did NOT work well together. Better names were mailhog_project_s and mailhog_project_p

Volumes in Docker COmposer

If you want two containers to share some data between them (that doesn't appear on host system) do this

services:
   wordpress:
    volumes:
      # enter name and corresponding directory on first container
      - project_p:/var/www/html

   wordpress_cli:
    volumes:
      # enter name and corresponding directory on second container
      - project_p:/var/www/html
    # specify order
    depends_on:
      - wordpress

# then declare volume generally
volumes:
  project_p:

How to set up development environments that vary from production?

merge multiple docker compose config files: $ docker-compose -f docker-compose.yml -f docker-compose-production.yml build (uses both regular docker-composer.yml file and the docker-compose-production.yml file)
pass environment variables to the docker run command - e.g. docker run -e MYSQL_USER=homestead mysql:latest

be careful: the docker composer commands (like ps) will give different outputs depending on what you pass to it

$ docker-compose ps
# No results
Name   Command   State   Ports

vs. when I pass the docker-composer-development.yml as well

$ docker-compose -f docker-compose.yml -f docker-compose-development.yml ps

Name              Command             State    Ports
----------------------------------------------------
mysql   docker-entrypoint.sh mysqld   Exit 0

How to copy a file out of a docker instance into home machine

docker cp <containerid>:/<path> <host-path>

get the container ID with $ docker container ls

then carry out with copy with something like:

docker container cp  267a3ee553e2:/var/www/html/wp-content/ .

Dockerfile

ADD command and URLs

The Add command accepts URLs

This is better than curling since, if you curl into docker, some ways of using volumes delete the file immediately. Also, curl (the package) might not be available.

# this works
ADD https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar /usr/local/bin/wp

# this does not
curl -O https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar
mv wp-cli.phar /usr/local/bin/wp # ERROR: File not found

Semicolon & Sons