Thursday, 12 September 2019

Dissatisfied With Docker

I am not satisfied with Docker.

Untrusted users should be able to run their own container instances. Running a basic container instance means forking a process, putting it in the right kernel namespaces, and setting up mounts and virtual network interfaces, and those can all be done without privileges. Unfortunately, in Docker, access to the Docker daemon is tantamount to being root. Fixing this would improve both convenience and security.

In fact, a global system daemon should not be needed. Either users should be able to run their own daemons or container management should avoid having a daemon at all, by storing container state in a shared database.

Docker container builds are too slow. Installing a container image requires piping massive amounts of image data to the daemon over a socket, which is ridiculous. This could be avoided by passing a file descriptor to the container manager ... or you might even get rid of container image management and start containers by pointing directly to the files comprising the image.

Docker container instances start too slowly. It shouldn't take seconds to start or stop a simple small container. The system calls required to start a simple container run in milliseconds.

No doubt these observations are old news, so I assume there are better container implementations out there. Which one do I want?

15 comments:

  1. Podman for single host stuff, cri-o for Kubernetes. Both are true rootless container runtimes without an all powerful daemon in play.

    https://podman.io
    https://cri-o.io

    ReplyDelete
    Replies
    1. Thanks. If I get a chance to revisit our setup I will check out podman.

      Delete
  2. Singularity is a good choice.

    ReplyDelete
  3. a direct rebuttal https://www.redhat.com/sysadmin/rootless-podman that agrees with your issue but not with your solution

    ReplyDelete
    Replies
    1. I didn't propose a solution, and I don't disagree with anything written there...

      Delete
  4. LXD is a good choice https://linuxcontainers.org/lxd/

    ReplyDelete
  5. Sylabs Singularity runs containers as unprivileged users, with no daemon:

    https://github.com/sylabs/singularity
    https://sylabs.io/singularity/

    You will still need privileges to *build* the container if you want to do privileged operations (like run the package manager in the container)

    ReplyDelete
  6. Docker 19.03 supports rootless mode

    https://get.docker.com/rootless
    https://www.slideshare.net/AkihiroSuda/dockercon-2019-hardening-docker-daemon-with-rootless-mode

    ReplyDelete
    Replies
    1. That's good, but I also want daemonless mode.

      Delete
  7. I would use podman in a heartbeat but it doesn't support docker-compose.

    I also really want to be able to run individual Docker projects out of different specified directories, instead of the whole system sharing /var/lib/docker.

    ReplyDelete
    Replies
    1. This is being worked on: https://github.com/containers/podman-compose

      Delete
  8. Does anyone know how to:
    1) launch your latest docker images (i.e. with new versions of your services) via docker-compose
    2) wait until they are started to switch networking to them and shut down the previous versions???

    This seems like a very useful process for those of us who want to use docker in production without downtime... Otherwise, how are we supposed to upgrade to our latest dockerized services without downtime?

    ReplyDelete
    Replies
    1. You can use docker service update --pull to update a running service. If you update the tag of the service, you can use the --image tag

      Delete
  9. OT: don't justify the text on the comments, on mobile it is difficult to read with all those spaces inserted, on some comments it only displays tree words per row.

    ReplyDelete
  10. The daemon aspect is valuable because it provides API and a central place to dispatch events, logging, reattaching to consoles, managing network state, etc. Those things could be isolated to less-than-root capabilities, I suspect. The event dispatch stuff is quite powerful; check out “nginx-proxy” for an example.

    I agree that it would be great if more were possible without root. Right now I use the fact that cloud VMs or such are cheap to just split by user that way, but it’s not ideal.

    Container startup feels slow if you compare it to fork(), but if you’re coming from VMs as the unit of execution then they’re a big step faster (and smaller)!

    docker-compose feels to me like it does too much magic with things like networking without giving useful facilities like “consider me up when my ports are bound, and then start dependents”.

    ReplyDelete