Can Docker over ssh save your battery from running out?
The primary reason I have Docker installed on my MacBook is so that I can run
docker build - if I need
docker run, I can do that in other ways like connecting to a remote computer or using my remote Kubernetes cluster.
Whilst this is a convenient way to build images, it can also be an unfortunate way to drain your battery life because it is necessary to run a Linux Virtual Machine. Being low on battery is inconvenient when travelling or away from a power source for extended periods of time.
My hypothesis is that if we could use a remote server for Docker builds, we could save battery. Let's look at a few ways we could do that, then try one of them out and finally review the experience.
Expose the Docker Daemon on a port from a remote machine
With this option the
DOCKER_HOSTvariable is set followed by
tcp://and an IP address and port. Since Docker runs as root in a privileged context, it's important to enable TLS and authentication before opening up Docker to the world. The configuration is a little tricky and time-consuming, but once in place will build relatively quickly. See also: Protect the Docker daemon socket
Use a SaaS container builder
Google Cloud offers Cloud Build which can be triggered by pushing code up into a Git repository. The pros are no local docker, but the cons are that we're now tied into a SaaS product which we are going to need to use a lot and which is not free. The feedback loop is also very very slow between each change.
If you have a remote Kubernetes cluster, then you can use external tooling like Jenkins, Tekton, docker buildx or a myriad of other solutions to perform a build and push into a trusted registry. This suffers from a slow feedback cycle and having a hard dependency on Kubernetes.
Starting with Docker 18.09, the Docker client supports connecting to a remote daemon via SSH. Using ssh is similar to the option of enabling the Docker Daemon to be accessed via a public port, but unlike that option, it doesn't need lengthy configuration of TLS because of link encryption. Using ssh also means that the Docker daemon is not exposed directly or opaquely on the Internet.
docker build with
Create a cloud VM
You can create a cloud VM, or setup a second machine at your office or in your home. I used DigitalOcean and provisioned a machine with 1 Core and 3GB RAM for 15 USD / mo.
Log into the machine with
ssh and install Docker:
curl -SLsf https://get.docker.com | sudo sh
Add a user which make use of Docker and which can log into the machine remotely. Assuming Ubuntu run:
useradd build -G docker -m -S /bin/bash
Copy your public SSH key info for the new user:
sudo -u build mkdir -p /home/build/.ssh cp /root/.ssh/authorized_keys /home/build/.ssh/ chown -R build:docker /home/build/.ssh/
Configure your client
You'll need Docker's client to be installed on your local computer, or you can install all of Docker and then stop its daemon.
From here, either use the
-H flag or set
DOCKER_ADDR as an environment variable.
Use your remote Docker daemon
docker -H ssh://email@example.com:22 info Client: Docker Engine - Community Version: 19.03.3 API version: 1.40 Go version: go1.12.10 Git commit: a872fc2 Built: Tue Oct 8 00:55:12 2019 OS/Arch: darwin/amd64 Experimental: false Server: Docker Engine - Community Engine: Version: 19.03.5 API version: 1.40 (minimum version 1.12) Go version: go1.12.12 Git commit: 633a0ea838 Built: Wed Nov 13 07:28:22 2019 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.2.10 GitCommit: b34a5c8af56e510852c35414db4c1f4fa6172339 runc: Version: 1.0.0-rc8+dev GitCommit: 3e425f80a8c931f88e6d94a8c831b9d5aa481657 docker-init: Version: 0.18.0 GitCommit: fec3683
This matches what I see when I log in over
ssh to my Droplet and run
With the environment variable:
export DOCKER_HOST=ssh://firstname.lastname@example.org:22 docker info
Running the command over my hotel room in from San Diego with a Droplet in San Francisco, there was noticeable lag.
export DOCKER_HOST=ssh://email@example.com:22 time docker info real 0m1.410s
export DOCKER_HOST="" time docker info real 0m0.117s
Try running a container:
docker run --rm -ti alpine:latest ping -c 10 google.com
docker run is executed, the daemon first checks its local library for the image. If that image has to be downloaded from the Internet, then the chances are that your service provider's Internet connection is faster than your own. That could result in much quicker times to run images.
This was all rather simple to set up and much easier than using TLS for the Docker Daemon.
Build a container
Rather than building hello-world, let's build one of my Golang projects.
cd /tmp/ git clone https://github.com/inlets/inlets cd inlets docker build -t inlets .
At this point you may notice some significant latency because the inlets folder is
25M in size. Docker must first create a tarball of the contents of the folder, then copy it over the ssh connection to the remote machine to run the build. Since
ssh is also encrypted (which we need), this will take a significant amount of time.
We may also notice that the
go build step takes much longer on our cheap, shared VPS than it did on our quad-core 2-3k USD MacBook.
To compare the two I did a build to warm everything up and to ensure the remote machine had the base-layers available. Then I altered the
main.go file and timed another build.
- Remote droplet -
- MacBook dual-core -
real 0m42.703s(Docker for Mac - 2CPU / 8GB RAM)
Despite a slower CPU on the remote machine, the majority of our time was actually spent sending the 25MB tarball up to the remote machine over ssh. This entire build context was sent for every minor change I made.
What did I learn?
My hypothesis was that using
docker build over
ssh may save battery life, and that was clearly the case. If you don't believe me, then run Docker for Mac for a few hours. I did have an unconcious assumption that the latency for the remote build would match or be close to my local one and that was not the case. Not only is there a significant penalty to uploading the build context, it doesn't appear to be uploaded in a differential or incremetnal way, so this hit is taken on every build. To get an equivalent remote CPU means paying a significant amount of money per month, even with a cheaper VPS provider like DigitalOcean.
What about BuildKit? Well it can build / pull layers in parallel and perform caching, but unfortunately it didn't help due to the context still needing to be uploaded. The total time was 2m10s and I wonder if this is something that can be optimized for in the future?
On the plus-side, the remote machine was able to download a pre-built image from the Docker Hub and execute it faster than I could with my hotel WiFi connection.
I do believe that Docker may be useful for remote builds, for some people if the latency is low and the payload (files uploaded to docker) is relatively small. Perhaps the real advantage here is that Docker created a simple and secure way to remotely manage a single Docker host using
Try it out and let me know what you think.
- Twitter - @alexellisuk
- GitHub - github.com/alexellis