Docker swarm communication error fix: Storage Driver update

After adding a new worker node to a swarm, it had issues communicating with other services/containers on the overlay network — this worked fine when adding previous nodes (both manager and worker). It turned out that this newest node was installed with a new image that defaulted the storage driver to overlay2 where the previously installed nodes were using aufs. All the nodes are on Ubuntu 16.04.

You can check for support with the grep aufs /proc/filesystems command. If you don’t see any results, you may need to install sudo apt-get install linux-image-extra-`uname -r` (or other updates). If you do have aufs support and Docker defaults to overlay2, you can spec the storage driver in /etc/docker/daemon.json via:

  "storage-driver": "aufs"

and restart Docker (via systemctl restart docker.service or similar). Note that any images or containers created with the old storage driver will not be accessible to the new one so you should delete/prune/etc what you won’t need and stop & remove all running containers prior to restarting.

Docker: fix terminal width issue

Sometimes the terminal size reverts to the 80×24 default size, particularly when using Docker-in-Docker. To fix, pass the terminal specs when calling `docker run` or `docker exec`:

docker run --rm -it -e COLUMNS=$COLUMNS -e LINES=$LINES -e TERM=$TERM -it ubuntu:16.04

(see issue here)

Simple Made Easy (link)

An oldie-but-good I was reminded of recently:

Rich Hickey emphasizes simplicity’s virtues over easiness’, showing that while many choose easiness they may end up with complexity, and the better way is to choose easiness along the simplicity path.

Sample takeaways:

  • We should aim for simplicity because simplicity is a prerequisite for reliability.
  • Simple is often erroneously mistaken for easy. “Easy” means “to be at hand”, “to be approachable”. “Simple” is the opposite of “complex” which means “being intertwined”, “being tied together”. Simple != easy.

Choose simple constructs over complexity-generating constructs; It's the artifacts, not the authoring; Create abstractions with simplicity as a basis; Simplify the problem space before you start; Simplicity often means making more things, not fewer; Reap the benefits!

He even predicts (to some extent) microservice architecture by declaring a set of separate strings hanging down independently can show a byproduct of simplicity can be more things, but things that are easier to understand and manage than if the strings were intertwined — and easier to update individually.

Swap column values in MySql (via temp variables)

If you want to swap values in one query in MySql without a temporary table, you can use variables to hold values that otherwise would be changed before you can use them.

E.g., swap start_date and end_date via:

UPDATE mydates SET start_date=@tmp:=start_date, start_date=end_date, end_date=@tmp WHERE start_date > end_date

Install amqp-ext PHP extension to Apache in Docker

I wanted to add RabbitMQ support to my PHP app and prefer to use the PECL amqp package:

This extension can communicate with any AMQP spec 0-9-1 compatible server, such as RabbitMQ, OpenAMQP and Qpid, giving you the ability to create and delete exchanges and queues, as well as publish to any exchange and consume from any queue.

Using pecl install won’t install all the dependencies. There are various options to how to solve this including fetching from Git repos, however this is the one that worked for me and seemed the most straight-forward (ymmv). See below for code used in a Dockerfile with base image FROM php:7.1.14-apache which uses debian:jessie, so the native commands should work in similar Debian/Ubuntu linux distros.

RUN curl -o librabbitmq4_0.8.0-1_amd64.deb
RUN curl -o librabbitmq-dev_0.8.0-1_amd64.deb
RUN dpkg -i librabbitmq4_0.8.0-1_amd64.deb
RUN dpkg -i librabbitmq-dev_0.8.0-1_amd64.deb
RUN pecl install amqp-1.9.3 && docker-php-ext-enable amqp

Note if you do use this in a Dockerfile, you can combine some of the statements to optimize layers. You also may need to update the lib version if it’s updated or if you change the PECL version to install.

Docker: get hash from image name

Particularly if you are using a static image name like “latest” or “develop”, it is useful to be able to know and verify the exact version via the sha256 hash. To get the value you can use:

docker inspect --format='{{index .RepoDigests 0}}' repo/image-name:tag

To use this in a CI/CD workflow for Docker Swarm, you can do something like this:

SHA=$(docker inspect --format='{{index .RepoDigests 0}}' repo/image-name:tag)
docker service update --image $SHA --detach=true --with-registry-auth service-name

« Previous Entries