Thursday, 01 March 2018

Deploying cross platform images to Docker registries

One thing I noticed when working with docker and cross platform registries was that sometimes you can pull the same image tag from a remote registry and get different images depending on which platform you requested. It was certainly working in a different way than the local list of images! Digging deeper I learned that it wasn't something new, and it was up for close to half a year now! You can read the official announcement here: https://blog.docker.com/2017/09/docker-official-images-now-multi-platform/

Basically, when you try to pull an image from a repository, your client would actually pull a manifest file listing either the details of the image, or a list of images that can be pulled based on the local machine's cpu architecture, os platform and version! This way you can pull the same image on various machines, and have it running on just about any platform you want. This is especially powerful with the release of Docker for Windows 18.03 where you can run both Windows and Linux images on the same machine side by side!

The manifest format is quite simple, and easy to digest! For example, this is the manifest for my Hello World app that I created to test multi platform docker deployments:

{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1077,
         "digest": "sha256:96142e1e7fab2c926d904861cc09fa757e2647dd03725df658e39ba5b92c81ed",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 2338,
         "digest": "sha256:cb9a593dd2c0bde0e6145a432a88bce0b9378951df3396940048f9cdc737128f",
         "platform": {
            "architecture": "amd64",
            "os": "windows",
            "os.version": "10.0.14393.2068"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 2016,
         "digest": "sha256:7a5870c86a3244b4e553c5e6da193e226178843a1fdab3b66461a4dfeb50bf34",
         "platform": {
            "architecture": "amd64",
            "os": "windows",
            "os.version": "10.0.16299.248"
         }
      }
   ]
}

You can see that there's two windows images available, based on whether you have the 1709 Fall Creator's Update installed (v10.0.16299.248) or not (v10.0.14393.2068)

If I pull this image on a linux machine, I will get the alpine based image. If I pull it on an older windows machine, I'll get the nanoserver based image, and on the latest Windows 10 builds I'll get a nanoserver-1709 based image!

So how hard was it to create this? Not that hard actually! The only issue was the fact that I couldn't easily find any documentation for this. I must have searched wrong. Luckily the help commands and a healthy dose of stackoverflow sent me on the right path! Seems like this is not yet available in the stable builds, and even the edge builds has this disabled by default! You have to open your Docker CLI config.json file (not to be confused with the docker daemon's config file). It will usually reside in your user profile directory at ~/.docker/config.json and set the experimental option to enabled (not true). If it wasn't there - you can add it. Your config file can look something like this:

{
	"experimental": "enabled"
}

Secondly, you'll have to create the manifest file. First push all the images you want combined to the remote repo (this will ensure their sha256 digests are calculated). Then create a new manifest. In my case I wanted to add a latest tag that would direct the user's client to install either the alpine, nanoserver or nanoserver-1709 tags. My command looked like this, you'll have to adjust yours with your package and tag names. (Once you have a manifest created, every time you want to re-create it, you'll have to add the --amend parameter like I've done below)


docker manifest create --amend artiomchi/helloworld:latest artiomchi/helloworld:alpine artiomchi/helloworld:nanoserver artiomchi/helloworld:nanoserver-1709
Created manifest list docker.io/artiomchi/helloworld:latest

Once the manifest is created, you can then inspect it and make alterations (for example to customise the OS version numbers). In my case the generated manifest was exactly what I wanted:

docker manifest inspect artiomchi/helloworld:latest

Finally, once you're ready - push the manifest to the remote repository:

docker manifest push --purge artiomchi/helloworld:latest

Note the -p argument I passed to the push command - it will purge the local copy of the manifest after the push was successful. Interestingly enough, there is NO command to remove the local manifest, and the --amend attribute on the create tag doesn't actually change the existing manifest. So I ended up in a situation where I couldn't update the manifest until I pushed it to the server again, overwriting what was there, but using the purge argument this time. Luckily I already had the same copy of the manifest on this machine as was on the repository server, but this could have been worse if I had an older copy cached.

Overall, this was quite easy, very powerful and will make for a great end user experience for the users, I would definitely recommend doing the same if you're publishing cross platform docker repositories:

References: