Linux Container Ecosystem — Pre Container Camp Primer

Container Camp number 4 is just around the corner on the 15th of April 2016 at Bespoke in downtown San Francisco. As proud supporters of Container Camp, keen Linux Container enthusiasts, and early adopters, we thought we’d take the opportunity to put together a Linux Container ecosystem overview as your pre-Container Camp refresher.

What are Containers?

Linux containers (LXC) are essentially isolated Linux processes, made possible by various new Linux kernel features over the last ten years or so (namely “cgroups”). The isolation that cgroups brings allows processes to be allocated a limited amount of CPU and memory (among other resources), and to be unable to access other parts of the filesystem (it is essentially a development of the “chroot”).

Raw LXC that is built in to Linux has been notoriously hard to use, and the explosion in excitement and adoption of Linux containers today is largely attributable to Docker’s fantastic user experience for developers. By also addressing container packaging, distribution and discovery, Docker brought LXC to the mainstream. LXC existed before Docker, and technologies similar to LXC were available in FreeBSD and Solaris before Linux. But little of this mattered to most people until Docker came along.

A Brief History of Containers

At Container Camp London 2015, Bryan Cantrill took us on a historical tour of how we got to where we are today: containers and microservices. Since VMs became ubiquitous, modern cloud computing has meant VMs running with hardware-level virtualisation. Bryan argues that this is too low in the stack, PaaS (platform-level virtualisation/abstraction) is too high, but that containers (OS-level virtualisation) are just right. In his 2015 San Francisco keynote he continued the story, bemoaning the fact that the state of the art of deploying Docker to production is to run containers in a VM, as it negates the economic advantage we gain from containers. It will be interesting to see any advances on these current practices in the Container Camp talks.

Whilst the debate over systemd’s very existence seems to have died down in the past year, there has been another interesting systemd-related story taking shape: Docker doesn’t play well with systemd. Red Hat’s Dan Walsh’s has spoken about his frustrations trying to get good Docker integration into RedHat’s systemd-based Linux distribution. The trouble is that both Docker and systemd want to orchestrate containers. Red Hat already ship a custom set of patches with Docker, will they fork Docker in order to integrate nicely with systemd? Will Docker relent? This is part of a continued shake-up of Linux’s role; we’re running Linux OSs as containers under Docker on Linux VMs on hypervisors, which is kind-of crazy. Some of those containers run systemd inside; many of the VMs run a systemd-based Linux distribution. Will we see more container-focused operating systems with the Docker daemon running as PID1, like RancherOS does? Will unikernels catch on and save us from this? I’m hoping we’ll hear a few people talking about this at Container Camp.

Why Use Containers?

The business case for containers is too-often missing from the discussion. Whilst containers are primarily about process and resource isolation, this is not the most important value they can offer to your business. What containers enable has a lot more to do with breaking down the developer / operations divide (the DevOps philosophy). By providing a great user experience for developers around the packaging and sharing of container images, Docker showed us the value of shifting the ownership of OS dependencies from operations to the developer. This has a number of advantages:

• Developers are better placed than operations to maintain dependencies for the software that they write
• Whilst operations will always need to maintain approved Linux distributions and dependencies for some things, they are largely unburdened from this for new software
• Shipping an application with its dependencies saves pain in production, such as missing or incorrect versions of shared libraries
• Huge improvements in dev-test feedback loop

As a result we can move faster, getting features in our customers’ hands sooner. Docker helps to unlock the kind of pace at which modern software companies need to move in order to be competitive.

Standards in the Container Ecosystem

The main standards bodies around containers at the moment are Appc, the Open Container Initiative (OCI) and the Cloud Native Computing Foundation (CCNF). Appc was announced by CoreOS along with rkt. OCI was announced at DockerConEU 2015 and included the founders of Appc as founding members. The CCNF is focused more at the orchestration level, addressing the issues of building containerised, microservice-based distributed systems at scale. Out of the work of Appc and OCI we’ve not yet seen truly interoperable, widely adopted standards; there has been some progress, but it’s been slow and inter-stakeholder politics is evident. Let’s hope this changes going forward.

Container Runtimes, Tooling and Services Round-Up

Container Runtimes

Docker is clearly the most important container runtime available today. There are a few others popping up, including rkt from CoreOS and LXD from Canonical. LXD is unlikely to gain much traction outside of Canonical’s sphere of influence (unless Microsoft and Canonical have more surprises in store for us — LXD on Windows perhaps?) but with rkt soon to be fully supported in Kubernetes, and increasingly popular as a suitable runtime for building platforms, it’s one to watch. It’s likely we’ll see more container runtimes from other big companies.

Container Platforms

The Docker Platform — We talk about Docker as a container runtime, but the reality is that it is increasingly more of a platform. Docker wants complete vertical control over your container stack and it doesn’t play well with OS init systems such as systemd. The Docker platform is made up of the following suite of products:

Docker Machine — a tool for creating VMs with Docker installed, and configuring them with Swarm. It is also used to set environment variables to point your CLI client to different Docker APIs (e.g. on a VM).

Docker Compose — known as Fig in the early days of Docker, Docker Compose is a tool for describing and launching applications made up of a number of containers that need to speak to each other.

Docker Swarm — Swarm is Docker’s clustering and orchestration solution. You can think of Swarm as a proxy for the Docker API that sits in front of a number of Docker engine APIs, and the underlying clustering mechanism.

Docker Toolbox — Toolbox is a handy Windows & Mac installer to help you get started with Docker Machine, Compose and Swarm. You don’t need it for Linux.

Docker Cloud — born out of the acquisition of Tutum, Docker Cloud gives IT Ops teams the ability to manage and deploy their Dockerised distributed applications. Works with existing IaaS providers as well as on-prem.

Docker Universal Control Plane — Docker’s home-grown web UI for managing Docker hosts and deploying containers, as part of Docker Datacenter.

Docker Trusted Registry — store your Docker images on-premise using this private registry.

Docker Datacenter — this is Docker’s complete on-prem package, including both Universal Control Plane and Trusted Registry.

The core platform comprised of Docker Engine, Compose and Swarm are what we can compare to other platforms such as Kubernetes.

Kubernetes — The big name in container platforms is Google’s open-source Kubernetes project. Modeled on what Google have learned from running containers in production for years, Kubernetes presents a cluster of machines as a single system to simplify container operations. A number of higher-level systems have been built on top of it, including Google Container Engine, Tectonic from CoreOS and container PaaS Deis.

Apache Mesos — Mesos describes itself as a distributed systems kernel. By abstracting individual machine resources it presents a scalable and powerful data-centre abstraction that can run many kinds of jobs, containers being just one of them.

Hosted Container Services

GCE — Newer than Amazon’s offering, but far more feature-rich and powerful. GCE is essentially Google-hosted Kubernetes.

Amazon ECS — Mature, easy to use, yet rudimentary, ECS is suitable for simple services and for teams that are relatively new to containers.

Container-Based Operating Systems

CoreOS — The original bare-bones, built-for-containers operating system, and still miles ahead of the competition. The team at CoreOS are behind an impressive array of projects- Etcd, rkt, Clair, Fleet, Flannel and Quay.io.

There are now a few other container-based operating systems. Project Atomic, sponsored by RedHat, brings familiar tools like RPM, not to mention a name people know and trust; RancherOS takes the “minimalist” concept to the extreme, containerising pretty much everything in the OS; Canonical and VMWare have also joined the party with Snappy and Photon, respectively. There is also the Data Center Operating System from Mesosphere, which is quite a different beast.

Networking, Monitoring and Storage

The container community has spawned uncountable projects and products in order to fill in the gaps in what is still relatively rudimentary in terms of a complete user experience. At the forefront of these are networking and monitoring solutions from Weaveworks, storage solutions from ClusterHQ and monitoring tools from just about everyone, with Sysdig perhaps being the most exciting.

See You in San Francisco on the 15th of April!

Just when you think you’re getting your head around the container ecosystem, a slew of new projects are announced. We’ve still not reached peak confusion in containers.

With talks from Google, Docker, Joyent, CoreOS, Deis, Netflix and more, what better way to join in the discussion and keep up to date than by attending Container Camp? It’s not too late to pick up a ticket. Be sure to use our promo code “YLDIO” for $100 off until April 8th! I’m especially looking forward to the keynote from Alex Williams.

Have questions about running containers in production? Just love talking about containers? YLD have been running containers in production since 2014. Come and say hi to me, Tom and Andy and we’d be delighted to have a chat.