Building Docker images can be slow, and Docker’s build system is also missing some critical security features, in particular the ability to use build secrets without leaking them. So over the past few years the Docker developers have been working on a new backend for building images, BuildKit. With the release of Docker 20.10 in late 2020, BuildKit is finally marked as stable–and you don’t need to upgrade to use it, you can use it with existing Docker 19.03 installs. […]

A segfaulting program might be the symptom of a bug in C code–or it might be that your process is running out of memory. Crashing is just one symptom of running out of memory. Your process might instead just run very slowly, your computer or VM might freeze, or your process might get silently killed. Sometimes if you’re lucky you might even get a nice traceback, but then again, you might not. So how do you identify out-of-memory problems? With […]

Docker uses layer caching to speed up builds, but layer caching isn’t always enough. When you’re rapidly developing your Python application and therefore frequently changing the list of dependencies, you’re going to end up downloading the same packages. Over and over and over again. This is no fun when you depend on small packages. It’s extra no fun when you’re downloading machine learning libraries that take hundreds of megabytes. With the release of a stable Docker BuildKit, Docker now supports […]

Python is not the fastest language around, so any performance boost helps, especially if you’re running at scale. It turns out that depending where you install Python from, its performance can vary quite a bit: choosing the wrong version of Python can cut your speed by 10-20%. Let’s look at some numbers. Comparing builds Python I ran three benchmarks from the pyperformance suite on four different builds of Python 3.9 (code is here): python:3.9-buster, the “official” Python Docker image. Ubuntu […]

New software releases can bring bug fixes, new features, and faster performance. For example, NumPy 1.20 added type annotations, and improved performance by using SIMD when possible. If you’re installing NumPy, you might want to install the newest version. Unfortunately, if you’re using an old version of pip, installing the latest version of a Python package might fail—or install in a slower, more complex way. Why? The combination of glibc versioning, the CentOS end-of-life schedule, and how pip installs packages. […]

You probably don’t want to be an asshole. Being an asshole, as Siderea’s classic essay The Asshole Filter points out, is about being transgressive, about violating social boundaries and rules. And so within the cultural norms of our society, most of us try to avoid being an asshole, by sticking to the expected social boundaries. In programming as in social life, there are boundaries we try not violate: we build software with abstractions, boundaries between the complexity beneath and the […]

If you run a security scanner on your Docker image, you might be in for a shock: often you’ll be warned of dozens of security vulnerabilities, even on the most up-to-date image. After the third or fourth time you get this result, you’ll start tuning the security scanner out. Eventually, you won’t pay attention to the security scanner at all—and you might end up missing a real security vulnerability that slipped through. This is not your fault: the problem is […]

No one enjoys waiting, and waiting for your software to build and tests to run isn’t fun either—in fact, it’s quite expensive. And if you’re building your Docker image in a CI system like GitHub Actions with ephemeral runners—where a new environment gets spinned up for every build—by default your builds are going to be extra slow. In particular, when you spin up a new VM with a new Docker instance, the cache is empty, so when you run the […]

Somebody is always wrong on the Internet, and bad Docker packaging advice is quite common. But one particular piece of advice keeps coming up, and it’s dangerous enough to merit its own article. In a whole bunch of places you will be told not to install security updates when building your Docker image. I’ve been submitting PRs to fix this, so it’s up in fewer places now. But previously this advice was given by the official Docker docs’ best practices […]