In the afternoon I finished The Bone Code by Kathy Reichs. I liked the book a lot, a real page turner.
Schema migrations are tricky, especially when planned downtime is not an option. Changes need to be carefully designed to stay backwards compatible so that both the new and old schema are available simultaneously. They must also be designed to not interfere with any other queries, for example by locking up tables. Plenty has been written on how to this right but one pattern I haven’t seen mentioned often is using views.
Source: Zero-downtime schema migrations in Postgres using views, an article by Fabian Lindfors.
In this post, I show how Parquet can encode very large datasets in a small file footprint, and how we can achieve data throughput significantly exceeding disk IO bandwidth by exploiting parallelism (multithreading).
Source: Extreme IO performance with parallel Apache Parquet in Python, an article by Wes McKinney.
Sancia Grado is a thief, and a damn good one. And her latest target, a heavily guarded warehouse on Tevanne’s docks, is nothing her unique abilities can’t handle.
But unbeknownst to her, Sancia’s been sent to steal an artifact of unimaginable power, an object that could revolutionize the magical technology known as scriving. The Merchant Houses who control this magic—the art of using coded commands to imbue everyday objects with sentience—have already used it to transform Tevanne into a vast, remorseless capitalist machine. But if they can unlock the artifact’s secrets, they will rewrite the world itself to suit their aims.
Now someone in those Houses wants Sancia dead, and the artifact for themselves. And in the city of Tevanne, there’s nobody with the power to stop them.
To have a chance at surviving—and at stopping the deadly transformation that’s under way—Sancia will have to marshal unlikely allies, learn to harness the artifact’s power for herself, and undergo her own transformation, one that will turn her into something she could never have imagined.
In the evening I started in Foundryside The Founders Trilogy Book 1 by Robert Jackson Bennett.
What do developers mean when they say Postgres is “extensible”? They’re referring to low-level APIs which can change the core functionality of the database. In this post, we will explore a secret — meaning undocumented — feature called hooks which allow developers to not only add features to Postgres, but modify the way that queries are executed and data is parsed.
Source: Hooks: The secret feature powering the Postgres ecosystem, an article by Everett Berry.
To keep this lineup manageable, we only included books originally published as a bound book on paper—which ruled out classics like Journey to the West and One Thousand and One Nights—and ensured they were currently and easily available in print. We also limited the list to one book per author, to avoid crowding it with multiple titles in the same series.
Source: 50 Best Fantasy Books of All Time, an article by Adam Morgan.
The bodyguard, Michael Bryce, continues his friendship with assassin, Darius Kincaid, as they try to save Darius' wife Sonia.
In the evening Esme, Alice and I watched Hitman's Wife's Bodyguard. I didn't like the movie much and give it a 5 out of 10.
A test is code that executes code. When you start developing a new feature for your Python project, you could formalize its requirements as code. When you do so, you not only document the way your implementation’s code shall be used, but you can also run all the tests automatically to always make sure your code matches your requirements. One such tool which assists you in doing this is
pytestand it’s probably the most popular testing tool in the Python universe.
Source: A Gentle Introduction to Testing with PyTest, an article by Bas Steins.
Data auditing is a system that tracks changes to tables' contents over time. PostgreSQL has a robust set of features which we can leverage to create a generic auditing solution in 150 lines of SQL.
Source: Postgres Auditing in 150 lines of SQL, an article by Oliver Rice.
The time-based job scheduler
cron(8)has been around since Version 7 Unix, and its
crontab(5)syntax is familiar even for people who don’t do much Unix system administration. It’s standardised, reasonably flexible, simple to configure, and works reliably, and so it’s trusted by both system packages and users to manage many important tasks.
However, like many older Unix tools,
cron(8)‘s simplicity has a drawback: it relies upon the user to know some detail of how it works, and to correctly implement any other safety checking behaviour around it. Specifically, all it does is try and run the job at an appropriate time, and email the output. For simple and unimportant per-user jobs, that may be just fine, but for more crucial system tasks it’s worthwhile to wrap a little extra infrastructure around it and the tasks it calls.
There are a few ways to make the way you use
cron(8)more robust if you’re in a situation where keeping track of the running job is desirable.
Source: Cron best practices, an article by Tom Ryder.
I’ve been working through Leonard Susskind’s The Theoretical Minimum course, and one thing I’ve found interesting is the ubiquity of tensors - they seem to pop up everywhere in physics. I’ve been trying to build some intution behind what makes them so widely applicable, and I wanted to share my notes on this in the hopes that others might also find this useful. I’d also welcome any insights or corrections.
Source: Why Tensors? A Beginner's Perspective, an article by Faizan Muhammad.
I've used Linux containers directly and indirectly for years, but I wanted to become more familiar with them. So I wrote some code. This used to be 500 lines of code, I swear, but I've revised it some since publishing; I've ended up with about 70 lines more.
Source: Linux containers in 500 lines of code, an article by Lizzie Dixon.
A codemod is simply a script that refactors your source code automatically. The most basic codemod can be a “find & replace” script, but a more advanced codemod operates on the semantic structure of the code.
It takes some effort to write a codemod, but once you have it, you can find all instances of old patterns in the code and automatically update them to the new ones! This ensures we don’t have a mix of old and new patterns in the codebase, so old patterns won’t be propagated. On the off-chance an engineer commits code with an old pattern, you can simply run the codemod again to refactor those instances.
Source: Refactoring a Python Codebase with LibCST, an article by Adam Stepinski.
Tow-Boot is a relatively new project for bringing some consistency to the way we boot Linux on ARM. This is both aimed at the Linux single board computers but also the new wave of Linux phones.
Source: Booting ARM Linux the standard way, an article by Martijn Braam.
In the early evening I finished City of the Dead, an Alex Delaware novel by Jonathan Kellerman. I really liked the book, an excellent read.
On the way to hurricane-ravaged Isle of Palms, a barrier island off the South Carolina coast, Tempe receives a call from the Charleston coroner. The storm has tossed ashore a medical waste container. Inside are two decomposed bodies wrapped in plastic and bound with electrical wire. Tempe recognizes many of the details as identical to those of an unsolved case she handled in Quebec fifteen years earlier. With a growing sense of foreboding, she travels to Montreal to gather evidence.
Meanwhile, health authorities in South Carolina become increasingly alarmed as a human flesh-eating contagion spreads. So focused is Tempe on identifying the container victims that, initially, she doesn’t register how their murders and the pestilence may be related. But she does recognize one unsettling fact. Someone is protecting a dark secret—and willing to do anything to keep it hidden.
In the evening I started in The Bone Code by Kathy Reichs.
This is the third article in a series where I try to understand systemd by creating small containerized examples. In Part 1, we created a minimal systemd setup in a container. In Part 2 we took a close look at systemd’s dependency management. Now we are turning to systemd services. There is a lot to say about services, so I will split the information across the next few posts, starting in this post with the basics of defining a service unit.
Source: systemd by example - Part 3: Defining services, an article by Sebastian Jambor.
There are five prevailing viewpoints on the topic of testing private methods:
- Don’t Use Private Methods In The First Place
- Always Test Private Methods
- Never Test Private Methods
- Test Private Methods Sometimes
- Extract Private Methods Into A Separate Class
Source: Still No Consensus On Testing Private Methods, an article by Jesse Duffield
Digital Signature Algorithms (DSAs) underpin modern technology enabling phone calls, emails, operating system updates, and payments to process securely. Every DSA is built upon one way functions, which are is easy to perform one way but hard to reverse. Ed25519 is one such DSA and in this article I'll show how to use it.
Source: A Deep dive into Ed25519 Signatures.
In the afternoon I needed a working network in Ubuntu 20.04 recovery mode. This was as simple as entering the following line at the command prompt:
This is the story of CVE-2022-0847, a vulnerability in the Linux kernel since 5.8 which allows overwriting data in arbitrary read-only files. This leads to privilege escalation because unprivileged processes can inject code into root processes.
Source: The Dirty Pipe Vulnerability, an article by Max Kellermann.
Despite the fact that it is not a ‘real’ vulnerability, escaping privileged Docker containers is nevertheless pretty funny. And because there will always be people who will come up with reasons or excuses to run a privileged container (even though you really shouldn’t), this could really be handy at some point in the future.
Source: Escaping privileged containers for fun, an article by Jordy Zomer.