week 04, 2023

Features I'd like in PostgreSQL

I’ve put in my fair share of time with PostgreSQL. It has no shortage of functionality and I believe its expansive feature set drove a lot of its growing popularity in the 2010s. As my time with the database has worn on I’ve come up with a handful of features that I wish the database had. Some of these are ideas borrowed from other systems, some are inventions of my own. The real tragedy is that I simply don’t have the time to implement the patches myself.

Source: Features I'd like in PostgreSQL.

float64 to float32: Saving memory without losing precision

Libraries like NumPy and Pandas let you switch data types, which allows you to reduce memory usage. Switching from numpy.float64 (“double-precision” or 64-bit floats) to numpy.float32 (“single-precision” or 32-bit floats) cuts memory usage in half. But it does so at a cost: float32 can only store a much smaller range of numbers, with less precision.

So if you want to save memory, how do you use float32 without distorting your results? Let’s find out!

Source: float64 to float32: Saving memory without losing precision, an article by Itamar Turner-Trauring.

Debugging like a pilot

Pilots and programmers both face similar challenges when it comes to dealing with critical issues mid-flight or mid-project. Both professions require quick thinking, problem solving, and the ability to manage stress in high-pressure situations.

Source: Debugging like a pilot.

PAR Trouble

In order to make an program written in Perl working on a computer running Windows 10 I had to include several DLLs but which ones and where were they located? Read about it in PAR trouble. %

tags: [rust] ...

Rust’s Ugly Syntax

People complain about Rust syntax. I think that most of the time when people think they have an issue with Rust’s syntax, they actually object to Rust’s semantics. In this slightly whimsical post, I’ll try to disentangle the two.

Source: Rust's Ugly Syntax, an article by Alex Kladov.

Merging with diff3

Over the course of the last few articles, we’ve investigated how Git calculates diffs between different versions of a file, first looking at the Myers diff algorithm and then its linear space variant. As a programmer, you are probably most familiar with diffs as a way for you to see what’s changed; you see them whenever you run the git diff or git show commands, or when you use GitHub’s compare view or read a pull request. But diffs aren’t only for human consumption, although their design is strongly influenced by human expectations. They also form an ingredient in other computations that Git has to perform, the most common one being merging.

Source: Merging with diff3, an article by James Coglan.

Too many words about Rust's function syntax

Rust does a really good job of being orthogonal in many ways. While it isn’t a tiny language, that things fit together nicely is one of the things that can make it feel a lot smaller, in my opinion. There’s one thing, though, that is really a bit inconsistent today, and that’s the syntax and semantics of functions.

Source: Too many words about Rust's function syntax, an article by Steve Klabnik.

All you need is higher kinded types

Haskellers (and even worse, dependent type brained people) are prone to making complicated types to maximise abstraction and correctness. However, you might be surprised to know that you (technically) need nothing more than ADTs, HKTs, and rank-N types to do almost all of the magic you need.

Source: All you need is higher kinded types, an article by Las Safin.

You People (2023)

Follows a new couple and their families, who find themselves examining modern love and family dynamics amidst clashing cultures, societal expectations and generational differences.

In the evening Esme and I watched You People. The movie was OK, and I give it a 6 out of 10.

Surviving Without A Superuser - Coming to v16

As I've written about before, a PostgreSQL superuser always has the ability to take over the operating system account in which PostgreSQL is running, but sometimes you'd like to have a role that can administer the database but not break out of it. In existing releases, there's no good way to accomplish that. You can either make a new role so weak that it can't perform ordinary administration tasks, or you can make it so strong that it can easily break into the operating system account and thus take over the superuser role as well. Unless you hack the source code, which some people have done, there's no real way to set up an account that has enough power to usefully administer the database in meaningful ways but yet not enough power to take over everything. I've committed a number of patches to v16 to try to improve the situation, and I think that we can look forward to big improvements in this area once it is released.

Source: Surviving Without A Superuser - Coming to v16, an article by Robert Haas.

What Doesn't Fit in Git

It's directly tied to your versioned code. Referenced by a git commit. But it doesn't fit in git. Parts of the development workflow that ideally would be in version control but aren't because of the design of git.

Source: What Doesn't Fit in Git, an article by Matt Rickard.

The Pale Blue Eye (2022)

A world-weary detective is hired to investigate the murder of a West Point cadet. Stymied by the cadets' code of silence, he enlists one of their own to help unravel the case - a young man the world would come to know as Edgar Allan Poe.

In the evening I watched The Pale Blue Eye. I liked the movie and give it a 7 out of 10.

Fun with macOS's SIP

While developing mirrord, which heavily relies on injecting itself into other people’s binaries, we ran into some challenges posed by macOS’s SIP (System Integrity Protection). This post details how we ultimately overcame these challenges, and we hope it can be of help to other people hoping to learn about SIP, as we’ve learned the hard way that there’s very little written about this subject on the internet.

Source: Fun with macOS's SIP, an article by Aviram Hassan and Tal Zwick.

Puzzling Postgres

First surprise: some Postgres drivers (eg pgx in Go) use prepared statements under the hood, without the user explicitely opting in.

Second surprise: in Postgres, queries executed via a prepared statement can have a different query plan than if executed directly. The query planner can choose to use a “generic” query plan that ignores the value of the prepared statement’s parameters, potentially resulting in a very inefficient query plan for the actual values you send.

Third surprise: Postgres’ query planner has a bug in its cost estimation.

Consequence: we had a query that was particularly slow in production, but it was very hard to reproduce as we didn’t know our ORM was using prepared statements under the hood and nothing was looking problematic with a simple EXPLAIN ANALYZE.

Source: Puzzling Postgres: a story of solving an unreproducible performance issue, an article by William Duclot.

Introducing Zero to Nix

We at Determinate Systems are pleased to announce our initial release of Zero to Nix, a brand new learning resource for Nix. You can check it out at Zero to Nix is mostly targeted at two groups:

  1. People who are curious about Nix but haven’t yet taken the time to really explore it (perhaps because they’ve felt intimated by it).
  2. People who are pretty sure they would benefit from Nix and have tried to cross the chasm to using it in their daily workflows but haven’t gotten there yet.

Source: Introducing Zero to Nix, an article by Luc Perkins.

Announcing nixos-rebuild: a "new" deployment tool for NixOS

The title of this post is tongue-in-cheek; nixos-rebuild is a tool that has been around for a long time and there’s nothing new about it. However, I believe that not enough people know how capable this tool is for building and deploying remote NixOS systems. In other words, nixos-rebuild is actually a decent alternative to tools like morph or colmena.

Source: Haskell for all: Announcing nixos-rebuild: a "new" deployment tool for NixOS, an article by Gabriella Gonzalez.