Plurrrr

While developing mirrord, which heavily relies on injecting itself into other people’s binaries, we ran into some challenges posed by macOS’s SIP (System Integrity Protection). This post details how we ultimately overcame these challenges, and we hope it can be of help to other people hoping to learn about SIP, as we’ve learned the hard way that there’s very little written about this subject on the internet.

First surprise: some Postgres drivers (eg pgx in Go) use prepared statements under the hood, without the user explicitely opting in.

Second surprise: in Postgres, queries executed via a prepared statement can have a different query plan than if executed directly. The query planner can choose to use a “generic” query plan that ignores the value of the prepared statement’s parameters, potentially resulting in a very inefficient query plan for the actual values you send.

Third surprise: Postgres’ query planner has a bug in its cost estimation.

Consequence: we had a query that was particularly slow in production, but it was very hard to reproduce as we didn’t know our ORM was using prepared statements under the hood and nothing was looking problematic with a simple EXPLAIN ANALYZE.

Last time I wrote about how Python’s¹ type system and syntax is now flexible enough to represent and utilise algebraic data types ergonomically. Here, I’ll develop that idea further by way of a motivating example, for which I shall make use of some functional programming “tricks” to arrive at an efficient Python implementation.