Plurrrr

home

Slow UPDATEs in PostgreSQL

I found that some query was dealing with a relatively small fraction of rows in the table, but the query execution was based on some regular index. Postgres indexes come with a lot of various flavors: besides the type of the index you may choose, there are multicolumn, functional, partial, and – since Postgres 11 – covering indexes. For the SELECT queries, I was oxptimizing, using a partial index was a natural choice. It improved the SELECT indeed, as expected. But it also made UPDATEs to the same table significantly slower!

This was a big surprise. I know that this will be a surprise to many of my readers too. Let's buckle up and travel thru a series of short experiments to explore how this may happen: we'll check various index options looking at the performance of both SELECT and UPDATE queries, and in the end, we'll try to draw some useful conclusions, as well as discuss how we can build an optimization approach that would follow the "first, do no harm" principle.

Source: How partial, covering, and multicolumn indexes may slow down UPDATEs in PostgreSQL, an article by Nikolay Samokhvalov.

The secret of the macOS Monterey network quality tool

It seems that Apple has quietly added a new tool in macOS Monterey for measuring your device’s Internet connectivity quality. You can simply call the executable networkQuality, which executes the following tests:

  • Upload/download capacity (your Tx/Rx bandwidth essentially)
  • Upload/download flows, this seems to be the number of test packets used for the responsiveness tests
  • Upload/download responsiveness measured in Roundtrips Per Minute (RPM), which according to Apple, is the number of sequential round-trips, or transactions, a network can do in one minute under normal working conditions

Source: The secret of the macOS Monterey network quality tool, an article by Dan Petrov.

How Python list really works

This post is largely about the arrays — the #1 data structure in the world. If you are not a data structure guru yet, I guarantee that you will better understand Python lists, their advantages and limitations. If you already know everything — there is no harm in refreshing the key points.

Source: How Python list really works, an article by Anton Zhiyanov.

It's Now Possible To Sign Arbitrary Data With Your SSH Keys

Did you know that you can use the ssh-keygen command to sign and verify signatures on arbitrary data, like files and software releases? Although this feature isn't super new - it was added in 2019 with OpenSSH 8.0 - it seems to be little-known. That's a shame because it's super useful and the most viable alternative to PGP for signing data. If you're currently using PGP to sign data, you should consider switching to SSH signatures.

Source: It's Now Possible To Sign Arbitrary Data With Your SSH Keys, an article by Andrew Ayer.

Async Cancellation I

Sometimes we start things but decide midway through that we would prefer to rather not be doing them. That process is sometimes referred to as cancellation. Say we accidentally clicked "download" on a large file in the browser. We should have a way to tell the computer to stop downloading it.

Source: Async Cancellation I, an article by Yoshua Wuyts.

Ducklings

In the afternoon, on the way to our new house, Alice and I encountered a duck with four ducklings. Since I had a lunchbox with me with some bread; I had forgotten to take my lunch to work yesterday, we gave some to the ducks. I noticed that there where only four ducklings while I recalled there were five originally.

Five ducklings and their mother
Alice feeding the five ducklings and their mother.

When Alice and I continued our trip by bike we encountered the missing duckling calling for its siblings and mother. Alice tried to goad the duckling back to its family but it was very scared. Luckily we still had some bread left which worked better. And soon the duckling was reunited with its family.

Ready Player One (2018)

When the creator of a virtual reality called the OASIS dies, he makes a posthumous challenge to all OASIS users to find his Easter Egg, which will give the finder his fortune and control of his world.

In the evening we watched Ready Player One. While quite different from the book its based on I did like the movie; 7.5 out of 10.

Make your monorepo feel small with Git’s sparse index

One way that Git scales to the largest monorepos is the sparse-checkout feature, which allows you to focus on a subset of the files. This is supposed to make it feel like you are actually in a small repository, even though you are contributing to a large repository.

There’s only one problem: the Git index is still large in a monorepo, and users can feel it. Until now.

Source: Make your monorepo feel small with Git’s sparse index, an article by Derrick Stolee

Git Techniques at Risk Ledger

Using git can be daunting at first. Like my good friend Chris once said, "everyone knows the happy path but the minute it gets hairy we're all screwed".

I've decided to take a closer look at what git commands our engineering team uses on a day to day to get out of hairy situations.

Source: Git Techniques at Risk Ledger, an article by Bruno Calogero.

Twelve Years of Go

Today we celebrate the twelfth birthday of the Go open source release. We have had an eventful year and have a lot to look forward to next year.

Source: Twelve Years of Go, an article by Russ Cox.

Taming the Moose: Classing up Perl attributes

The lesson as always is to check CPAN (or the appropriate mix of your language’s software repository, forums like Stack Overflow, etc.) for anything that could conceivably have application outside of your particular circumstances. Twenty-five years into my career and I’m still leaping into code without first considering that someone smarter than me has already done the work.

Source: Taming the Moose: Classing up Perl attributes, an article by Mark Gardner.

Upcoming Features in Go 1.18

Go 1.18 will be a significant release of the programming language that will contain some major features that I’m excited about. The upcoming version is scheduled for early 2022. The first beta should be out in a month. Let’s take a look at some of the new features that will be available.

Source: Upcoming Features in Go 1.18, an article by Sebastian Holstein.

The Trouble with Peace

Peace is just another kind of battlefield . . . Savine dan Glokta, once Adua's most powerful investor, finds her judgement, fortune and reputation in tatters. But she still has all her ambitions, and no scruple will be permitted to stand in her way. For heroes like Leo dan Brock and Stour Nightfall, only happy with swords drawn, peace is an ordeal to end as soon as possible. But grievances must be nursed, power seized, and allies gathered first, while Rikke must master the power of the Long Eye . . . before it kills her. Unrest worms into every layer of society. The Breakers still lurk in the shadows, plotting to free the common man from his shackles, while noblemen bicker for their own advantage. Orso struggles to find a safe path through the maze of knives that is politics, only for his enemies, and his debts, to multiply. The old ways are swept aside, and the old leaders with them, but those who would seize the reins of power will find no alliance, no friendship, and no peace lasts forever.

In the evening I started in The Trouble with Peace, The Age of Madness book 2 by Joe Abercrombie.

Plugins in Rust: Reducing the Pain with Dependencies

The last article of this series covered how the plugin system could be implemented from scratch. This is a lot of work if you’re dealing with a relatively large codebase and therefore a complex interface in your plugin system, so let’s see how we can make our lives easier. I’ve been wanting to try abi_stable for this since the beginning, which was specifically created for plugins. But we aren’t really locked to that crate, so I’ll show other alternatives as well, which can even be combined to your liking.

Source: Plugins in Rust: Reducing the Pain with Dependencies.

Lesser Known PostgreSQL Features

Most of us are not aware of all the features in tools we use on a daily basis, especially if it's big and extensive like PostgreSQL. With PostgreSQL 14 released just a few weeks ago, what a better opportunity to shed a light on some lesser known features that already exist in PostgreSQL, but you may not know.

Source: Lesser Known PostgreSQL Features, an article by Haki Benita.

Emacs & Emojis: A ❤ Story

Emacs grew support for displaying colour emojis recently (and this is included in the release branch, which will become Emacs 28.1 in some months). This includes support for the grapheme cluster emojis (that consist of a number of Unicode code points, joined together with zero-width joiners and magic).

Source: Emacs & Emojis: A ❤ Story, an article by Lars Ingebrigtsen.

A terrible schema from a clueless programmer

The first time you encounter something, you're probably going to make some mistakes. There's a post going around tonight about how someone forgot to put an index on some database thing and wound up doing full table scans (or something like that). The rub is that instead of just being slow, it also cost a fair amount of money because this crazy vendor system charged by the row or somesuch. So, by scanning the whole table, they touched all of those rows, and oh hey, massive amounts of money just set ablaze!

The usual venues are discussing it, and I get the impression some people have the wrong approach to this. I want to describe a truly bad database schema I encountered, and then tell you a little about what it did to the system performance.

Source: A terrible schema from a clueless programmer.

It's Time to Get Hyped About Const Generics in Rust

One of the most interesting emergent properties of the Rust language has been the desire and ability to push more and more information into the type system. Other programming languages allow this, but Rust’s combination of speed, safety, and extremely powerful compile-time computation through both types and macros, have lead to a large ecosystem of highly capable libraries that do much of their work before the program ever runs.

Recent developments in the type system are making it easier than ever for programmers to take advantage of that power, and things are only going to get more interesting.

Source: It's Time to Get Hyped About Const Generics in Rust, an article by Leonora Tindall.

Guide of CPython’s Parser

The Parser in CPython is currently a PEG (Parser Expression Grammar) parser. The first version of the parser used to be an LL(1) based parser that was one of the oldest parts of CPython implemented before it was replaced by PEP 617. In particular, both the current parser and the old LL(1) parser are the output of a parser generator. This means that the way the parser is written is by feeding a description of the Grammar of the Python language to a special program (the parser generator) which outputs the parser. The way the Python language is changed is therefore by modifying the grammar file and developers rarely need to interact with the parser generator itself other than use it to generate the parser.

Source: Guide of CPython’s Parser, an article by Pablo Galindo Salgado.

Reducing Memory Allocations in Golang

Go’s place between C and Python in terms of abstraction and garbage collection memory management model has made it attractive to programmers looking for a fast but reasonably high level language. However, there is no free lunch. Go’s abstractions, especially with regards to allocation, come with a cost. This article will show ways to measure and reduce this cost.

Source: Reducing Memory Allocations in Golang, an article by Christopher Tarry.

Hypocritical Gophers

Most of my time over the past two years has been developing primary in Go. If you’re not familiar with the Go programming language let me sum it up for you real quick: take a modern, statically-linked, type-strict language and remove every advancement in development in the last 10 years or so, do some pretentious talks about how your language is the best one out there, and you have Go. Now, don’t get me wrong, Go is a fantastic language. The problem is that its also a horrible language.

Source: Hypocritical Gophers, an article by Peter Christian Fraedrich.

An oral history of Bank Python

Today will I take you through the keyhole to look at a group of software systems not well known to the public, which I call "Bank Python". Bank Python implementations are effectively proprietary forks of the entire Python ecosystem which are in use at many (but not all) of the biggest investment banks. Bank Python differs considerably from the common, or garden-variety Python that most people know and love (or hate).

Source: An oral history of Bank Python, an article by Cal Paterson.

A Haskell memory leak in way too much detail with Cachegrind

So, if you’re interested in low-level optimization of your Haskell programs then this article is for you. I’ll demonstrate the use of a valgrind tool, cachegrind, to inspect the behavior of the canonical leaky Haskell program: a lazy left fold. You’ll get three things: (1) a step-by-step tutorial on running cachegrind on your programs. (2) a section-by-section breakdown of the cachegrind profile, and (3) some guidance on interpreting the results.

Source: A Haskell memory leak in way too much detail with Cachegrind, an article by Jeffrey Young.

Async await in Swift explained with code examples

Async await is part of the new structured concurrency changes that arrived in Swift 5.5 during WWDC 2021. Concurrency in Swift means allowing multiple pieces of code to run at the same time. This is a very simplified description, but it should give you an idea already how important concurrency in Swift is for the performance of your apps. With the new async methods and await statements, we can define methods performing work asynchronously.

You might have read about the Swift Concurrency Manifesto by Chris Lattner before, which was announced a few years back. Many developers in the Swift community are excited about the future to come with a structured way of defining asynchronous code. Now that it’s finally here, we can simplify our code with async-await and make our asynchronous code easier to read.

Source: Async await in Swift explained with code examples, an article by Antoine van der Lee.

Goto

In this programmer’s opinion, you should use whichever constructs leads to the best code, factoring in maintainability and speed. And sometimes this might be goto!

Source: goto, an article by Brian “Beej Jorgensen” Hall.