Perl Testing in 2023

With my open source work, I’ve historically taken an approach which relies more on integration testing than unit testing, but with some of my newer projects, I’ve tried adopting principles from $paidwork and applying them to my free software.

This is a quick run-down of how I’m structuring my test suite in newer projects. It’s likely that many of my existing projects will never adopt this structure, but some may.

Source: Perl Testing in 2023, an article by Toby Inkster.

Monoids in the Category of...

This is not a monad tutorial. You do not need to read this, especially if you’re new to Haskell. Do something more useful with your time. But if you will not be satisfied until you understand the meme words, let’s proceed. I’ll assume knowledge of categories, functors, and natural transformations.

Source: Monoids in the Category of..., an article by Jack Kelly.

Features I'd like in PostgreSQL

I’ve put in my fair share of time with PostgreSQL. It has no shortage of functionality and I believe its expansive feature set drove a lot of its growing popularity in the 2010s. As my time with the database has worn on I’ve come up with a handful of features that I wish the database had. Some of these are ideas borrowed from other systems, some are inventions of my own. The real tragedy is that I simply don’t have the time to implement the patches myself.

Source: Features I'd like in PostgreSQL.

float64 to float32: Saving memory without losing precision

Libraries like NumPy and Pandas let you switch data types, which allows you to reduce memory usage. Switching from numpy.float64 (“double-precision” or 64-bit floats) to numpy.float32 (“single-precision” or 32-bit floats) cuts memory usage in half. But it does so at a cost: float32 can only store a much smaller range of numbers, with less precision.

So if you want to save memory, how do you use float32 without distorting your results? Let’s find out!

Source: float64 to float32: Saving memory without losing precision, an article by Itamar Turner-Trauring.

Debugging like a pilot

Pilots and programmers both face similar challenges when it comes to dealing with critical issues mid-flight or mid-project. Both professions require quick thinking, problem solving, and the ability to manage stress in high-pressure situations.

Source: Debugging like a pilot.

PAR Trouble

In order to make an program written in Perl working on a computer running Windows 10 I had to include several DLLs but which ones and where were they located? Read about it in PAR trouble. %

tags: [rust] ...

Rust’s Ugly Syntax

People complain about Rust syntax. I think that most of the time when people think they have an issue with Rust’s syntax, they actually object to Rust’s semantics. In this slightly whimsical post, I’ll try to disentangle the two.

Source: Rust's Ugly Syntax, an article by Alex Kladov.

Merging with diff3

Over the course of the last few articles, we’ve investigated how Git calculates diffs between different versions of a file, first looking at the Myers diff algorithm and then its linear space variant. As a programmer, you are probably most familiar with diffs as a way for you to see what’s changed; you see them whenever you run the git diff or git show commands, or when you use GitHub’s compare view or read a pull request. But diffs aren’t only for human consumption, although their design is strongly influenced by human expectations. They also form an ingredient in other computations that Git has to perform, the most common one being merging.

Source: Merging with diff3, an article by James Coglan.

Too many words about Rust's function syntax

Rust does a really good job of being orthogonal in many ways. While it isn’t a tiny language, that things fit together nicely is one of the things that can make it feel a lot smaller, in my opinion. There’s one thing, though, that is really a bit inconsistent today, and that’s the syntax and semantics of functions.

Source: Too many words about Rust's function syntax, an article by Steve Klabnik.

All you need is higher kinded types

Haskellers (and even worse, dependent type brained people) are prone to making complicated types to maximise abstraction and correctness. However, you might be surprised to know that you (technically) need nothing more than ADTs, HKTs, and rank-N types to do almost all of the magic you need.

Source: All you need is higher kinded types, an article by Las Safin.

You People (2023)

Follows a new couple and their families, who find themselves examining modern love and family dynamics amidst clashing cultures, societal expectations and generational differences.

In the evening Esme and I watched You People. The movie was OK, and I give it a 6 out of 10.

Surviving Without A Superuser - Coming to v16

As I've written about before, a PostgreSQL superuser always has the ability to take over the operating system account in which PostgreSQL is running, but sometimes you'd like to have a role that can administer the database but not break out of it. In existing releases, there's no good way to accomplish that. You can either make a new role so weak that it can't perform ordinary administration tasks, or you can make it so strong that it can easily break into the operating system account and thus take over the superuser role as well. Unless you hack the source code, which some people have done, there's no real way to set up an account that has enough power to usefully administer the database in meaningful ways but yet not enough power to take over everything. I've committed a number of patches to v16 to try to improve the situation, and I think that we can look forward to big improvements in this area once it is released.

Source: Surviving Without A Superuser - Coming to v16, an article by Robert Haas.

What Doesn't Fit in Git

It's directly tied to your versioned code. Referenced by a git commit. But it doesn't fit in git. Parts of the development workflow that ideally would be in version control but aren't because of the design of git.

Source: What Doesn't Fit in Git, an article by Matt Rickard.

The Pale Blue Eye (2022)

A world-weary detective is hired to investigate the murder of a West Point cadet. Stymied by the cadets' code of silence, he enlists one of their own to help unravel the case - a young man the world would come to know as Edgar Allan Poe.

In the evening I watched The Pale Blue Eye. I liked the movie and give it a 7 out of 10.

Fun with macOS's SIP

While developing mirrord, which heavily relies on injecting itself into other people’s binaries, we ran into some challenges posed by macOS’s SIP (System Integrity Protection). This post details how we ultimately overcame these challenges, and we hope it can be of help to other people hoping to learn about SIP, as we’ve learned the hard way that there’s very little written about this subject on the internet.

Source: Fun with macOS's SIP, an article by Aviram Hassan and Tal Zwick.

Puzzling Postgres

First surprise: some Postgres drivers (eg pgx in Go) use prepared statements under the hood, without the user explicitely opting in.

Second surprise: in Postgres, queries executed via a prepared statement can have a different query plan than if executed directly. The query planner can choose to use a “generic” query plan that ignores the value of the prepared statement’s parameters, potentially resulting in a very inefficient query plan for the actual values you send.

Third surprise: Postgres’ query planner has a bug in its cost estimation.

Consequence: we had a query that was particularly slow in production, but it was very hard to reproduce as we didn’t know our ORM was using prepared statements under the hood and nothing was looking problematic with a simple EXPLAIN ANALYZE.

Source: Puzzling Postgres: a story of solving an unreproducible performance issue, an article by William Duclot.

Introducing Zero to Nix

We at Determinate Systems are pleased to announce our initial release of Zero to Nix, a brand new learning resource for Nix. You can check it out at Zero to Nix is mostly targeted at two groups:

  1. People who are curious about Nix but haven’t yet taken the time to really explore it (perhaps because they’ve felt intimated by it).
  2. People who are pretty sure they would benefit from Nix and have tried to cross the chasm to using it in their daily workflows but haven’t gotten there yet.

Source: Introducing Zero to Nix, an article by Luc Perkins.

Announcing nixos-rebuild: a "new" deployment tool for NixOS

The title of this post is tongue-in-cheek; nixos-rebuild is a tool that has been around for a long time and there’s nothing new about it. However, I believe that not enough people know how capable this tool is for building and deploying remote NixOS systems. In other words, nixos-rebuild is actually a decent alternative to tools like morph or colmena.

Source: Haskell for all: Announcing nixos-rebuild: a "new" deployment tool for NixOS, an article by Gabriella Gonzalez.

Always read the first error message first

In training classes, one of the things I always say (besides “Ask questions!”) is “Always look at the first error message, not the last one.” It’s so tempting to look at the last line of the compiler’s output first: after all, it’s the one that’s right there in front of you when the compiler finishes running. You think, “Well, I need to fix all the errors at some point anyway; might as well start here.” It’s a trap! Suppress that temptation, and scroll up to the first error message first.

Source: Always read the first error message first, an article by Arthur O’Dwyer.

Replacing Pandas with Polars. A Practical Guide

Everyone is talking about Polars. Polars is supposed to replace Pandas. Will it? Maybe 10 years from now. You can’t untangle Pandas from everywhere it exists overnight. Do you still want to replace Pandas with Polars and be one of the cool kids? Ok. Let’s take a look at a practical guide to replacing Pandas with Polars, comparing functionally used by most people.

Source: Replacing Pandas with Polars. A Practical Guide, an article by Daniel Beach.

Getting Started With Java Development in 2023

The immense breadth of Java and its ecosystem, having grown and matured over nearly three decades, can also be challenging though for folks just starting their careers as a Java developer. Which Java version should you use? How to install it? Which build tool and IDE are the right ones? For all these, and many other questions, there are typically a number of options, which can easily overwhelm you if you are new to Java. As the platform has evolved, tools have come and gone, things which were hugely popular years ago have fallen into obsolescence since then. As related information can still be found on the internet, it can be hard to identify what’s still relevant and what not.

The idea for this blog post is to provide an opinionated guide for folks getting started with Java development in 2023, helping you with your very first steps with that amazing platform. Note I’m not saying that the things I’m going to recommend are the best ones for each and every situation. The focus is on providing a good getting-started experience. Some of the recommended tools or approaches may make less sense to use as you get more experienced and other choices might be better suited for you then, based on the specific situation and its requirements.

Also, very importantly, there is a notion of personal taste and preference to these things, those are my recommendations, and those of others might look different, which is perfectly fine.

Source: Getting Started With Java Development in 2023 — An Opinionated Guide, an article by Gunnar Morling.

Thoughts on the Python packaging ecosystem

This post is a bit long, so here’s the key points I’m making:

  • The Python packaging ecosystem unintentionally became the type of competitive space that it is today.
  • The community needs to make an explicit decision if it should continue operating under the model that led to status quo.
  • Pick from N different tools that do N different things is a good model.
  • Pick from N ~equivalent choices is a really bad user experience.
  • Picking a default doesn’t make other approaches illegal.
  • Communication about the Python packaging ecosystem is fragmented, and we should improve that.

Source: Thoughts on the Python packaging ecosystem, an article by Pradyun Gedam.

Rust should own its debugger experience

40% of Rust Developers believe Rust's debugging experience could use improvement. And that's not surprising: when we write code we make assumptions, and sometimes we assume wrong. This leads to bugs, which we then track down and fix. This is what we call "debugging", and is a core part of programming. The purpose-built tools which help us with debugging are called "debuggers".

Unlike the Rust compiler, the Rust project doesn't actually provide a "Rust debugger". Users of Rust are instead expected to use a third-party debugger such as gdb, lldb, or windbg to debug their programs. And support for Rust in these debuggers is not always great. Basic concepts such as "traits", "closures", and "enums" may have limited support. And debugging async code, or arbitrary user-defined data structures may be really hard if not impossible. This limits the utility of debuggers, and in turn limits the Rust user's debugging experience.

Source: Rust should own its debugger experience, an article by Yoshua Wuyts.

Breaking the Snake: How Python went from 2 to 3

I was on the cusp of the Python 2→3 transition being the last class in my grad school to learn Python 2, so I remember being put off by Python 3. By then the transition had been going on for almost a decade so a lot of libraries had already been updated. But the final date for Python 2 was no where in sight. I was lucky that my transition was relatively painless, but it wasn’t for a lot of people. Lets take a look back and see how it started, but first a little housekeeping.

Source: Breaking the Snake: How Python went from 2 to 3, an article by Diego Crespo.

The Surprising Things That CSS Can Animate

When you think of animating CSS properties, which ones come to mind? I recently started wondering about the ones that don’t come to mind — properties that aren’t typically associated with animation, but turn out to be animatable.

This articles explores some of the unexpected things that CSS can animate and some nifty things you can do by animating them.

Source: The Surprising Things That CSS Can Animate, an article by Will Boyd.

Some reasons to avoid Cython

If you need to speed up Python, Cython is a very useful tool. It lets you seamlessly merge Python syntax with calls into C or C++ code, making it easy to write high-performance extensions with rich Python interfaces.

That being said, Cython is not the best tool in all circumstances. So in this article I’ll go over some of the limitations and problems with Cython, and suggest some alternatives.

Source: Some reasons to avoid Cython, an article by Itamar Turner-Trauring.

Examples of problems with integers

Hello! A few days back we talked about problems with floating point numbers.

This got me thinking – but what about integers? Of course integers have all kinds of problems too – anytime you represent a number in a small fixed amount of space (like 8/16/32/64 bits), you’re going to run into problems.

So I asked on Mastodon again for examples of integer problems and got all kinds of great responses again.

Source: Examples of problems with integers, an article by Julia Evans.

Why I Like Nox

Ever since I got involved with open-source Python projects, tox has been vital for testing packages across Python versions (and other factors). However, lately, I’ve been increasingly using Nox for my projects instead. Since I’ve been asked why repeatedly, I’ll sum up my thoughts.

Source: Why I Like Nox, an article by Hynek Schlawack.

Inside Man (2006)

A police detective, a bank robber, and a high-power broker enter high-stakes negotiations after the criminal's brilliant heist spirals into a hostage situation.

In the evening I watched Inside Man. I liked the movie a lot and give it an 8 out of 10.

Rust concepts I wish I learned earlier

This blog post is meant to cover my experience going down the Rust rabbit hole and tips I wish some learning resources could have taught better. Personally, I cannot learn a new language from simply watching youtube videos, but rather through seeking out solutions for myself, making mistakes, and feeling humbled by the process.

Source: Rust concepts I wish I learned earlier, an article by Raul Jordan.

How to avoid bounds checks in Rust (without unsafe!)

You can often hear online that indexing into a slice, such as my_slice[i] is slow in Rust and you should do something else instead for performance.

The details, however, are murky. There’s little in the way of benchmarks, and hardly any documentation on removing this overhead without resorting to unsafe code.

So after optimizing a bunch of high-profile crates by removing bounds checks (and also removing unsafe code from some of them), I figured I should write down the results, and the techniques I discovered.

Source: How to avoid bounds checks in Rust (without unsafe!), an article by Sergey "Shnatsel" Davidoff.

Symmetric encryption

Symmetric encryption is widely used today, and there are efficient algorithms, some even implemented on hardware. Examples of symmetric encryption algorithms are AES (Advanced Encryption Standard), 3DES, ChaCha, Salsa, Twofish, Blowfish, and Serpent. In this type of encryption, we use the same key to encrypt and decrypt messages (therefore, if someone can send encrypted messages, he can decrypt them as well).

Source: Symmetric encryption, an article by Diego Kingston.

A history of ARM, part 3: Coming full circle

As the 20th century came to a close, ARM was on the precipice of massive change. Under its first CEO, Robin Saxby, the company had grown from 12 engineers in a barn to hundreds of employees and was the preferred choice in RISC chips for the rapidly expanding mobile market. But the mobile and computer worlds were starting to merge, and the titans of the latter industry were not planning to surrender to the upstarts of the former.

Source: A history of ARM, part 3: Coming full circle, an article by Jeremy Reimer.