With my open source work, I’ve historically taken an approach which relies more on integration testing than unit testing, but with some of my newer projects, I’ve tried adopting principles from $paidwork and applying them to my free software.
This is a quick run-down of how I’m structuring my test suite in newer projects. It’s likely that many of my existing projects will never adopt this structure, but some may.
Source: Perl Testing in 2023, an article by Toby Inkster.
This is not a monad tutorial. You do not need to read this, especially if you’re new to Haskell. Do something more useful with your time. But if you will not be satisfied until you understand the meme words, let’s proceed. I’ll assume knowledge of categories, functors, and natural transformations.
Source: Monoids in the Category of..., an article by Jack Kelly.
I have recently discovered that Rust Futures, or rather,
async fncalls can lead to surprising performance problems if they are nested too deeply.
Source: The size of Rust Futures, an article by Arpad Borsos.
I’ve put in my fair share of time with PostgreSQL. It has no shortage of functionality and I believe its expansive feature set drove a lot of its growing popularity in the 2010s. As my time with the database has worn on I’ve come up with a handful of features that I wish the database had. Some of these are ideas borrowed from other systems, some are inventions of my own. The real tragedy is that I simply don’t have the time to implement the patches myself.
Source: Features I'd like in PostgreSQL.
Libraries like NumPy and Pandas let you switch data types, which allows you to reduce memory usage. Switching from
numpy.float64(“double-precision” or 64-bit floats) tonumpy.float32(“single-precision” or 32-bit floats) cuts memory usage in half. But it does so at a cost:float32can only store a much smaller range of numbers, with less precision.So if you want to save memory, how do you use
float32without distorting your results? Let’s find out!
Source: float64 to float32: Saving memory without losing precision, an article by Itamar Turner-Trauring.
Pilots and programmers both face similar challenges when it comes to dealing with critical issues mid-flight or mid-project. Both professions require quick thinking, problem solving, and the ability to manage stress in high-pressure situations.
Source: Debugging like a pilot.
A woman who raised herself in the marshes of the Deep South becomes a suspect in the murder of a man with whom she was once involved.
In the evening Alice, Esme, and I watched Where the Crawdads Sing. I liked the movie a lot and give it a 9 out of 10.
In order to make an program written in Perl working on a computer running Windows 10 I had to include several DLLs but which ones and where were they located? Read about it in PAR trouble. %
tags: [rust] ...
People complain about Rust syntax. I think that most of the time when people think they have an issue with Rust’s syntax, they actually object to Rust’s semantics. In this slightly whimsical post, I’ll try to disentangle the two.
Source: Rust's Ugly Syntax, an article by Alex Kladov.
Over the course of the last few articles, we’ve investigated how Git calculates diffs between different versions of a file, first looking at the Myers diff algorithm and then its linear space variant. As a programmer, you are probably most familiar with diffs as a way for you to see what’s changed; you see them whenever you run the git diff or git show commands, or when you use GitHub’s compare view or read a pull request. But diffs aren’t only for human consumption, although their design is strongly influenced by human expectations. They also form an ingredient in other computations that Git has to perform, the most common one being merging.
Source: Merging with diff3, an article by James Coglan.
The untold true story of the legendary detectives who brought down Bonnie and Clyde.
In the evening Alice and I watched The Highwaymen. I liked the movie and give it an 8 out of 10.
Rust does a really good job of being orthogonal in many ways. While it isn’t a tiny language, that things fit together nicely is one of the things that can make it feel a lot smaller, in my opinion. There’s one thing, though, that is really a bit inconsistent today, and that’s the syntax and semantics of functions.
Source: Too many words about Rust's function syntax, an article by Steve Klabnik.
The Rust team is happy to announce a new version of Rust, 1.67.0. Rust is a programming language empowering everyone to build reliable and efficient software.
Source: Announcing Rust 1.67.0.
Haskellers (and even worse, dependent type brained people) are prone to making complicated types to maximise abstraction and correctness. However, you might be surprised to know that you (technically) need nothing more than ADTs, HKTs, and rank-N types to do almost all of the magic you need.
Source: All you need is higher kinded types, an article by Las Safin.
Follows a new couple and their families, who find themselves examining modern love and family dynamics amidst clashing cultures, societal expectations and generational differences.
In the evening Esme and I watched You People. The movie was OK, and I give it a 6 out of 10.
As I've written about before, a PostgreSQL superuser always has the ability to take over the operating system account in which PostgreSQL is running, but sometimes you'd like to have a role that can administer the database but not break out of it. In existing releases, there's no good way to accomplish that. You can either make a new role so weak that it can't perform ordinary administration tasks, or you can make it so strong that it can easily break into the operating system account and thus take over the superuser role as well. Unless you hack the source code, which some people have done, there's no real way to set up an account that has enough power to usefully administer the database in meaningful ways but yet not enough power to take over everything. I've committed a number of patches to v16 to try to improve the situation, and I think that we can look forward to big improvements in this area once it is released.
Source: Surviving Without A Superuser - Coming to v16, an article by Robert Haas.
I’m currently studying algorithms and coding interview exercises and I came upon an exercise to implement Insertion Sort. This seems like a an opportunity to try out generics !
It's directly tied to your versioned code. Referenced by a git commit. But it doesn't fit in git. Parts of the development workflow that ideally would be in version control but aren't because of the design of git.
Source: What Doesn't Fit in Git, an article by Matt Rickard.
A world-weary detective is hired to investigate the murder of a West Point cadet. Stymied by the cadets' code of silence, he enlists one of their own to help unravel the case - a young man the world would come to know as Edgar Allan Poe.
In the evening I watched The Pale Blue Eye. I liked the movie and give it a 7 out of 10.
While its relatively easy (or brain dead easy with GhostBSD or NomadBSD distributions) to install and configure a FreeBSD Desktop – one have to keep in mind that its also important to keep that system updated and secure.
Source: Keep FreeBSD Desktop Updated.
In this post Peter describes an effective, graceful way of differentiating between unit and integration tests in a Go codebase. While build tags and the standard tooling around tests might be Go related, I believe that the pattern applies to other ecosystems as well. For instance Python and Pytest.
Source: Gate Integration Tests on Environment Variables, an article by Christian Ştefănescu.
Git-Sim is a command-line tool written in Python that allows Git users to quickly and easily generate images (or even video animations) illustrating the impact of a Git command will have on the local repo.
Source: git-sim: Visually simulate Git operations in your own repos, an article by Jacob Stopak.
While developing mirrord, which heavily relies on injecting itself into other people’s binaries, we ran into some challenges posed by macOS’s SIP (System Integrity Protection). This post details how we ultimately overcame these challenges, and we hope it can be of help to other people hoping to learn about SIP, as we’ve learned the hard way that there’s very little written about this subject on the internet.
Source: Fun with macOS's SIP, an article by Aviram Hassan and Tal Zwick.
First surprise: some Postgres drivers (eg
pgxin Go) use prepared statements under the hood, without the user explicitely opting in.Second surprise: in Postgres, queries executed via a prepared statement can have a different query plan than if executed directly. The query planner can choose to use a “generic” query plan that ignores the value of the prepared statement’s parameters, potentially resulting in a very inefficient query plan for the actual values you send.
Third surprise: Postgres’ query planner has a bug in its cost estimation.
Consequence: we had a query that was particularly slow in production, but it was very hard to reproduce as we didn’t know our ORM was using prepared statements under the hood and nothing was looking problematic with a simple
EXPLAIN ANALYZE.
Source: Puzzling Postgres: a story of solving an unreproducible performance issue, an article by William Duclot.
Last time I wrote about how Python’s1 type system and syntax is now flexible enough to represent and utilise algebraic data types ergonomically. Here, I’ll develop that idea further by way of a motivating example, for which I shall make use of some functional programming “tricks” to arrive at an efficient Python implementation.
Source: Functional Python, Part II: Dial M for Monoid, an article by Christopher Harrison.
We at Determinate Systems are pleased to announce our initial release of Zero to Nix, a brand new learning resource for Nix. You can check it out at zero-to-nix.com. Zero to Nix is mostly targeted at two groups:
- People who are curious about Nix but haven’t yet taken the time to really explore it (perhaps because they’ve felt intimated by it).
- People who are pretty sure they would benefit from Nix and have tried to cross the chasm to using it in their daily workflows but haven’t gotten there yet.
Source: Introducing Zero to Nix, an article by Luc Perkins.
The title of this post is tongue-in-cheek;
nixos-rebuildis a tool that has been around for a long time and there’s nothing new about it. However, I believe that not enough people know how capable this tool is for building and deploying remote NixOS systems. In other words,nixos-rebuildis actually a decent alternative to tools likemorphorcolmena.
Source: Haskell for all: Announcing nixos-rebuild: a "new" deployment tool for NixOS, an article by Gabriella Gonzalez.
In this post I’ll present a few
gitcommands and operations I run or have run on occasion, in no particular order, that the majority ofgitusers out there might not ever need.
Source: Git Commands You Probably Do Not Need, an article by Martin Myrseth.
In training classes, one of the things I always say (besides “Ask questions!”) is “Always look at the first error message, not the last one.” It’s so tempting to look at the last line of the compiler’s output first: after all, it’s the one that’s right there in front of you when the compiler finishes running. You think, “Well, I need to fix all the errors at some point anyway; might as well start here.” It’s a trap! Suppress that temptation, and scroll up to the first error message first.
Source: Always read the first error message first, an article by Arthur O’Dwyer.
Everyone is talking about Polars. Polars is supposed to replace Pandas. Will it? Maybe 10 years from now. You can’t untangle Pandas from everywhere it exists overnight. Do you still want to replace Pandas with Polars and be one of the cool kids? Ok. Let’s take a look at a practical guide to replacing Pandas with Polars, comparing functionally used by most people.
Source: Replacing Pandas with Polars. A Practical Guide, an article by Daniel Beach.
The immense breadth of Java and its ecosystem, having grown and matured over nearly three decades, can also be challenging though for folks just starting their careers as a Java developer. Which Java version should you use? How to install it? Which build tool and IDE are the right ones? For all these, and many other questions, there are typically a number of options, which can easily overwhelm you if you are new to Java. As the platform has evolved, tools have come and gone, things which were hugely popular years ago have fallen into obsolescence since then. As related information can still be found on the internet, it can be hard to identify what’s still relevant and what not.
The idea for this blog post is to provide an opinionated guide for folks getting started with Java development in 2023, helping you with your very first steps with that amazing platform. Note I’m not saying that the things I’m going to recommend are the best ones for each and every situation. The focus is on providing a good getting-started experience. Some of the recommended tools or approaches may make less sense to use as you get more experienced and other choices might be better suited for you then, based on the specific situation and its requirements.
Also, very importantly, there is a notion of personal taste and preference to these things, those are my recommendations, and those of others might look different, which is perfectly fine.
Source: Getting Started With Java Development in 2023 — An Opinionated Guide, an article by Gunnar Morling.
This post is a bit long, so here’s the key points I’m making:
- The Python packaging ecosystem unintentionally became the type of competitive space that it is today.
- The community needs to make an explicit decision if it should continue operating under the model that led to status quo.
- Pick from N different tools that do N different things is a good model.
- Pick from N ~equivalent choices is a really bad user experience.
- Picking a default doesn’t make other approaches illegal.
- Communication about the Python packaging ecosystem is fragmented, and we should improve that.
Source: Thoughts on the Python packaging ecosystem, an article by Pradyun Gedam.
40% of Rust Developers believe Rust's debugging experience could use improvement. And that's not surprising: when we write code we make assumptions, and sometimes we assume wrong. This leads to bugs, which we then track down and fix. This is what we call "debugging", and is a core part of programming. The purpose-built tools which help us with debugging are called "debuggers".
Unlike the Rust compiler, the Rust project doesn't actually provide a "Rust debugger". Users of Rust are instead expected to use a third-party debugger such as
gdb,lldb, orwindbgto debug their programs. And support for Rust in these debuggers is not always great. Basic concepts such as "traits", "closures", and "enums" may have limited support. And debugging async code, or arbitrary user-defined data structures may be really hard if not impossible. This limits the utility of debuggers, and in turn limits the Rust user's debugging experience.
Source: Rust should own its debugger experience, an article by Yoshua Wuyts.
I was on the cusp of the Python 2→3 transition being the last class in my grad school to learn Python 2, so I remember being put off by Python 3. By then the transition had been going on for almost a decade so a lot of libraries had already been updated. But the final date for Python 2 was no where in sight. I was lucky that my transition was relatively painless, but it wasn’t for a lot of people. Lets take a look back and see how it started, but first a little housekeeping.
Source: Breaking the Snake: How Python went from 2 to 3, an article by Diego Crespo.
When you think of animating CSS properties, which ones come to mind? I recently started wondering about the ones that don’t come to mind — properties that aren’t typically associated with animation, but turn out to be animatable.
This articles explores some of the unexpected things that CSS can animate and some nifty things you can do by animating them.
Source: The Surprising Things That CSS Can Animate, an article by Will Boyd.
A middle-aged Chinese immigrant is swept up into an insane adventure in which she alone can save existence by exploring other universes and connecting with the lives she could have led.
In the evening Esme and I watched Everything Everywhere All at Once. I liked the movie and give it an 8 out of 10.
If you need to speed up Python, Cython is a very useful tool. It lets you seamlessly merge Python syntax with calls into C or C++ code, making it easy to write high-performance extensions with rich Python interfaces.
That being said, Cython is not the best tool in all circumstances. So in this article I’ll go over some of the limitations and problems with Cython, and suggest some alternatives.
Source: Some reasons to avoid Cython, an article by Itamar Turner-Trauring.
Did you know, there's now a native way in JavaScript to do deep copies of objects?
That's right, this
structuredClonefunction is built into the JavaScript runtime.
Source: Deep Cloning Objects in JavaScript, the Modern Way, an article by Steve Sewell.
Hello! A few days back we talked about problems with floating point numbers.
This got me thinking – but what about integers? Of course integers have all kinds of problems too – anytime you represent a number in a small fixed amount of space (like 8/16/32/64 bits), you’re going to run into problems.
So I asked on Mastodon again for examples of integer problems and got all kinds of great responses again.
Source: Examples of problems with integers, an article by Julia Evans.
Lately I've been iterating on a reusable set of open-source continuous integration (CI) tools for my Python projects. Here's an overview, along with a sample GitHub Actions workflow file that runs these checks.
Source: An open source Python project CI pipeline, an article by Brenton Cleeland.
People get confused by Rust’s build system a lot. I have been trying for a while to figure out what makes Rust uniquely hard here, as a lot of the people who are confused are experienced compiler engineers who have used staged compilers in the past. Here are some theories I have.
Source: Why is Rust's build system uniquely hard to use?, an article by Joshua Nelson.
Ever since I got involved with open-source Python projects, tox has been vital for testing packages across Python versions (and other factors). However, lately, I’ve been increasingly using Nox for my projects instead. Since I’ve been asked why repeatedly, I’ll sum up my thoughts.
Source: Why I Like Nox, an article by Hynek Schlawack.
A police detective, a bank robber, and a high-power broker enter high-stakes negotiations after the criminal's brilliant heist spirals into a hostage situation.
In the evening I watched Inside Man. I liked the movie a lot and give it an 8 out of 10.
This blog post is meant to cover my experience going down the Rust rabbit hole and tips I wish some learning resources could have taught better. Personally, I cannot learn a new language from simply watching youtube videos, but rather through seeking out solutions for myself, making mistakes, and feeling humbled by the process.
Source: Rust concepts I wish I learned earlier, an article by Raul Jordan.
You can often hear online that indexing into a slice, such as
my_slice[i]is slow in Rust and you should do something else instead for performance.The details, however, are murky. There’s little in the way of benchmarks, and hardly any documentation on removing this overhead without resorting to
unsafecode.So after optimizing a bunch of high-profile crates by removing bounds checks (and also removing
unsafecode from some of them), I figured I should write down the results, and the techniques I discovered.
Source: How to avoid bounds checks in Rust (without unsafe!), an article by Sergey "Shnatsel" Davidoff.
Symmetric encryption is widely used today, and there are efficient algorithms, some even implemented on hardware. Examples of symmetric encryption algorithms are AES (Advanced Encryption Standard), 3DES, ChaCha, Salsa, Twofish, Blowfish, and Serpent. In this type of encryption, we use the same key to encrypt and decrypt messages (therefore, if someone can send encrypted messages, he can decrypt them as well).
Source: Symmetric encryption, an article by Diego Kingston.
A History of how the Unix Pipe command came together, and influenced programming.
Source: Pipe: How the System Call That Ties Unix Together Came About, an article by David Cassel.
For part two, I would like to introduce three changes to the standard library that address problems that the community has been thinking about and debating solutions to for years.
Source: What’s New in Go 1.20, Part II: Major Standard Library Changes, an article by Carl M. Johnson.
As the 20th century came to a close, ARM was on the precipice of massive change. Under its first CEO, Robin Saxby, the company had grown from 12 engineers in a barn to hundreds of employees and was the preferred choice in RISC chips for the rapidly expanding mobile market. But the mobile and computer worlds were starting to merge, and the titans of the latter industry were not planning to surrender to the upstarts of the former.
Source: A history of ARM, part 3: Coming full circle, an article by Jeremy Reimer.