With my open source work, I’ve historically taken an approach which
relies more on integration testing than unit testing, but with some
of my newer projects, I’ve tried adopting principles from $paidwork
and applying them to my free software.
This is a quick run-down of how I’m structuring my test suite in
newer projects. It’s likely that many of my existing projects will
never adopt this structure, but some may.
This is not a monad tutorial. You do not need to read this,
especially if you’re new to Haskell. Do something more useful with
your time. But if you will not be satisfied until you understand the
meme words, let’s proceed. I’ll assume knowledge of categories,
functors, and natural transformations.
I’ve put in my fair share of time with PostgreSQL. It has no
shortage of functionality and I believe its expansive feature set
drove a lot of its growing popularity in the 2010s. As my time with
the database has worn on I’ve come up with a handful of features
that I wish the database had. Some of these are ideas borrowed from
other systems, some are inventions of my own. The real tragedy is
that I simply don’t have the time to implement the patches myself.
Libraries like NumPy and Pandas let you switch data types, which
allows you to reduce memory usage. Switching from numpy.float64
(“double-precision” or 64-bit floats) to numpy.float32
(“single-precision” or 32-bit floats) cuts memory usage in half. But
it does so at a cost: float32 can only store a much smaller range
of numbers, with less precision.
So if you want to save memory, how do you use float32 without
distorting your results? Let’s find out!
Pilots and programmers both face similar challenges when it comes to
dealing with critical issues mid-flight or mid-project. Both
professions require quick thinking, problem solving, and the ability
to manage stress in high-pressure situations.
In order to make an program written in Perl working on a computer
running Windows 10 I had to include several DLLs but which ones and
where were they located? Read about it in PAR
trouble. %
tags: [rust]
...
Rust’s Ugly Syntax
People complain about Rust syntax. I think that most of the time
when people think they have an issue with Rust’s syntax, they
actually object to Rust’s semantics. In this slightly whimsical
post, I’ll try to disentangle the two.
Over the course of the last few articles, we’ve investigated how
Git calculates diffs between different
versions of a file, first looking at the Myers diff
algorithm
and then its linear space
variant. As
a programmer, you are probably most familiar with diffs as a way for
you to see what’s changed; you see them whenever you run the git
diff or git show commands, or when you use GitHub’s compare
view or read a pull request. But diffs aren’t only for human
consumption, although their design is strongly influenced by human
expectations. They also form an ingredient in other computations
that Git has to perform, the most common one being merging.
Rust does a really good job of being orthogonal in many ways. While
it isn’t a tiny language, that things fit together nicely is one of
the things that can make it feel a lot smaller, in my
opinion. There’s one thing, though, that is really a bit
inconsistent today, and that’s the syntax and semantics of
functions.
The Rust team is happy to announce a new version of Rust,
1.67.0. Rust is a programming language empowering everyone to build
reliable and efficient software.
Haskellers (and even worse, dependent type brained people) are prone
to making complicated types to maximise abstraction and
correctness. However, you might be surprised to know that you
(technically) need nothing more than ADTs, HKTs, and rank-N types to
do almost all of the magic you need.
Follows a new couple and their families, who find themselves
examining modern love and family dynamics amidst clashing cultures,
societal expectations and generational differences.
In the evening Esme and I watched You
People. The movie was OK, and
I give it a 6 out of 10.
As I've written about
before, a
PostgreSQL superuser always has the ability to take over the
operating system account in which PostgreSQL is running, but
sometimes you'd
like
to have a
role
that can administer the database but not break out of it. In
existing releases, there's no good way to accomplish that. You can
either make a new role so weak that it can't perform ordinary
administration tasks, or you can make it so strong that it can
easily break into the operating system account and thus take over
the superuser role as well. Unless you hack the source code, which
some people have done, there's no real way to set up an account that
has enough power to usefully administer the database in meaningful
ways but yet not enough power to take over everything. I've
committed a number of patches to v16 to try to improve the
situation, and I think that we can look forward to big improvements
in this area once it is released.
I’m currently studying algorithms and coding interview exercises and
I came upon an exercise to implement Insertion
Sort. This seems like
a an opportunity to try out generics !
It's directly tied to your versioned code. Referenced by a git
commit. But it doesn't fit in git. Parts of the development workflow
that ideally would be in version control but aren't because of the
design of git.
A world-weary detective is hired to investigate the murder of a West
Point cadet. Stymied by the cadets' code of silence, he enlists one
of their own to help unravel the case - a young man the world would
come to know as Edgar Allan Poe.
In the evening I watched The Pale Blue
Eye. I liked the movie and
give it a 7 out of 10.
While its relatively easy (or brain dead easy with GhostBSD or
NomadBSD distributions) to install and configure a FreeBSD
Desktop – one
have to keep in mind that its also important to keep that system
updated and secure.
In this
post
Peter describes an effective, graceful way of differentiating
between unit and integration tests in a Go codebase. While build
tags and the standard tooling around tests might be Go related, I
believe that the pattern applies to other ecosystems as well. For
instance Python and Pytest.
Git-Sim is a command-line tool written in Python that allows Git
users to quickly and easily generate images (or even video
animations) illustrating the impact of a Git command will have on
the local repo.
While developing mirrord, which heavily relies on injecting itself
into other people’s binaries, we ran into some challenges posed by
macOS’s SIP (System Integrity Protection). This post details how we
ultimately overcame these challenges, and we hope it can be of help
to other people hoping to learn about SIP, as we’ve learned the hard
way that there’s very little written about this subject on the
internet.
First surprise: some Postgres drivers (eg pgx in Go) use prepared
statements under the hood, without the user explicitely opting in.
Second surprise: in Postgres, queries executed via a prepared
statement can have a different query plan than if executed
directly. The query planner can choose to use a “generic” query plan
that ignores the value of the prepared statement’s parameters,
potentially resulting in a very inefficient query plan for the
actual values you send.
Third surprise: Postgres’ query planner has a bug in its cost
estimation.
Consequence: we had a query that was particularly slow in
production, but it was very hard to reproduce as we didn’t know our
ORM was using prepared statements under the hood and nothing was
looking problematic with a simple EXPLAIN ANALYZE.
Last
time
I wrote about how
Python’s1
type system and syntax is now flexible enough to represent and
utilise algebraic data types ergonomically. Here, I’ll develop that
idea further by way of a motivating example, for which I shall make
use of some functional programming “tricks” to arrive at an
efficient Python implementation.
We at Determinate Systems are
pleased to announce our initial release of Zero to
Nix, a brand new learning resource for
Nix. You can check it out at
zero-to-nix.com. Zero to Nix is mostly
targeted at two groups:
People who are curious about Nix but haven’t yet taken the time
to really explore it (perhaps because they’ve felt intimated by
it).
People who are pretty sure they would benefit from Nix and have
tried to cross the chasm to using it in their daily workflows
but haven’t gotten there yet.
The title of this post is tongue-in-cheek; nixos-rebuild is a tool
that has been around for a long time and there’s nothing new about
it. However, I believe that not enough people know how capable this
tool is for building and deploying remote NixOS systems. In
other words, nixos-rebuild is actually a decent alternative to
tools like morph or
colmena.
In this post I’ll present a few git commands and operations I run
or have run on occasion, in no particular order, that the majority
of git users out there might not ever need.
In training classes, one of the things I always say (besides “Ask
questions!”) is “Always look at the first error message, not the
last one.” It’s so tempting to look at the last line of the
compiler’s output first: after all, it’s the one that’s right there
in front of you when the compiler finishes running. You think,
“Well, I need to fix all the errors at some point anyway; might as
well start here.” It’s a trap! Suppress that temptation, and
scroll up to the first error message first.
Everyone is talking about
Polars. Polars is
supposed to replace Pandas. Will it? Maybe 10 years from now. You
can’t untangle Pandas from everywhere it exists overnight. Do you
still want to replace Pandas with Polars and be one of the cool
kids? Ok. Let’s take a look at a practical guide to replacing Pandas
with Polars, comparing functionally used by most people.
The immense breadth of Java and its ecosystem, having grown and
matured over nearly three decades, can also be challenging though
for folks just starting their careers as a Java developer. Which
Java version should you use? How to install it? Which build tool and
IDE are the right ones? For all these, and many other questions,
there are typically a number of options, which can easily overwhelm
you if you are new to Java. As the platform has evolved, tools have
come and gone, things which were hugely popular years ago have
fallen into obsolescence since then. As related information can
still be found on the internet, it can be hard to identify what’s
still relevant and what not.
The idea for this blog post is to provide an opinionated guide for
folks getting started with Java development in 2023, helping you
with your very first steps with that amazing platform. Note I’m not
saying that the things I’m going to recommend are the best ones for
each and every situation. The focus is on providing a good
getting-started experience. Some of the recommended tools or
approaches may make less sense to use as you get more experienced
and other choices might be better suited for you then, based on the
specific situation and its requirements.
Also, very importantly, there is a notion of personal taste and
preference to these things, those are my recommendations, and
those of others might look different, which is perfectly fine.
40% of Rust
Developers
believe Rust's debugging experience could use improvement. And
that's not surprising: when we write code we make assumptions, and
sometimes we assume wrong. This leads to bugs, which we then track
down and fix. This is what we call "debugging", and is a core part
of programming. The purpose-built tools which help us with debugging
are called "debuggers".
Unlike the Rust compiler, the Rust project doesn't actually provide
a "Rust debugger". Users of Rust are instead expected to use a
third-party debugger such as gdb, lldb, or windbg to debug
their programs. And support for Rust in these debuggers is not
always great. Basic concepts such as "traits", "closures", and
"enums" may have limited support. And debugging async code, or
arbitrary user-defined data structures may be really hard if not
impossible. This limits the utility of debuggers, and in turn limits
the Rust user's debugging experience.
I was on the cusp of the Python 2→3 transition being the last class
in my grad school to learn Python 2, so I remember being put off by
Python 3. By then the transition had been going on for almost a
decade so a lot of libraries had already been updated. But the final
date for Python 2 was no where in sight. I was lucky that my
transition was relatively painless, but it wasn’t for a lot of
people. Lets take a look back and see how it started, but first a
little housekeeping.
When you think of animating CSS properties, which ones come to mind?
I recently started wondering about the ones that don’t come to mind
— properties that aren’t typically associated with animation, but
turn out to be animatable.
This articles explores some of the unexpected things that CSS can
animate and some nifty things you can do by animating them.
A middle-aged Chinese immigrant is swept up into an insane adventure
in which she alone can save existence by exploring other universes
and connecting with the lives she could have led.
If you need to speed up Python, Cython is a very useful tool. It
lets you seamlessly merge Python syntax with calls into C or C++
code, making it easy to write high-performance extensions with rich
Python interfaces.
That being said, Cython is not the best tool in all
circumstances. So in this article I’ll go over some of the
limitations and problems with Cython, and suggest some alternatives.
This got me thinking – but what about integers? Of course integers
have all kinds of problems too – anytime you represent a number in a
small fixed amount of space (like 8/16/32/64 bits), you’re going to
run into problems.
So I asked on Mastodon
again for examples
of integer problems and got all kinds of great responses again.
Lately I've been iterating on a reusable set of open-source
continuous integration (CI) tools for my Python projects. Here's an
overview, along with a sample GitHub Actions workflow file that runs
these checks.
People get confused by Rust’s build system a lot. I have been trying
for a while to figure out what makes Rust uniquely hard here, as a
lot of the people who are confused are experienced compiler
engineers who have used staged compilers in the past. Here are some
theories I have.
Ever since I got involved with open-source Python projects,
tox has been vital for testing packages across
Python versions (and other factors). However, lately, I’ve been
increasingly using Nox for my projects
instead. Since I’ve been asked why repeatedly, I’ll sum up my
thoughts.
A police detective, a bank robber, and a high-power broker enter
high-stakes negotiations after the criminal's brilliant heist
spirals into a hostage situation.
In the evening I watched Inside
Man. I liked the movie a lot
and give it an 8 out of 10.
This blog post is meant to cover my experience going down the Rust
rabbit hole and tips I wish some learning resources could have
taught better. Personally, I cannot learn a new language from simply
watching youtube videos, but rather through seeking out solutions
for myself, making mistakes, and feeling humbled by the process.
You can often hear online that indexing into a slice, such as
my_slice[i] is slow in Rust and you should do something else
instead for performance.
The details, however, are murky. There’s little in the way of
benchmarks, and hardly any documentation on removing this overhead
without resorting to unsafe code.
So after optimizing a bunch of high-profile crates by removing
bounds checks (and also removing unsafe code from some of them), I
figured I should write down the results, and the techniques I
discovered.
Symmetric encryption is widely used today, and there are efficient
algorithms, some even implemented on hardware. Examples of symmetric
encryption algorithms are AES (Advanced Encryption Standard), 3DES,
ChaCha, Salsa, Twofish, Blowfish, and Serpent. In this type of
encryption, we use the same key to encrypt and decrypt messages
(therefore, if someone can send encrypted messages, he can decrypt
them as well).
For part two, I would like to introduce three changes to the
standard library that address problems that the community has been
thinking about and debating solutions to for years.
As the 20th century came to a close, ARM was on the precipice of
massive change. Under its first CEO, Robin Saxby, the company had
grown from 12 engineers in a barn to hundreds of employees and was
the preferred choice in RISC chips for the rapidly expanding mobile
market. But the mobile and computer worlds were starting to merge,
and the titans of the latter industry were not planning to surrender
to the upstarts of the former.