a tumblelog

Extracting the list of O'Reilly Animals

O’Reilly lists almost all of the animals from their covers, even if “animals” is a bit of a loose term that encompasses “Catholic Priests” (Ethics of Big Data) or “Soldiers or rangers, with rifles” (SELinux). You can page through that list 20 results at a time, or search it. But, as with most lists I see online, I want to grab the entire list at once. Show me a paginated resources and I’ll show you the program I automated to unpaginate it.

Source: Extracting the list of O'Reilly Animals, an article by brian d foy.

When to use HTTPS for local development

Using http://localhost for local development is fine most of the time, except in some special cases. This post explains when you need to run your local development site with HTTPS.

Source: When to use HTTPS for local development, an article by Maud Nalpas.

Changing the color of an SVG icon

Today, for work, I had to change the color of an SVG icon from black to red (#ff0000) when a certain event occurred. I used the img element to display the SVG icon. Just adding a class with color set to the aforementioned hexadecimal color code didn't work. So I Googled and found an article called Change Color of SVG on Hover by Chris Coyier.

I decided to give the filter trick mentioned in this article a try and used the recommended online tool created by Barrett Sonntag which calculates the correct filter settings. When the class got added to the img element it indeed changed to red.

Use postgresql for authorization layer

Postgresql is my favorite relational database. There are a lot of cool things that Postgresql can do and one of those is to handle the authorization layer. Some people argue that putting authorization logic into the database makes the application code harder to read, and that’s actually true if there is only 1 code base connecting to a database. However, if there are more than 1 code bases communicating with a database, having a centralized authorization logic starts to make more sense. In this blog post, I am not going to discuss the pros and cons of having the authorization logic inside the database, but instead I am going to describe how I put all my authorization logic into postgresql.

Source: Use postgresql for authorization layer, an article by Tan Nguyen.

Tutorial: Troubleshooting Linux SSH Login Delay

I’ll start from a relatively simple problem - logging in to a server via SSH always takes 10 seconds. The delay seems to be pretty constant, there don’t seem to be major network problems and the server is not overloaded. Yet, remote logins always take 10 seconds.

If you’be been around, you probably already know a couple of likely causes for this, but I want to approach this problem systematically. How to troubleshoot such issues without relying on lucky guesses or having previous experience with usual suspects. You never know, next time the root cause may be different - or you have to troubleshoot a completely different application.

Source: Tutorial: Troubleshooting Linux SSH Login Delay - Why does logging in always take 10 seconds?, an article by Tanel Põder.

Monoids etc

Monoids are simpler than categories. A monoid is defined by a collection (set) of elements and an operation that allows us to combine two element and produce a third one of the same kind.

Source: Category Theory Illustrated - Monoids, an article by Boris Marinov

Command PATH security in Go

Today’s Go security release fixes an issue involving PATH lookups in untrusted directories that can lead to remote execution during the go get command. We expect people to have questions about what exactly this means and whether they might have issues in their own programs. This post details the bug, the fixes we have applied, how to decide whether your own programs are vulnerable to similar problems, and what you can do if they are.

Source: Command PATH security in Go, an article by Russ Cox.

Cgit, Nginx & Gitolite: A Personal Git Server

I've been on a "own my online presence" kick for more than a year now. So for this (overly protracted) essay, I thought I'd publish my notes on how I created my own Git server.

Source: Cgit, Nginx & Gitolite: A Personal Git Server, an article by Bryan Brattlof.

Raspberry Pi Enters Microcontroller Game With $4 Pico

Raspberry Pi was synonymous with single-board Linux computers. No longer. The $4 Raspberry Pi Pico board is their attempt to break into the crowded microcontroller module market.

The microcontroller in question, the RP2040, is also Raspberry Pi’s first foray into custom silicon, and it’s got a dual-core Cortex M0+ with luxurious amounts of SRAM and some very interesting custom I/O peripheral hardware that will likely mean that you never have to bit-bang again. But a bare microcontroller is no fun without a dev board, and the Raspberry Pi Pico adds 2 MB of flash, USB connectivity, and nice power management.

Source: Raspberry Pi Enters Microcontroller Game With $4 Pico, an article by Elliot Williams.

Floating Point Basics

What are floating-point numbers? When is it a bad idea to compare them? How precise are they? Let’s explore this.

Source: Floating Point Basics, an article by Hampus Wessman.

How We Ported Linux to the M1

So when Apple decided to allow installing custom kernels on the Macs with M1 processor, we were very happy to try building another Linux port to further our understanding of the hardware platform. As we were creating a model of the processor for our security research product, we were working on the Linux port in parallel.

Source: How We Ported Linux to the M1.

My Backup Strategy

Nerds love discussing their backup strategies, so I thought I give it a try.

Source: My Backup Strategy, an article by Craig Weber.

The author lists the following goals:

  1. Avoid permanent data loss.
  2. Avoid bit-rot.
  3. Maintain High-Availability.
  4. Maintain Security.

A Tour of Go 1.16's io/fs package

The upcoming Go 1.16 release has a lot of exciting updates in it, but my most anticipated addition to the Go standard library is the new io/fs and testing/testfs packages.

Go’s io.Reader, io.Writer, and os.File interfaces go a long way in abstracting common operations on opened files. However, until now there hasn’t been a great story for abstracting an entire filesystem.

Source: A Tour of Go 1.16's io/fs package, an article by Ben Congdon.

A Second-Hand Router and a New Gigabit Switch

Last Saturday my brother helped me with placing a wired ethernet connection in my home office. Somehow directly connecting my late 2014 Mac mini to the wall ethernet jack didn't work; no ethernet. In the evening he dropped a 4 port switch at my place. Using this 4 port switch to connect the Mac mini did work. When I tested the Internet speed I got 10Mbps so I suspected that the switch was nog a gigabit switch and ordered a TP-Link TL-SG108 with 8 ports yesterday.

8-port gigabit desktop switch TP-Link TL-SG108
8-port gigabit desktop switch TP-Link TL-SG108.

I also have a lot of networking issues with Wi-Fi and suspect the router, a TP-Link TL-WDR4300 N750 which I flashed with OpenWrt. So last Monday I browsed a Dutch website for second hand products; Marktplaats, and placed an offer of €15 on a TP-Link TL-WDR4300, the same model as I already owned since I know it works very well with OpenWrt. The offer was accepted and with shipping to my home the total was €22.25.

TP-Link TL-WDR4300 N750
Router TP-Link TL-WDR4300 N750.

Both items arrived today in the morning. I had also ordered two UTP cables with the desktop switch, one meter each, but those probably arrive tomorrow.

Today, when I did a speed test with the old switch in order to compare it with the new one I was surprised to get a speed way above 10Mbps. At times I even got close to 100Mbps. When I checked underneath the switch it stated it was a 1Gb switch 😫.

Speedtest result old router and old switch
Speedtest result old router and old switch.

When I tested the new switch I had the same problem as with the Mac Mini: the switch didn't detect a network on the incoming side. So now I suspect something is wrong with the cable that goes all the way down to the router, and my brother will soon drop by to have a look at it.

I also want to check if the router I ordered is going to make any difference with the wireless network connection.

Porting Firefox to Apple Silicon

The release of Apple Silicon-based Macs at the end of last year generated a flurry of news coverage and some surprises at the machine’s performance. This post details some background information on the experience of porting Firefox to run natively on these CPUs.

Source: Porting Firefox to Apple Silicon, an article by Gian-Carlo Pascutto.

Common kestrel

In the afternoon what I believe to be a common kestrel, Falco tinnunculus, landed on our fence. I was too slow with my iPhone 6S to take a photo of the bird.

Container networking is simple

Working with containers always feels like magic. In a good way for those who understand the internals and in a terrifying - for those who don't. Luckily, we've been looking under the hood of the containerization technology for quite some time already and even managed to uncover that containers are just isolated and restricted Linux processes, that images aren't really needed to run containers, and on the contrary - to build an image we need to run some containers.

Now comes a time to tackle the container networking problem. Or, more precisely, a single-host container networking problem. In this article, we are going to answer the following questions:

  • How to virtualize network resources to make containers think each of them has a dedicated network stack?
  • How to turn containers into friendly neighbors, prevent them from interfering, and teach to communicate well?
  • How to reach the outside world (e.g. the Internet) from inside the container?
  • How to reach containers running on a machine from the outside world (aka port publishing)?

Source: Container networking is simple, an article by Ivan Velichko.

Non-Blocking Parallelism for Services in Go

Go has plenty of useful builtin functionality for safe, concurrent and parallel code. However neat those features may be, they cannot write your program for you. As is the case for many languagges, the most important morsels of knowledge are not in the features of the language, but in the well-known patterns that compose those features into solutions that can address frequently reoccurring problems. I’m relatively new to using Go as my daily bread-and-butter language and recently encountered a useful pattern that I thought worth sharing. I’m told that at Palantir it is called the tickler pattern.

Source: Non-Blocking Parallelism for Services in Go, an article by Peter Goldsborough.

Again on 0-based vs. 1-based indexing

André Garzia made a nice blog post called “Lua, a misunderstood language” recently, and unfortunately (but perhaps unsurprisingly) a bulk of HN comments on it was about the age-old 0-based vs. 1-based indexing debate. You see, Lua uses 1-based indexing, and lots of programmers claimed this is unnatural because “every other language out there” uses 0-based indexing.

I’ll brush aside quickly the fact that this is not true — 1-based indexing has a long history, all the way from Fortran, COBOL, Pascal, Ada, Smalltalk, etc. — and I’ll grant that the vast majority of popular languages in the industry nowadays are 0-based. So, let’s avoid the popularity contest and address the claim that 0-based indexing is “inherently better”, or worse, “more natural”.

Source: Again on 0-based vs. 1-based indexing, an article by Hisham H. Muhammad.

That XOR Trick

There are a whole bunch of popular interview questions that can be solved in one of two ways: Either using common data structures and algorithms in a sensible manner, or by using some properties of XOR in a seemingly hard to understand way.

While it seems unreasonable to expect the XOR solutions in interviews, it is quite fun to figure out how they work. As it turns out, they are all based on the same fundamental trick, which we will derive in a bottom-up way in this post.

Source: That XOR Trick, an article by Florian Hartmann.

Unconditional loops are unconditionally awesome

Here’s a thing I don’t see appreciated enough about Rust: loop. I know I don’t think about it all that much, but pretty much every time I use it I feel a bit of satisfaction.

Source: Unconditional loops are unconditionally awesome, an article by Brian Anderson.

Three Reasons Fungi Are Not Plants

Today, we know that fungi are not plants, but the botanical history of fungi provides an interesting perspective on our scientific biases, on how we classify organisms and how these impact our collective knowledge.

Source: Three Reasons Fungi Are Not Plants, an article by Brian Lovett.

Big O Notation - explained as easily as possible

Data Structures and Algorithms is about solving problems efficiently. A bad programmer solves their problems inefficiently and a really bad programmer doesn't even know why their solution is inefficient. So, the question is, How do you rank an algorithm's efficiency?

The simple answer to that question is the Big O Notation. How does that work? Let me explain!

Source: Big O Notation - explained as easily as possible, an article by Conrad Reeves.

REST Servers in Go: Part 1 - standard library

This is the first post in a series about writing REST servers in Go. My plan with this series is to implement a simple REST server using several different approaches, which should make it easy to compare and contrast these approaches and their relative merits.

Developers who just start using a language often ask "what framework should I use to do X" as one of their first questions. While this makes total sense for web applications and servers in many languages, in Go the answer to this question is nuanced. There are strong opinions both for and against using frameworks. My goal in these posts is to examine the issue objectively from several angles.

Source: REST Servers in Go: Part 1 - standard library, an article by Eli Bendersky.

Creating Comfy FreeBSD Jails Using Standard Tools

Docker has stormed into software development in recent years. While the concepts behind it are powerful and useful, similar tools have been used in systems for decades. FreeBSD’s jails in one of those tools which build upon even older chroot(2) To put it shortly, with these tools, you can make a safe environment separated from the rest of the system.

Jails in FreeBSD is by no means a new tool (introduced in 4.X), but for a reason or another, I haven’t used them that often, which is a shame since they are so powerful. So I wanted to explore this concept in a concise and summarized manner.

Source: Creating Comfy FreeBSD Jails Using Standard Tools, an article by Topi Kettunen.

Running a fake power plant on the internet for a month

One of the systems often used to get more information about digital attackers are called honeypots. These mechanisms detect attempts at unauthorised use of computer systems. You could think of these as a digital version of bait cars used by the police to catch thieves. For this particular project I wrote a small HoneyTrap listener (an open-source project by DTACT) that can interact with systems scanning for devices on the s7comm protocol.

Source: Running a fake power plant on the internet for a month, an article by Stefan Grimminck.

Snow in the Netherlands

Late in the afternoon, after my brother had helped me with providing a wired internet connection in my home office, I went outside with Adam and Esme to have some fun in the snow. Alice didn't want to come, she had been outside earlier with Esme and Adam, but Adam couldn't get enough of the snow; he had been asking "When is it going to snow" ever since he arrived in the Netherlands. Well, today his dream came true.

Esme and Adam having a snowball fight
Esme and Adam having a snowball fight.
Snow on grass near the waterside
Snow on grass near the waterside.

After a snowball fight in and near a small playground close to our house we walked to another, larger playground. Esme and Adam worked together creating a small snowman.

Adam placing the head on the snowman
Adam placing the head on the snowman.
Adam posing with the snowman
Adam posing with the snowman.

Terminal Multiplexers

It's easy to take the terminal for granted. After all, modern terminal emulators like gnome-terminal, konsole, and the others included with Linux desktop environments are feature-rich applications that satisfy most of our needs. But sometimes we need more. We need to have multiple shell sessions running in a single terminal. We need to display more than one application in a single terminal. We need to move a running terminal session from one computer to another. In short, we need a terminal multiplexer.

Terminal multiplexers are programs that can perform these amazing feats. In this adventure, we will look at three examples: GNU screen, tmux, and byobu.

Source: Linux Command Line Adventure: Terminal Multiplexers an article by William E. Shotts, Jr.

Pipenv & Requests Author Kenneth Reitz Interview

Kenneth Reitz is a well-known software engineer, international keynote speaker, open-source advocate, who also focuses on photography and music production. He is well known for his many open-source projects, specifically Requests "simple, yet elegant HTTP library" and Pipenv Python Development Workflow for Humans. We are excited to have interviewed Kenneth! In this interview he talks about the top issues Python developers face while scaling applications, shares his thoughts on async / await paradigm in Python, speaks about his project PyTheory, and the "natural affinity" for software development.

Source: Interview with Kenneth Reitz, Pipenv & Requests libraries author by Evrone.

Running a CPU benchmark on Apple Silicon M1

When I started working with my Mac Mini M1, I felt it was faster, but I couldn’t really compare with a proper benchmark. I work a lot with video so I created an Apple (M1 and older) benchmark that is focused on CPU-heavy video programs: ffmpeg and primitive.

Source: Running a CPU benchmark on Apple Silicon M1, an article by Peter Forret.

Ask Ethan: Is Light Fundamentally A Wave Or A Particle?

One of the most bizarre aspects of quantum physics is that the fundamental entities that make up the Universe, what we know as the indivisible quanta of reality, behave as both a wave and a particle. We can do certain experiments, like firing photons at a sheet of metal, where they act like particles, interacting with the electrons and kicking them off only if they individually have enough energy. Other experiments, like firing photons at small thin objects — whether slits, hairs, holes, spheres, or even DVDs — give patterned results that show exclusively wave-like behavior. What we observe appears to depend on which observations we make, which is frustrating, to say the least. Is there some way to tell, fundamentally, what the nature of a quanta is, and whether it’s wave-like or particle-like at its core?

Source: Ask Ethan: Is Light Fundamentally A Wave Or A Particle?, an article by Ethan Siegel.

Big Sur boot volume layout

When you format a disk and install Big Sur on it, making it a startup or boot volume, it has a standard structure, something that has changed greatly over the last few years. This article summarises all you need to know about the layout of APFS boot disks and volumes in Big Sur, whether running on an Intel or Apple Silicon Mac. Detailed information for Catalina and earlier is here.

Source: Big Sur boot volume layout.

Emacs Tramp tricks

Emacs is amazing. It’s a very different sort of thing than a code text editor like Vim or an IDE like VSCode. It’s a different way of thinking of how to interact with a computer, where you build up techniques on top of simple tricks that let you get amazing things done. Of course, part of the appeal/challenge is that you need to figure out how to make it work yourself. Since there’s not much in the way of gentle tutorials, lets look at what you can do with tramp to get a sense of what is possible.

Source: Emacs Tramp tricks, an article by Will Schenk.

Distributing Mac apps outside the App Store, a quick start guide

The Mac has always been very different from its close relative, iOS, especially when it comes to what a user is or is not allowed to run on their system. Even with the introduction of Apple Silicon, Apple has made it very clear that the Mac is still the Mac, and is still hackable, even when running on the new architecture.

What this means for us developers is that, when targeting the Mac platform, we have choices: we can distribute our apps independently, outside the Mac App Store, through the Mac App Store exclusively, or through both at the same time.

This article is my brain dump on the subject. It is meant to be a guide on the things that you’ll need to know about when distributing a Mac app outside the App Store, rather than a how-to tutorial. My hope is that having everything listed here will help demystify the process for beginners, and the descriptions of my own process will be useful as starting points.

Source: Distributing Mac apps outside the App Store, a quick start guide, an article by Guilherme Rambo.