week 28, 2021

Inserting One Billion Rows in SQLite

Recently, I ran into a situation where I needed a test database with lots of rows and needed it fast. So I did what any programmer would do: wrote a Python script to generate the DB. Unfortunately, it was slow. Really slow. So I did what any programmer would do: went down the rabbit hole of learning more about SQLite, Python, and eventually Rust… in my quest to get a 1B row database under a minute. This blog post is a summary of this fun and educational exercise.

Source: Inserting One Billion Rows in SQLite Under A Minute.

The perils of M1 Ownership

In the next few days those using M1 Macs will be updating to Big Sur 11.5, blissfully ignorant of how, as an admin user, their Mac could refuse to update. Because now, in addition to regular users, admin users and root, there’s another class of admin user: the Owner. Let me explain.

Source: Last Week on My Mac: The perils of M1 Ownership, an article by Howard Oakley.

Useful Kotlin Idioms You Should Know

Kotlin was designed to be very similar to Java to make migration as smooth as possible. However, Kotlin was also designed to improve the developers’ experience by providing a more expressive syntax and a more sophisticated type system. To take full advantage of the language and write more concise code, learning Kotlin idioms is a must. Without them, it is easy to fall back into old Java patterns.

So, where do you begin? In this blog post, we would like to highlight some good places to start on the path to learning idiomatic Kotlin.

Source: Useful Kotlin Idioms You Should Know, an article by Anton Arhipov.

Sizing Up Types in Rust

When learning Rust, understanding the difference between statically and dynamically sized types seems critical. There are some good discussions out there already (e.g. here and here). Whilst these explain the mechanics, they didn’t tell me why its done like this in Rust. The articles made sense, but I was still confused! Eventually I had my “eureka” moment, so I figured I should share that.

Source: David J. Pearce (Sizing Up Types in Rust), an article by David J. Pearce.

Rebasing, work & feature branching

I'm a member of a few software teams and, very frequently, we commit code to our main branch. Common themes within my teams are not to have a develop branch or even release cycles. Instead, we release regularly.

This isn't common practice in big corporates but, within nimble software-focused companies, it's fairly commonplace.

The question is, how do we do this without stepping on each other's toes.

Source: Rebasing, work & feature branching, an article by Will Hackett.

time(1) and CPU frequency scaling

Is it wrong to measure processes using time(1)? In my particular case, it was wrong. But it really depends on what you want to know. You can also look at it like this: The fact that the user value increases and the frequency drops so much actually tells you that this might be a bit too much load for your machine.

Source: time(1) and CPU frequency scaling, an article by Peter Hofmann.

Unsafe Rust: How and when (not) to use it

In this guide, we’ll explain everything you need to know about unsafe Rust. We’ll focus on the following:

  • Myths about unsafe Rust
  • When not to write unsafe code
  • Dealing with uninitialized memory
  • Mutating the immutable
  • Intrinsic(s) motivation
  • Inline assembly
  • Interfacing foreign functions
  • Tools for writing unsafe Rust

Source: Unsafe Rust: How and when (not) to use it, an article by Andre Bogus.

Don't get bitten by missing foreign key indexes

Postgres doesn't give you indexes on foreign keys by default (unlike primary keys). This makes some sense when you think about it - there's no necessity for them in order to keep the constraint checking efficient.

In reality, you nearly always want an index on any foreign keys - every time you join tables on said key.

Source: foreign key indexes, an article by Oliver Russell.

Long Live Perl!

“Perl is for old farts,” some say.

Well, I definitely am that. I actually learned the language very deeply, in fact, I wrote the classes pragma, which is certainly a testament to the valid (albeit hypocritical) “read-only” criticism of the language. Looking back I realize any attempt to make Perl into an object-oriented language was frankly just missing the point—and original goal—of the language.

But, my young friends, Perl is still also for very astute systems and security coders who want a subtle, decisive edge over everyone else—be it in capture the flag, or in life. Let the uninformed diss the language—while you completely and utterly own them. True technologists evaluate the technology, not just the trend.

Source: Long Live Perl!. Still the Gold Standard, an article by Rob Muhlestein.

Juvenile Chromatopelma cyaneopubescens

In the morning, around 10, I took a few photos of the Chromatopelma cyaneopubescens I keep. The tarantula recently molted and based on its size I consider it a juvenile. It still doesn't have the mature blue / green colors its common name; the green bottle blue tarantula or GBB for short, is based on.

Close-up of a recently molted juvenile Chromatopelma cyaneopubescens
Close-up of a recently molted juvenile Chromatopelma cyaneopubescens.

How the Python import system works

If you ask me to name the most misunderstood aspect of Python, I will answer without a second thought: the Python import system. Just remember how many times you used relative imports and got something like ImportError: attempted relative import with no known parent package; or tried to figure out how to structure a project so that all the imports work correctly; or hacked sys.path when you couldn't find a better solution.

Source: Python behind the scenes #11: how the Python import system works, an article by Victor Skvortsov.

Let's write a Toy Emulator in Python

In this tutorial, we will build an Emulator for Ben Eater's 8-bit breadboard computer.

I'm a big fan of Ben Eater because of his video series on 8-bit breadboard computer and 6502 computer. If you haven't seen any of his videos, I recommend you to start with the 8-bit computer playlist. He builds everything from the ground up on a breadboard and teaches you about the internals and how they work.

In this tutorial we will be creating an emulator in python that can run programs such as printing Fibonacci series or Triangular numbers.

Source: Let's write a Toy Emulator in Python, an article by Jayachandra Kasarla.

Beating TimSort at Merging

Here is a problem. You are tasked with improving the hot loop of a Python program: maybe it is an in-memory sequential index of some sort. The slow part is the updating, where you are adding a new sorted list of items to the already sorted index. You need to combine two sorted lists and keep the result sorted. How do you do that update?

Source: Beating TimSort at Merging, an article by Adam Gordon Bell.

See also Past TimSort threads, for anyone interested, a Hacker News comment.

GitHub URL Hacks

GitHub’s UI has improved a lot over the years but sometimes you just need quick access without clicking. Here are a few GitHub URL tips to get you data you want faster. One cool thing is all of these tips give raw text output so they work great with curl and other CLI tools.

Source: GitHub URL Hacks, an article by Justin Garrison.

How to Agile

Ask 3 developers "what is agile" and you'll get four different answers. That's because there is no "definitive agile", and the Agile Manifesto which started it all is extremely loose (on purpose!).

This article will give you a brief background on agile, but more importantly it'll give you a few tips on how to apply it to your own team.

Source: How to Agile, an article by Paul Copplestone.

A Freshly Molted Brachypelma smithi

In the afternoon I noticed that the female Brachypelma smithi I keep had molted. I am not sure if this happened today or yesterday.

A freshly molted Brachypelma smithi
A freshly molted Brachypelma smithi.

In the early evening I took a photo of the inside of the exuviae (molt) in which the spermathecae are clearly visible making this tarantula a female.

Spermathecae of Brachypelma smithi
Spermathecae of Brachypelma smithi (center).

See also Brachypelma smithi: it's a girl.

The Tools and Techniques I Use to Build Simple Static Websites

I’m always looking for ways I can improve my skills and my workflow. I eagerly skim through tutorial articles and lists of little known tricks. I never know when someone will recommend a helpful plugin, a new hidden feature of a language I use, or some solution to a thorn in my workflow.

Now it’s my turn. In this post, I’ll break down the tools and plugins I use regularly, some handy code snippets, and my general process from design to deployment. Regardless of your skill level, my hope is you come away from this article with something useful you didn’t know before.

Source: The Tools and Techniques I Use to Build Simple Static Websites, an article by Matthew Hoelter.

Updating the Go Memory Model

The current Go language memory model was written in 2009, with minor updates since. It is clear that there are at least a few details that we should add to the current memory model, among them an explicit endorsement of race detectors and a clear statement of how the APIs in sync/atomic synchronize programs.

Source: Updating the Go Memory Model (Memory Models, Part 3), an article by Russ Cox.