Type I and type II errors happen when you erroneously spot winners in your experiments or fail to spot them. With both errors, you end up going with what appears to work or not. And not with the real results.
Source: Type I and Type II Errors: The Inevitable Errors in Optimization Experiments, an article by Disha Sharma.
This time, we are going to describe our journey of getting Haskell garbage collection times under control when dealing with heap sizes of up to 100 GB per instance. Ultimately, we managed to reduce the garbage collection time while parsing 320,000 product rows from 68s to 3s, reducing the total parsing time from 110s to 46s.
Source: Lessons in Managing Haskell Memory, an article by Fabian Thorand and Yorick Sijsling.
As you probably know, setting things up in a developer's environment is wild — you don't know what to expect. It's hard to set things up deterministically, and reproduce one environment into another. Homebrew for instance, tries to do as best as it can, but since it treats the environment as a global space in which dumping things, alike a singleton class that anyone can modify, that often results in hard-to-debug errors.
The first time that I heard about Nix was on this blog post from Pinterest, but it didn't catch my attention until now. I started reading about it and watching some internal videos that Burke is creating to evangelize the idea. The more I read about it, the more amazed I am with the idea.
Source: Diving into Nix, an article by Pedro Piñera.
In the morning I tested image upload in a phpBB forum I installed recently. I could upload a small photo as an attachment, but a larger one failed. So I configured the board settings to accept attachments of maximum 10MB. But now I got an HTTP error.
The error log of the Nginx web server showed a client intended to
send too large body message. To fix this, I logged in and changed to
the root account using /bin/su -. Next, I added the following line:
client_max_body_size 10M;
to the http section of /etc/nginx/nginx.conf and reloaded the
server using systemctl reload nginx.
Next, I changed one setting in /etc/php/7.3/fpm/php.ini and added
another one. Those settings are as follows:
upload_max_filesize=10M
post_max_size=10M
Finally, I restarted php7.3-fpm as follows:
systemctl restart php7.3-fpm
Note that I use php7.3-fpm on Ubuntu 18.04 LTS because the version
that (still) comes with it, PHP 7.2, comes with a php-xml package
that prevents phpBB version 3.3 from working correctly. See this how
to
for instructions.
I remember when mosh first came out and thinking "that sounds like a great idea, maybe I'll set it up sometime" and then never getting around to it. Several times over the years I've thought similarly, but it never seemed worth it. A week ago a coworker asked if there were good tools for handling remote terminal use over laggy internet connections, and I suggested mosh. They really liked it, which then had me feeling silly for never trying it, and got me to actually do it. So: I'm typing this over mosh, and it's excellent! I should have done this years ago.
Source: Mosh, an article by Jeff Kaufman.
Having a good database schema design is crucial for building applications that will need to scale in traffic and complexity. If you happen to do a bad design choice you will see that it takes a lot of effort to stop the pattern from propagating to your backends’ services and controllers, and finally to your frontend.
So there is a need to be able to evaluate if a database design is better than another.
But what does good mean when we are talking about design?
Source: A humble guide to database schema design, an article by Mike Alche.
WireGuard is a relatively new VPN tunnel protocol that aims to be very fast and easy to setup. It follows the Unix Philosophy closely in that it only does one thing (creating secured VPN tunnels) and does it well.
Source: Getting Started with WireGuard, an article by Miguel Mota.
In the evening I saw my Chromatopelma cyaneopubescens grooming itself on top of a piece of coco peat that I keep moist to provide a humid section to its otherwise dry container.
I used a dosing syringe to moisten the piece of coco peat a little more and saw the tarantula lower itself to the substrate; it was taking some of the moisture.
Soon after it returned to its "house": a few small twigs that it has covered in some webbing.
Last Saturday I ordered the following four tarantula's with Fatbob's Exotics:
Three of which were on the wishlist I made back in March.
I also ordered one scorpion:
Although the order added up to €25 and shipping was €7 I was asked to pay €31.
This morning the order, shipped on Monday, arrived. After some shopping for containers, cork tubes, fake plants, and coco peat I started the rehousing of the arachnids as all animals were shipped in small vials.
First, I prepared a terrarium for the Psalmopoeus irminia, an arboreal species, guided by a care sheet. Rehousing the small spider was easy. Next, I prepared a terrarium for the Heteroscodra maculata, another arboreal species. Again I used a care sheet for guidance. However, when I opened the vial to rehouse the tarantula I noticed that it was much smaller than the Psalmopoeus irminia, and for sure too small for the terrarium I just prepared. Luckily, I had a much smaller container, and used that one instead.
Next, I rehoused the Pterinochilus murinus into a terrarium I had prepared using a care sheet for guidance. When I carefully tried to move the tarantula with a soft brush it attacked the brush. A common name for this species is Orange Bitey Thing; a play on the common name Orange Baboon Tarantula (OBT) for this species.
For the final two arachnids I needed also a much smaller terrarium. So Esme and I went out again. First I went to a shop to buy two small cups with garlic sauce; the right size for the tiny Chaerilus sp. Java, a scorpion. We also bought a few additional containers in another shop.
For the scorpion I used some coco peat and a small piece of cork to provide a hiding place. For the Hapalopus sp Columbia I used once more a care sheet for guidance.
As Alice and Adam had collected a bunch of snails at my mother's place last Sunday, I helped Alice with preparing a nice terrarium for their collection.
Comparison operators are frequently used in JavaScript applications, and are an important part of controlling the logic flow of the app. But if you don’t have a good understanding of how they work you can run into a couple issues. Understanding strict comparisons and abstract comparisons is also really important, and in this article we’ll cover both of these comparisons.
Source: JavaScript Comparison Operators, an article by Preston Lamb.
A lot of web development is transforming JSON one way or another. In TypeScript/JavaScript this is straightforward, since JSON is built into the language. But can we also achieve good ergonomics in Haskell and Rust? Dear reader, I am glad you asked! 🙌
Source: Common JSON patterns in Haskell, Rust and TypeScript, an article by Christian Kjær Laustsen.
Nmap is probably one of the most known scanning tools. Nmap is a free and open-source network scanning tools created by Gordon Lyon. It’s used to scan hosts for open ports and running services. In the present post, you will have a great understanding of the most useful scanning commands implemented by Nmap. Let’s dive in.
But as your software grows, you start feeling the growing pains – your main function starts becoming more and more convoluted. You start having all sorts of small bits and pieces plugged in here and there – healthchecks, database setup code, metrics, tracers, external API connections, etc etc.
And what if your application grows into a microservice architecture? What do you do when you have five different microservices demanding the same bunch of setup code, specific to your environment?
In this article, I will introduce you to Fx. It’s a Go framework which solves both problems outlined above using dependency injection.
Source: Dependency Injection in Go using Fx, an article by Preslav Mihaylov.
Just before 1PM I gave a a mealworm, Tenebrio molitor, to my Brachypelma smithi. This was the second time it ate in my care, the previous time was last Saturday.
I probably schedule a weekly feeding for this little one.
In the early evening Adam and Alice collected ladybird beetles, Coccinella sp. in order to release them on the insect hotels we bought recently.
Several of the beetles ran away from the insect hotels and most likely won't stay.
John Gruber's Markdown is almost a perfect content authoring format. Here are 6 things it got wrong (in my opinion, of course).
Source: 6 Things Markdown Got Wrong.
Before noon, Esme and I went to garden centre "De Carlton", which is just a short ride from our house. I was not sure if the place wou ld be open but it was. I wanted to buy two orchids, Phalaenopsis sp. And after looking around for a while Esme fell in love with a large specimen of Monstera deliciosa.
When I found out it was only 5 euro to have the large plant delivered I decided to buy it as displayed standing inside a nice basket.
On our way back, Esme spotted a large branch. In "De Carlton" they have a large branch hanging with plants and lamps attached to it. She wants something like this above our dining table. Despite the branch being large and heavy she insisted on carrying it on her bike while walking.
Past 4pm the two orchids and the Monstera were delivered home. I took photos of the Monstera in its new location, and a photo of each of the orchids against a white background.
The branch Esme took is probably going to be a weekend project. I can't wait to see it in place.
For starters, viruses are easily the most abundant life form on Earth, if you accept the proposition that they're alive. Try multiplying a billion by a billion, then multiply that by ten trillion, and that (10 to the 31st power) is the mind-numbing estimate of how many individual viral particles are estimated to populate the planet.
Source: What's a virus, anyway? Part 1: The bare-bones basics, an article by Bruce Goldman.
A while ago, I took a project where the goal was to develop a Natual Language Processing (NLP) library that would support multiple pre-trained embedding methods (BERT, Elmo, Word2Vec, etc.) — similar to Hugging Face’s transformers but with more models and support for multiple languages. The library should take a document corpus on the input and transform it into an aggregated embedding for each document. The library should be developed in Python 3 and the goal was to integrate it into an online platform for Machine Learning. In the development process, I’ve made many mistakes and I am sharing it with you, so you don’t repeat them. Most importantly I learned a lot!
Source: Don’t repeat my mistakes while developing a Machine Learning library, an article by Roman Orac.
In the afternoon I rehoused my smallest tarantula, a Chromatopelma cyaneopubescens, from the plastic container it came in when I bought it to a much higher plastic container; 0.65 litre or 0.17 gallon. This species likes to web a lot and with a few twigs, which easily fit in the higher container, I could provide more anchor points.
In the above photo you can see the spider resting on top of some of its old webbing (left) which I transferred when I rehoused the tarantula. You can also see the 1mm holes I drilled for cross-ventilation. I also drilled four 1mm holes in the lid. The substrate to the right is kept slightly moist as this species prefers a dry climate. I used all substrate that came in the original container, which in this new container adds up to a depth of nearly 4cm or 1.57 inches.
SH is a powerful tool which often grants a lot of access to anyone using it to log into a server. In this post, I’m going to talk about a few different ways that you can easily improve the security of your SSH model without needing to deploy a new application or make any huge changes to user experience.
Source: How to SSH Properly, an article by Gus Luxton.
And so I decided to write this blog post to try and explain to setuptools users why
pyproject.tomlexists and what it does as it's the future of packaging in the Python ecosystem (if you are not a conda user 😉).
Source: What the heck is pyproject.toml?, an article by Brett Cannon.
The venerable “mixin” is a technique I learned as a Python developer. Now, after writing Ruby code for the past year, I’m excited to compare how these two languages approach mixins, including similarities, differences, and traps. There will be code!
Source: Mixins in Python and Ruby Compared, an article by Andrew Brookins.
Docker containers are not inherently secure as many people think. So, it’s crucial use tools and scanners to rid your containers of vulnerabilities…
Source: Analyzing Docker Image Security, an article by Martin Heinz.
After 5 years of using Go I am finally moving on. Go has served me well and has been the best language I could have possibly used for the longest time, but it is now the moment for me to let it Go.
Over time Go has not failed to show me its limitations and its issues, to the point where I decided to switch to something more future proof and with a more thriving community.
I don’t want to write this post as a list of things that pushed me away from my previous language, I find that kind of post sterile and of very little use for the readers, this is just a post on what I find great in JavaScript and what made me decide to switch.
Source: I Am Switching to JS.
Now that we've looked at what state machines are, and hopefully convinced you of their usefulness, let's take a look at how to implement them in Rust. Much of what I'm saying here was inspired by Hoverbear's "Rust state machine pattern" post, which I recommend reading.
Source: State Machines, an article by Yoshua Wuyts.
PostgreSQL has a rich set of indexing functionality, and there are many articles explaining the syntax, usage, and value of the index. In this article, I will write basic and useful queries to see the state of database indexes.
Source: Useful Queries For PostgreSQL Index Maintenance, an article by Ibrar Ahmed.
I have repeatedly been confounded to discover just how many mistakes in both test and application code stem from misunderstandings or misconceptions about time. By this I mean both the interesting way in which computers handle time, and the fundamental gotchas inherent in how we humans have constructed our calendar – daylight savings being just the tip of the iceberg.
In fact I have seen so many of these misconceptions crop up in other people’s (and my own) programs that I thought it would be worthwhile to collect a list of the more common problems here.
Source: Falsehoods programmers believe about time, an article by Noah Sussman.
Thus, I thought it might be helpful to lay down the heuristic that generates such answers. I by no means claim these are precise or evidence-based in the scientific sense, but I think they might be helpful, maybe even a good start point for further discussion on the subject.
The 10th anniversary of the iPad isn’t a destination, it’s just an arbitrary point from which to take stock of where things have been and consider where they are going. To do that, it’s instructive to look at more than the endpoints of the iPad’s history and consider what has happened in between. Viewed from that perspective, the state of the iPad ten years later, while at times frustrating, also holds reason for optimism. No single product in Apple’s lineup has more room to grow or potential to change the computing landscape than the iPad does today.
Source: The iPad at 10: Emerging from the Shadow of the iPhone, an article by John Voorhees.
As a Python programmer, you’ll frequently deal with Booleans and conditional statements—sometimes very complex ones. In those situations, you may need to rely on tools that can simplify logic and consolidate information. Fortunately, any() in Python is such a tool. It looks through the elements in an iterable and returns a single value indicating whether any element is true in a Boolean context, or truthy.
Source: How to Use any() in Python, an article by Alex Ronquillo.
We’re programmers, so writing code is what we do, is it not ? As the title suggests, our job is a bit more complicated than stroking keys on a keyboard in front of a screen all day. If you go beyond programming languages, and frameworks and processes, beyond test suites and sprints and Jira tickets, you will always find a problem that needs to be solved.
I would say that we, as programmers, are, first and foremost, problem solvers. We take a problem that someone else has and, using all the tools at our disposal, produce a solution.
Recently I've been taking a look at replacements for common command line tools (and coreutils) -
ls,cat,find,grep, etc. Turns out, people have been rewriting a lot of them in Rust
Source: My tools are going Rusty, an article by Ellie Huxtable.
Most of today’s popular CI services support running jobs in arbitrary Docker containers. In this post, I’ll describe why Docker is such a great fit for CI and how it can be made even better with custom images.
Source: Building a Custom Docker Image for CI – A Quick Guide, an article by John Ruble.
In the afternoon I wrote a long blog post on getting started with the Perl version of tumblelog on Ubuntu 18.04 LTS.
After 6PM I gave a mealworm, Tenebrio molitor, to my Brachypelma smithi. This was the first time it accepted food since I bought it. It molted recently, which might have been the reason it didn't accept food on earlier occasions as it was in pre-molt.
In the above photo you can see a few grains of sand on its carapace which ended up there after the tarantula's struggle with the mealworm.
I was recently looking at the source of a wxPython-based GUI application, about 45.5KLOC in size, not counting the libraries used (e.g. Twisted). The code was written by Java developers who are relatively new to Python, and it suffers from some performance issues (like a 30-second startup time). In examining the code, I found that they had done lots of things that make sense in Java, but which suck terribly in Python. Not because “Python is slower than Java”, but because there are easier ways to accomplish the same goals in Python, that wouldn’t even be possible in Java.
So, the sad thing is that these poor folks worked much, much harder than they needed to, in order to produce much more code than they needed to write, that then performs much more slowly than the equivalent idiomatic Python would.
Source: Python Is Not Java, an article by PJ Eby.
I’ve been writing a lot of Python recently. As a relative newcomer to the language, it continues to impress me with its versatility and “quality of life” features. I’m not surprised that it’s been gaining so much popularity among developers as of late.
Two such features I’ve discovered recently are the
permutationsandcombinationsfunctions of Python’s itertools module. The module is basically a set of convenience functions to produce iterators to suit various needs.
Source: The Usefulness of Python’s Permutations and Combinations Functions, an article by Kevin Dawe.
Beginners seem to ask this question when they are feeling daunted by the challenge before them. Maybe they are hoping for a helpful answer, but it seems like most answers will just be a jumping off point for feeling bad about their own progress.
Everyone learns differently. They learn from different sources, at different paces. Suppose you ask this question and someone answers “one month”? Will you feel bad about yourself because you’ve been at it for six weeks? Suppose they say, “ten years”? Now what do you think?
Source: How long did it take you to learn Python?, an article by Ned Batchelder.