Totally confused by all the tools for managing dependencies? Pip,
venv, Docker, conda, virtualenvwrapper, pipenv, … Which one should
you use? Why do we even have all these different tools? Can they
work together?
No wonder. The world of Python dependency management is a mess, but
once you understand the tools and why they exist, it’s going to be
easier to choose the one you want and deal with the others in
environments where you can’t choose your favorite ones.
Have you ever tried to click photos with your camera in the laptop,
of course, you did, but here is a twist, we will click the photo in
a Docker container, many of you may be wondering how you can do
that? Well, let me get started with this interesting thing.
mDNS is a protocol that resolves hostname to IP addresses within a
small network that lacks the name server. By default, mDNS
exclusively resolves hostnames ending with .local. But there will be
a problem with the hosts that implement .local doesn't support mDNS
protocol and can be found via a conventional unicast DNS sever. In
those cases, necessary network configuration should be changed.
Principal components analysis (PCA) is one of a family of techniques
for taking high-dimensional data, and using the dependencies between
the variables to represent it in a more tractable, lower-dimensional
form, without losing too much information. It has been widely used
for data compression and de-noising. However, its entire
mathematical process is sometimes ambiguous to the user.
In this article, I would like to discuss the entire process of PCA
mathematically, including PCA projection and reconstruction, with
most of the derivations and proofs provided. At the end of the
article, I implemented PCA projection and reconstruction from
scratch. After reading this article, there should be more black box
in PCA anymore.
True or not, a strong password hashing is crucial for a large
ecosystem like the WordPress one, which has always been a juicy
target for hackers. So, I decided to take a closer look at the
hashing system and try to crack WordPress hashes from scratch!
In this multi-part series of articles about Kubernetes, I'll try and
capture what I think everyone who wants to learn and work with
Kubernetes should know about.
When confronting a new data science problem, one of the first
questions to ask is which technology to use. There is hype; there
are standard tools; there are bleeding-edge technologies, entire
platforms and off-the-shelf solutions.
In the last year, the Haskell language and associated technology
have been seen developing into the most mature ecosystem we’ve seen
to date, with innovation happening left and right across a variety
of fronts. Editor tooling, for instance, is reaching levels of
maturity we only dreamed about years ago. Simultaneously, there has
been a fair bit of discussion about the economics of the Haskell
ecosystem and the confounding factors that have led to its potential
stagnation. Most recently, for instance, there have been discussions
about “Simple Haskell” as a set of best practices to spur more
successful industry projects.
I’ve been toying around with some ideas for how to use custom
properties (aka CSS variables) for global settings in a project. The
idea is to provide control to designers/developers over consistent
styles across multiple components.
One of the interesting facts about writing your own parser
combinators library, is that you will learn (or consolidate) other
knowledges in the process, like: Functors, Applicatives and, of
course, Monads, and more generaly, how to design DSL in Haskell.
So, how do you test the functionality of how your app responds to a
web request without making an actual request and returning a
response. One approach is to mock the requests and responses.
This is the first in a series of blog posts intended to provide a
gentle introduction to
flakes, a new Nix feature
that improves reproducibility, composability and usability in the
Nix ecosystem. This blog post describes why flakes were introduced,
and give a short tutorial on how to use them.
Deep Learning is all about linear algebra and calculus. If you try
to read any deep learning paper, matrics calculus is a needed
component to understanding the concept.
This article series is a guide to modern Python tooling with a focus
on simplicity and minimalism. It walks you through the creation of
a complete and up-to-date Python project structure, with unit tests,
static analysis, type-checking, documentation, and continuous
integration and delivery.
In this article, let's look at some of the ways to do batch HTTP
requests in Python and some of the tools at our disposal. Mainly,
we'll look at the following ways:
With the
release of
Python 3.9.0b1, the first of four planned betas for the development
cycle, Python 3.9 is now feature-complete. There is still plenty to
do in terms of testing and stabilization before the October final
release. The release announcement lists a half-dozen Python
Enhancement Proposals (PEPs) that were accepted for 3.9. We have
looked at some of those PEPs along the way; there are some updates
on those. It seems like a good time to fill in some of the gaps on
what will be coming in Python 3.9
Feeding a Caribena versicolor sling and a tiny scorpion
Today I was finally able to feed the Caribena versicolor sling that
has been in my care since the first of this
month. I held a small
mealworm, Tenebrio molitor, with tweezers close to it, and it
"jumped" on the small prey item. In the past the small tarantula had
refused food items of the same size, no idea why. Maybe still getting
used to its enclosure.
Caribena versicolor with prey: a small Tenebrio molitor larva.
I was also finally able to feed a second instar Chaerilus
sp. "Java"; a very small scorpion that I have been keeping since the
7th of April
2020. While it has small
springtails in its enclosure, which maybe it eats, I prefer to
actually see it eat. So in the late afternoon I managed, after a
few attempts, to start it accepting and eating a very tiny mealworm
larva.
Diving into Go by building a CLI application
In this blog post we’ll build a CLI application in Go, which we’ll
call go-grab-xkcd. This application fetches comics from
XKCD and provides you with various options
through command-line arguments.
Among the many use cases Python covers, data analytics has become
perhaps the biggest and most significant. The Python ecosystem is
loaded with libraries, tools, and applications that make the work of
scientific computing and data analysis fast and convenient.
But for the developers behind the Julia
language — aimed specifically at
“scientific computing, machine learning, data mining, large-scale
linear algebra, distributed and parallel computing”—Python isn’t
fast or convenient enough. Python represents a trade-off, good for
some parts of data analytics work but terrible for others.
Piping is one of the core concepts of Linux & Unix based operating
systems. Pipes allow you to chain together commands in a very
elegant way, passing output from one program to the input of another
to get a desired end result.
In the afternoon I noticed that the Psalmopoeus irminia sling I keep
had molted; I saw the exoskeleton dangling from a piece of moss coming
out of the cork tube it lives in. Because I had an appointment I
couldn't take photos, so that I did in the evening.
The exoskeleton of a Psalmopoeus irminia.
In the evening, after I had taken the above photo I also spotted a
cast-off exoskeleton in the terrarium in which I keep a Pterinochilus
murinus sling. On the 12th of this
month it had already
opened its burrow, and I suspected back then that it had molted. And
now I had proof. Maybe I overlooked the exoskeleton earlier because it
was underneath the leaf of a plastic plant.
The (partial) exoskeleton of a Pterinochilus murinus.
I also checked on the Chromatopelma cyaneopubescens sling I keep,
which recently also
molted. Because it moved
out of its webbing I decided to try to feed it, and it readily
accepted a pre-killed mealworm, Tenebrio molitor.
5 Types Of ZSH Aliases You Should Know
I use Docker, Kubernetes, and Microsoft Azure every day. That said,
it makes sense for me to have aliases supporting me with these tools
and environments. However, maybe you are using different clouds and
command-line tools so that you will end up with different
aliases. The key takeaway should be that you create and use aliases
to help you get your job done.
Sometimes I find the git diff command a little inconvenient. It
can throw a lot of information at the screen at once. I use git diff not only for verifying my changes before a commit, but also to
review pull requests, or for finding bugs introduced between two
commits. In the situations when you’re looking at a lot of changed
files, having to scroll up and down so much is tedious.
Initially I wanted to write articles on those two topics separately
(mocking time and testing event loops), but during the process I
realized that the things I want to talk about are too interrelated:
when I need to mock time, it's usually to test some event loop with
it, and when I test event loops, typically mocked time is also
involved in that.
So in the end, it felt better to just combine all that in a single
article.
In 2020 there are a lot of developers and designers who want to
learn the basics of CSS. In this series of articles, I will teach
you those main topics. In this specific article, I will review the
essential CSS properties of typography while using many visual
examples.
The PDF, or Portable Document Format, is one of the
most common formats for sharing documents over the
Internet. PDFs can
contain text, images, tables, forms, and rich media like videos and
animations, all in a single file.
This abundance of content types can make working with PDFs
difficult. There are a lot of different kinds of data to decode when
opening a PDF file! Fortunately, the Python ecosystem has some great
packages for reading, manipulating, and creating PDF files.
This week I was debugging a misbehaving Python program that makes
significant use of Python’s
asyncio. The
program would eventually take very long periods of time to respond
to network requests. My first suspicion was a CPU-heavy coroutine
hogging the thread, preventing the socket coroutines from running,
but an inspection with pdb showed this wasn’t the case. Instead, the
program’s author had made a couple of fundamental mistakes using
asyncio. Let’s discuss them using small examples.
Five tips to be a more effective command-line user
This article intends to be helpful for those who are already command
line (CLI) users. Complete beginners are of course encouraged to
read on, even though they may not grasp all the advantages
immediately and perhaps there is a lot of other more important
things to learn when starting. On the other hand, I expect long time
CLI users to already work similarly. I do hope they might also find
interesting tricks to adopt.
This file is used to sequence functions that could fail together,
allowing you to avoid doing an if err != nil check on every single
fallible function call.
One common kind of data stored in a configuration file is
options. In this post, I'll talk about some nuances we have to be
aware of when storing options in JSON and unmarshaling them to Go.
Specifically, the most important difference between options and any
other data is that options are often, well... optional. Our
program can have a large number of possible configuration options,
but we may want to configure any particular invocation with only a
subset - leaving all the others at their default values.
It’s not a particularly new or exciting take to say that we’ve
outgrown Bash. The Unix shell environment was novel and powerful in
its day, but its age is really starting to show.
Using make and git diff for a simple and powerful test harness
People regularly go for complex test harnesses; and when you’re in
an ecosystem, they can be a good fit. But outside ecosystems, it’s
actually quite easy to produce a fairly good test harness.
Almost two years ago, I moved from a cushy corporate Scala job to a
riskier mid-stage startup using
Haskell. This
post summarises my thoughts about Haskell after experiencing it
first hand.
I wrote bash scripts for several years now. You can find old bash
scripts of mine on GitHub but most of them aren’t maintained
anymore - and i have a reason for that.
Recently, I was tipped off about certain sites performing localhost
port scans against visitors, presumably as part of a user
fingerprinting and tracking or bot detection. This didn't sit well
with me, so I went about investigating the practice, and it seems
many sites are port scanning visitors for dubious reasons.
The great thing about Python is that it makes developers’ lives easy
— import a couple of libraries to do the hard stuff for you, and
you’re off to the races. This holds true when creating a threaded
web scanner that’s capable of making multiple concurrent requests —
with Python it’s easy to accomplish in a short amount of dev time.
An integer in Python is not a traditional 2, 4, or 8-byte
implementation but rather it is implemented as an array of digits in
base 230 which enables Python to support super long
integers. Since
there is no explicit limit on the size, working with integers in
Python is extremely convenient as we can carry out operations on
very long numbers without worrying about integer overflows. This
convenience comes at a cost of allocation being expensive and
trivial operations like addition, multiplication, division being
inefficient.
Today, shortly after Adam returned from school, one of his birthday
presents finally arrived; a female Acanthoscurria geniculata. I had
ordered this tarantula the 6th of May, 2020 with Robert of
Fatbob's Exotics:
0.1 Acanthoscurria geniculata 2,5-3cm body €25
Together with shipping within the Netherlands it came down to
€32. Originally, Adam wanted to have a female Brachypelma hamorii but when
I contacted Robert the 5th of May the tarantula was no
longer available.
Adam holding the vial with his female Acanthoscurria geniculata.
Due to the colder nights it took a while before Robert could ship the
spider safely. But yesterday it got picked up by DPD and today it
arrived. Not on time for Adam's birthday, which was the
12th of May, so he had been asking now and then "When is my
tarantula coming?"
Female Acanthoscurria geniculata on Adam's hand.
For the terrarium I drilled plenty of holes in two sides of a plastic
box and several more in the lid. Together with Adam I added plenty of
substrate; slightly moist coco peat. I had also cut a cork tube in
half, which Adam added, together with two plastic plants. Finally, I
added a plastic bottle lid with water to provide a source of moisture
to the spider.
Acanthoscurria geniculata in its terrarium.
We tried to feed the tarantula a mealworm larva, Tenebrio molitor,
but it was not interested. In my experience sometimes tarantulas
accept food after travelling and sometimes they don't. Later in the
afternoon when I checked the terrarium I noticed that she had been
moving substrate out of the cork tube and made a nice pile next to it.
Close up of the female Acanthoscurria geniculata.
In the evening Adam and I checked again, but she was still well hidden
in her new burrow. We also tried to feed his Heterometrus
petersii. While the scorpion did pick up the mealworm I put next to
it, and even held it for a while, it was too stressed out because I
had lifted the piece of wood it normally hides under to actually eat
it. Adam and I decided to try again tomorrow. Adam is a little worried
about his scorpion since it hasn't been eating since we bought it, the
23rd of February,
2020 at the Invertebrate
Show in Harmelen.
We did leave a pre-killed mealworm next to the burrow entry of his
tarantula, though. Let's see if it's gone tomorrow.
Python performance: it’s not just the interpreter
I have a particular view of Python performance from my experience on
the Pyston project, and since this view
is somewhat nonstandard I wanted to take some time to explain it and
give a motivating example.
A common
explanation
for Python's low performance is that it is an interpreted language.
In this post I hope to show that while the interpreter adds
overhead, it is not the dominant factor for even a small
microbenchmark. Instead we'll see that dynamic features --
particularly, features inside the runtime -- are to blame.
Here is my Katas for creating BASH programs that work. Nothing is
new here, but from my experience pepole like to abuse BASH, forget
computer science and create a Big ball of
mud from their
programs. Here I provide methods to defend your programs from
braking, and keep the code tidy and clean.
