Fri 03 Dec 2021

Some guidelines for writing web scrapers

I’ve been working on, a job board for RTL and FPGA engineers, since September. Right now, RTLjobs is an aggregator. We index open FPGA/RTL roles at a number of companies, check them for relevance, and post them to our site. I’ve already written two job scraping systems to help ease the work of indexing jobs. Here’s some general rules I’ve come to hold for writing web scrapers and other ETL systems.

Source: Some guidelines for writing web scrapers, an article by Nash Reilly.

Result builders in Swift

The result builder proposal (originally it was called function builders) was implemented in Swift 5.4. This feature allows us to build up a result value using a sequence of components. At first sight, you might think, hey this looks like an array with a series of elements, except the coma in between the items, but nope, this is completely different. But why is it good for us?

Result builder can be used to create entirely new Domain-Specific Languages (DSLs) inside Swift. Creating a DSL has many advantages, since DSLs are usually tied to a specific problem, the syntax that you use to describe the language is very lightweight, yet powerful and capable. Since Swift DSLs are type safe, it is much safer to use one instead of manually concatenate objects. Swift DSLs also allows us to use basic control flows inside these embedded micro-languages. 🤔

Source: Result builders in Swift, an article by Tibor Bödecs.