a tumblelog
Tue 16 Feb 2021

Many Small Queries Are Efficient In SQLite

The Appropriate Uses For SQLite page says that dynamic pages on the SQLite website typically do about 200 SQL statements each. This has provoked criticism from readers. Examples:

  • "200 SQL statements is a ridiculously high number for a single page"
  • "For most sites, 200 queries is way, way, way too much."
  • "[This is] bad design"

Such criticism would be well-founded for a traditional client/server database engine, such as MySQL, PostgreSQL, or SQL Server. In a client/server database, each SQL statement requires a message round-trip from the application to the database server and back to the application. Doing over 200 round-trip messages, sequentially, can be a serious performance drag. This is sometimes called the "N+1 Query Problem" or the "N+1 Select Problem" and it is an anti-pattern.

Source: Many Small Queries Are Efficient In SQLite.

Python strings are immutable, but only sometimes

The standard wisdom is that Python strings are immutable. You can't change a string's value, only the reference to the string.

Source: Python strings are immutable, but only sometimes, an article by Austin Z. Henley.

Don't Mess with Backprop

Biologically Plausible Deep Learning (BPDL) is an active research field at the intersection of Neuroscience and Machine Learning, studying how we can train deep neural networks with a "learning rule" that could conceivably be implemented in the brain.

The line of reasoning that typically motivates BPDL is as follows:

  1. A Deep Neural Network (DNN) can learn to perform perception tasks that biological brains are capable of (such as detecting and recognizing objects).
  2. If activation units and their weights are to DNNs as what neurons and synapses are to biological brains, then what is backprop (the primary method for training deep neural nets) analogous to?
  3. If learning rules in brains are not implemented using backprop, then how are they implemented? How can we achieve similar performance to backprop-based update rules while still respecting biological constraints?

A nice overview of the ways in which backprop is not biologically plausible can be found here, along with various algorithms that propose fixes.

Source: Don't Mess with Backprop: Doubts about Biologically Plausible Deep Learning, an article by Eric Jang.