Sun 12 Sep 2021

The hidden performance overhead of Python C extensions

Python is slow, and compiled languages like Rust, C, or C++ are fast. So when your application is too slow, rewriting some of your code in a compiled extension can seem like the natural approach to speeding things up.

Unfortunately, compiled extensions are sometimes actually slower than the equivalent Python code. And even when they’re faster, the performance improvement might be far less than you’d imagine, due to hidden overhead caused by two factors:

  1. Function call overhead.
  2. Serialization/deserialization overhead.

Let’s see where these hidden performance overheads comes from, and then see some solutions to get around them.

Source: The hidden performance overhead of Python C extensions, an article by Itamar Turner-Trauring.