The hidden performance overhead of Python C extensions
Python is slow, and compiled languages like Rust, C, or C++ are fast.
So when your application is too slow, rewriting some of your code in a compiled extension can seem like the natural approach to speeding things up.
Unfortunately, compiled extensions are sometimes actually slower than the equivalent Python code.
And even when they’re faster, the performance improvement might be far less than you’d imagine, due to hidden overhead caused by two factors:
- Function call overhead.
- Serialization/deserialization overhead.
Let’s see where these hidden performance overheads comes from, and then see some solutions to get around them.
Problem #1: Call overhead
The first performance overhead we’re going to face is that of function calls.
Let’s write a function in Cython, a Python variant language that compiles to