Python #1 on Github + speedup via Codon / Cython
Introduction
Recently there's a report from Github on the rise of snake language to be the top on Github. Yes, Python overtakes Javascript!
Below is a nice time series graph showing language progression over time:
Inspired by this achievement, I once again write how to speedup Python to almost near C/C++ speed! The tools we are using: Codon and Cython to speedup integrate calculation in Python.
Python code
First lets take a look at the original python code.
def f(x):
return x ** 2 - x
def integrate_f(a, b, N):
s = 0
dx = (b - a) / N
for i in range(N):
s += f(a + i * dx)
return s * dx
Fairly simple and straightforward.
Codon code
Now here's the Codon version:
def f(x: float) -> float:
return x ** 2 - x
def integrate_f(a: float, b: float, N: int) -> float:
i: int = 0
s: float = 0.0
dx: float = 0.0
dx = (b - a) / N
for i in range(N):
s += f(a + i * dx)
return s * dx
Can you see how similar it is between Python vs Codon code? The only difference is the type annotations / type hints in the Codon code.
Cython code
Last but not least, let's take a look at Cython implementation:
import cython as cy
@cy.cfunc
def f(x: cy.float) -> cy.float:
return x ** 2 - x
@cy.ccall
def integrate_f(a: cy.float, b: cy.float,
N: cy.int) -> cy.float:
i: cy.int = 0
s: cy.float = 0.0
dx: cy.float = 0.0
dx = (b - a) / N
for i in range(N):
s += f(a + i * dx)
return s * dx
The primary code structure remains largely unchanged from the original Python code, with the addition of a few minor modifications:
- Import cython.
- Cython decorator.
- Type annotations / type hints.
Performance benchmark
Now to the fun stuff: benchmarking!
at 31.1ms in 10 runs Cython ran 5% faster than Codon and 84x faster than Python
Summary
Below is the screenshot of my desktop showing the different version of codes and the benchmark results.
In my opinion Codon delivers the best compromise:
- Syntax are 99% the same compared to Python code.
- As performant as Cython.
So yeah, pure Python is slow. But who is dumb enough to use "pure Python" anyway?? As I have said many times: unlocking the C engine deep inside Python is not difficult, it's just most people don't know how to do that.
Take advantage of Python's amazing ecosystem! Leverage the right tools and libraries to write efficient and scalable code. Be practical and pragmatic, even if it's not "pure Python". Utilize the amazing tools provided in the Python ecosystem:
- Processing data: Numpy (C, Fortran) or Scipy (C, C++, Fortran)
- Data wrangling & analysis: Pandas (C, Cython) or Polars (Rust)
- Machine Learning: Scikit-learn (C, C++, Cython) or Pytorch (C, C++)
- Web server: Granian (Rust)
- Package manager: UV (Rust)
- Linter: Ruff (Rust)
I have plenty other recipes to cover on this Python speedup topic, such as: how to implement Cython/Codon with Numpy, Pandas, even a web framework (Quart).
Bonus: let's see how C performs!
Using exactly the same algorithm, code structure and hyperfine parameters C got an average of 39.5ms in 10 runs. Meaning both Codon and Cython implementations assuming they are properly optimized, their performances are within C/C++ speed!