High Level Web Frameworks
Introduction
Weekend is here and that means I got free time to write some articles! For some time I have been looking to do a benchmark of web frameworks. Here in this article I will be doing high-level version. If you're interested on low-level web frameworks, you can find the article here
Coverage: Python and JS/TS
In the high-level section I will be using both Python and Javascript/Typescript frameworks. However the selection is limited, as I'm only interested in using Quart/Flask for Python and Bun+Elysia for JS/TS. These are the frameworks I'm actively using, plus Bun+Elysia currently is the most performant framework combo in JS/TS.
Test setup
- I will be using wrk which is a C library to test the web framework throughput. The wrk command template I'm using:
wrk -t1 -c10 -d10 --latency http://localhost:5000
Note there will be variations of the wrk command.
- CPU: Ryzen 4800u 16 cores, 8GB RAM.
- The test run multiple times using a variation of threads and connections.
- For Python I'm using different web servers:
- FastWSGI,
- Hypercorn and
- Granian.
- Primary metrics: request per second and average latency.
Benchmark results
Python
| Framework | req/s | avg latency | thread - conn | Notes |
|---|---|---|---|---|
| Flask | 1,348 | 7,290 us | t1 - c10 | no-logging |
| --fastwsgi | 9,186 | 1,110 us | t1 - c10 | no-logging |
| --fastwsgi | 10,078 | 4,800 us | t4 - c50 | no-logging |
| blankline | --- | --- | --- | --- |
| Quart | ||||
| --hypercorn | 2,811 | 3,610 us | t1 - c10 | no-logging |
| --hypercorn | 8,778 | 5,560 us | t4 - c50 | no-logging |
| --granian | 5,232 | 2,010 us | t1 - c10 | no-logging |
| --granian | 9,999 | 5,000 us | t4 - c50 | no-logging |
Javascript/Typescript
| Framework | req/s | avg latency | thread - conn | Notes |
|---|---|---|---|---|
| Bun | 49,939 | 141 us | t1 - c10 | no-logging |
| Bun | 68,836 | 722 us | t4 - c50 | no-logging |
| blankline | --- | --- | --- | --- |
| Elysia | 60,452 | 112 us | t1 - c10 | no-logging |
| Elysia | 65,874 | 753 us | t4 - c50 | no-logging |
Conclusion
Python:
Flask (sync framework) using builtin server is kinda sad to see. I actually took the time to test using multiple thread and multiple connection but it doesn't improve. However as we can see to improve the rps (request per second) metric is not hard: using different web server (FastWSGI). Flask received 8x liftup in rps just by changing the web server.
Quart (async framework) using Hypercorn as the builtin server starts low < 3k rps however when we increase the threads and connections it managed to scale up.
Javascript/Typescript:
Bun starts very strong at 46k rps and managed to scale up and even surpassed Elysia's number when I increased the threads and connections.
Elysia (built on top of Bun) is the most performant of all high-level web frameworks. Using 4 threads and 50 connections it topped up at 65k rps.
Note: Both Bun and Elysia are frameworks written in Zig, making it performant even faster than Deno (which is written in Rust, btw).