Mitata - a beautiful benchmarking tool
Introduction
This is my first programming -based blog in several weeks (+/- 4 weeks?) due to the turbulent financial markets and geopolitic situations requiring 100% of my attention. After all when markets are crashing that is the best timing to look for opportunities in the financial markets. My main background is a hedge fund professional, if this finance stuff interest you then take a look at my finance page.
Update: added external comment section at the bottom page. Powered by komments.
Mitata
Okay enough with the finance stuff, here I want to introduce you to a benchmarking tool I've just found: Mitata. I love doing benchmarks coz one of the topics that interests me the most is about optimization and that means being able to measure precisely the changes between iterations are super important.
Hell, I've even tried to optimize my Python code (via Codon for general purpose and Fireducks for tabular data processing). For those using Python I highly recommend you to check Codon as means of unlocking the C engine inside Python and also check Fireducks to speedup your Pandas code to make it even more performant than Polars.
What's special about Mitata
I usually use hyperfine which works well, however one thing that make Mitata special is the graphical presentation options it got.
Below is example of a hyperfine benchmark presentation:
Which is fine and works great!
Check these Mitata screenshots:
check the date : 21 March -- yeah I got lots of backlog to post! Blame the turbulent financial markets lol...
Mitata is more beautiful with its graphical presentation and there are other visualizations that is not shown in the above images. Currently available visualizations are: barplot, boxplot and lineplot.
One of the most important feature of any benchmarking tool is accuracy, as mentioned in the Github page: Mitata's accuracy is down to picosecond.
From my 2 months using it, I have no issue at all regarding the accuracy and I absolutely loving the visualization it provides!
Mitata is not only for JS/TS
Yes, Mitata is mostly popular in the JS/TS community (Bun and ElysiaJS often produces benchmarks using Mitata), those outside can use it too especially those working with compiled programs.
Above is an example of how I use Mitata:
- Create program paths.
- Determine the features we are using (barplot and summary).
- Inside the code block do the bench function.
- Run the program.
That's it! Very simple to generate good looking benchmark in Mitata and there are many other methods you can find in Mitata github page.
Conclusion
Play around with Mitata to see if you like it! Below is a screenshot both hyperfine and mitata running benchmark on the same set of programs:
As you can see the results are similar. I will continue using both hyperfine and mitata to verify any benchmarking I'm doing, they're both great tool! One caveat when using mitata: for example the above benchmark: measuring the performance of different languages on processing cyclic data structure (known to be Rust's weakness -- hence why Rust performs the worst in the bench), the order of the program matters..the one goes first usually got better numbers and the last got worse numbers.
This is why I usually put the least performant program in the first order -- to make sure they're not handicapped in any way. But actually that means I put handicap for the other programs that starts later lol! So yeah that's why I'm using both hyperfine and mitata at the moment -> to crosscheck results. If mitata fixed this issue, I'm leaning towards using mitata as my primary benchmarking tool!