Skip to content

Compare implementations

Rewrote a function and want to know whether it's actually faster? Profile both at once. Mark the original as the baseline with -b (--baseline) and the candidate with -p (--profile); profit lines them up and adds a Delta column.

profit run my_script.py -b my_module:old_impl -p my_module:new_impl

examples/compare.py defines two ways to flatten a nested list — an explicit stack loop and a recursive version — and runs each 500 times:

profit run examples/compare.py -b flatten_loop -p flatten_recursive

profit comparing two implementations, with a Delta column

The baseline row is pinned at 1x. Each candidate's Delta shows how its mean per-call time differs: a negative delta and a ratio below 1x mean it's faster (with a 🚀 to mark the win), a positive delta and a ratio above 1x mean it's slower. In the run above, the recursive version comes out ahead.

Because flatten_recursive calls itself, its Calls reads 150500/500 — 150,500 total calls across 500 top-level invocations. The comparison is on mean per-call time, so it stays meaningful even when the two functions ran a different number of times. See Reading the output.

Tips

  • Pass several -p candidates against one -b to compare more than two implementations at once.
  • To compare expressions rather than functions in a script, the same -b/-p flags work on profit timeit.

Next