IPython %timeit what is loop and iteration in the options?

Question

I am wondering about the %timeit command in IPython

From the docs:

%timeit [-n<N> -r<R> [-t|-c] -q -p<P> -o] setup_code

Options:

-n: execute the given statement times in a loop. If this value is not given, a fitting value is chosen.

-r: repeat the loop iteration times and take the best result. Default: 3

For example, if I write:

%timeit -n 250 -r 2 [i+1 for i in range(5000)]

So, -n 250 executes [i+1 for i in range(5000)] 250 times? Then what does -r 2?

Answer 1

It specifies the number of repeats, the number of repeats are used to determine the average. For example:

%timeit -n 250 a = 2
# 61.9 ns ± 1.01 ns per loop (mean ± std. dev. of 7 runs, 250 loops each)

%timeit -n 250 -r 2 a = 2
# 62.6 ns ± 0 ns per loop (mean ± std. dev. of 2 runs, 250 loops each)

The number of executions will be n * r but the statistic is based on the number of repeats (r) but the number of "loops" for each repeat is determined based on the number (n).

Basically you need a large enough n so the minimum of the number of loops is accurate "enough" to represent the fastest possible execution time, but you also need a large enough r to get accurate "statistics" on how trustworthy that "fastest possible execution time" measurement is (especially if you suspect that some caching could be happening).

For superficial timings you should always use an r of 3, 5 or 7 (in most cases that's large enough) and choose n as high as possible - but not too high, you probably want it to finish in a reasonable time :-)