So recently I have needed to do a time comparison of two different c programs. Unfortunately I was not able to use a c timing method as I couldn’t edit the source code of one of the programs hence my first attempt was to use Python.

Python has an inbuilt module known as time as well as something called Timeit.

Timeit is supposed to help you from falling into the traps of timing an algorithm. There are many, many traps it appears. First off you must decide what time to use. There seems to be a couple of options here such as Wall time and CPU time. Wall time is the real time taken to run the algorithm e.g. the actual time you experience compared to CPU time which measures the number of cycles the CPU takes to run your program. In theory CPU time should be a more accurate representation for comparison and hopefully independent of anything else that is possibly running.

Unfortunately by default Timeit seems to use the wall time and then by default run your program 10000 times. Selecting the best time from those runs gives you your runtime. Obviously the reason to do this is that the shortest time is a better measure than the average time which would include situations where the operating system is getting in the way.

However unfortunately this didn’t seem to provide me with a theoretically consistent time. Therefore I tried to use the CPU time. Eventually I cam across the inbuilt Linux function time. This produces 3 times: real time, user time and kernel time. The user time is the CPU time that I required and seemed to be consistent with theory.

This all boils down to this essentially. “Timing is not scientific”