Lecture 6

Python Performance Tuning

Setup

• Install line_profiler, using either pip install line_profiler or conda install line_profiler

What is profile?

Profiling

Deterministic profiling

• Monitor all function calls/returns and other events
• Precise timing
• By instrumenting the program - inserting intructions into the program that collect this timing information

Statistical profiling

• Randomly samples the effective instruction pointer
• Less overhead, but less precise

Why call counts & profiling statistics are useful?

• identify bugs
• identify possible tuning points
• identify "hot loops" that should be carefully optimized
• identify high level system design or algorithm adoption

Python standard profiling modules

cProfile

• lowest overhead, written in C, may not be as widely available

profile

• Written in Python, so much higher overhead, but easier to extend

Non-standard line_profiler, but popular for ipython & juputer-notebook