A returns calculation over a 50 million element float64 array is rewritten from a Python loop into NumPy and gets 60 times faster. A colleague then adds two lines and resident memory doubles. Explain the speedup, and say which NumPy operations return a view and which allocate.

A returns calculation over a 50 million element float64 array is rewritten from a Python loop into NumPy and gets 60 times faster. A colleague then adds two lines and resident memory doubles. Explain the speedup, and say which NumPy operations return a view and which allocate.

Approach: Account for the per-element interpreter overhead the loop was paying, then ask which array operations can share the original buffer and which must allocate a new one.

The speedup is vectorisation: the per-element work moves into a compiled loop over a contiguous buffer with no boxed Python floats, no bytecode dispatch and no reference counting, taking an element from 50 to 100 nanoseconds down to about 1 nanosecond, and the memory doubled because basic slicing returns a view while fancy indexing, boolean masking, astype, a transpose followed by a reshape, and every arithmetic expression without an out argument allocate a new array. A 50 million element float64 array is 400 MB, so one temporary array doubles the footprint and two triple it. Use the in-place operators or pass out to the ufunc so the destination buffer is reused, and check the base attribute of a result to see whether it shares memory with the original. Slicing with a step is still a view, and any indexing by an array of positions or by a mask is a copy.

Follow-up: How would you compute the same result under a hard 500 MB memory budget?

Key concepts: vectorisation, view, fancy indexing, temporary array.