A process grows from 2 GB to 9 GB of resident memory over a week while a heap profiler shows live allocations flat at 1.8 GB the whole time. Say what this is, the allocation pattern that causes it, how you confirm it, and two fixes.

A process grows from 2 GB to 9 GB of resident memory over a week while a heap profiler shows live allocations flat at 1.8 GB the whole time. Say what this is, the allocation pattern that causes it, how you confirm it, and two fixes.

Approach: Compare the bytes the allocator has taken from the kernel with the bytes currently live, then ask which allocation sizes stop a freed region from being handed back.

This is heap fragmentation rather than a leak, since live bytes stay flat while resident memory grows, so the allocator is holding freed space it cannot return because surviving objects are scattered across otherwise empty pages. The pattern is long-lived small objects allocated among short-lived larger ones, so every arena retains a few survivors and no whole region can be released back, and many threads each holding their own arena makes it worse. Confirm it by comparing the allocator statistics, such as malloc_info or the allocator stats dump, against the profiler live bytes, and by reading the process maps to see many partially used heap regions. Two fixes are to segregate the long-lived objects into their own pool so the general heap can be released, and to switch to an allocator with a tunable dirty page decay and set that decay so freed pages go back to the kernel.

Follow-up: What would the same two curves look like for a genuine leak, and which of them would move first?

Key concepts: fragmentation, resident memory, live bytes, arena.