storage: consider improvements on-by-default jemalloc observability

[jbowens]

In escalations like cockroachlabs/support#3153 where we require observability into jemalloc behavior, we're currently forced to ask users to collect additional jemalloc stats through setting vmodule flags. This back and forth adds latency to our investigations and frustrates users. There's probably something we can do to improve the observability we collect by default that can improve the situation. If I recall correctly, we currently attempt to notice an imminent oom death and dump a Go heap profile to the filesystem. Should we also dump detailed jemalloc stats alongside the heap profile? Are there stats that we can collect periodically without too much overhead.

Jira issue: CRDB-46489

[blathers-crl]

Hi @jbowens, I've guessed the C-ategory of your issue and suitably labeled it. Please re-label if inaccurate.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}