Bufferpools and RAM Commit/Deliver

Uncategorized Add comments

Bufferpool config is an often overlooked issue due to the rarity in which it nails you, but it can be important in those rare cases.

a Bufferpool is simply a resource limitation on a collection of RAM — typically this is a buffer, ie in-RAM space, it cannot be swapped-out because it represents in-flight transactions, uncommitted pages, or pre-fetched content that will be needed very soon.

a Commit is the RAM that is offered to a process — in glibc, this can default to 2G. This doesn’t say that every process automatically consumes 2G of RAM, but that the Kernel offers up to 2G to the process. Recall that due to sparse garbage in RAM pages, RAM offered to a process is NEVER reaped by the system back to the common pool.

A Commit can be dangerous when the OS over-commits RAM in a long-lived environment: if up to 100G is offered on a system with only 32G, you can see how if many threads grow their demand for RAM, the system will swap out some processes to meet demand. This is a typical action in a multiuser system with swap active (NOTE: Motorola tuned their commits on smartphones because on diskless systems, there’s obviously no swap)

In a long-lived database process, if the bufferpool is a commit, then it will soon grow to maximum commit. It can never be swapped out unless the database has bursty use-cases and has no active sessions for long periods. The bufferpools configured may be in addition to the heap space taken up by the process itself (in un-pooled resource space). The database itself may limit bufferpools, but consume a number of GB over the configured bufferpool space.

The other applications on the system also can demand up to their committed RAM — why limit one process while letting the others run amok on your server?

On long-lived systems, committed RAM becomes allocated and consumed RAM. Bufferpools need to be configured, and RAM usage monitored (or at least traps/exceptions raised when a critical DB starts swapping, an indication that review is critically needed)

Bufferpools and Commit/Demand discrepancies are silent but deadly killers, like the sharks and heart-disease of the resource-management domain.

Leave a Reply

WP Theme & Icons by N.Design Studio
Entries RSS Comments RSS Log in