Berkeley DB Reference Guide: Memory Pool Subsystem

Berkeley DB and the memory pool

The Memory Pool subsystem is the general-purpose shared memory buffer pool used by Berkeley DB. This module is useful outside of the Berkeley DB package for processes that require page-oriented, shared and cached file access.

A memory pool is a memory cache shared among any number of threads of control. The DB_INIT_MPOOL flag to the DB_ENV->open function opens and optionally creates a memory pool. When that pool is no longer in use, it should be closed using the DB_ENV->close function.

The DB_ENV->memp_fcreate function returns a DB_MPOOLFILE handle on an underlying file within the memory pool. The file may be opened using the DB_MPOOLFILE->open function. The DB_MPOOLFILE->get function is used to retrieve pages from files in the pool. All retrieved pages must be subsequently returned using the DB_MPOOLFILE->put function. At the time pages are returned, they may be marked dirty, which causes them to be written to the underlying file before being discarded from the pool. If there is insufficient room to bring a new page in the pool, a page is selected to be discarded from the pool using a least-recently-used algorithm. Pages in files may also be explicitly marked clean or dirty using the DB_MPOOLFILE->set function. All dirty pages in the pool from the file may be flushed using the DB_MPOOLFILE->sync function. When the file handle is no longer in use, it should be closed using the DB_MPOOLFILE->close function.

There are additional configuration interfaces that apply when opening a new file in the memory pool:

• The DB_MPOOLFILE->set_clear_len function specifies the number of bytes to clear when creating a new page in the memory pool.
• The DB_MPOOLFILE->set_fileid function specifies a unique ID associated with the file.
• The DB_MPOOLFILE->set_ftype function specifies the type of file for the purposes of page input and output processing.
• The DB_MPOOLFILE->set_lsn_offset function specifies the byte offset of each page's log sequence number (DB_LSN) for the purposes of transaction checkpoints.
• The DB_MPOOLFILE->set_pgcookie function specifies an application provided argument for the purposes of page input and output processing.

There are additional interfaces for the memory pool as a whole:

• It is possible to gradually flush buffers from the pool in order to maintain a consistent percentage of clean buffers in the pool using the DB_ENV->memp_trickle function.
• Because special-purpose processing may be necessary when pages are read or written (for example, compression or endian conversion), the DB_ENV->memp_register function allows applications to specify automatic input and output processing in these cases.
• The db_stat utility uses the DB_ENV->memp_stat function to display statistics about the efficiency of the pool.
• All dirty pages in the pool may be flushed using the DB_ENV->memp_sync function. In addition, DB_ENV->memp_sync takes an argument that is specific to database systems, and which allows the memory pool to be flushed up to a specified log sequence number (DB_LSN).
• The entire pool may be discarded using the DB_ENV->remove function.

Copyright Sleepycat Software