Michael Paquier: Postgres - Fun with LWLocks

PostgreSQL lightweight-lock manager, with its interface in src/include/storage/lwlock.h, is a facility aimed at controlling the access to shared memory data structures. One set of routines is at the center of this post: LWLockUpdateVar() LWLockWaitForVar() LWLockReleaseClearVar() These are the least popular APIs related to lightweight locks used in the PostgreSQL core code, being only used by the WAL insertion code to control the locking around the backends doing the insertion of WAL records when their writes happen. Most of the code is located in xlog.c and the structure called in shared memory called WALInsertLock that stores NUM_XLOGINSERT_LOCKS locks (8 as of this post). Increasing this value may be worth studying its impact on WAL insert performance, still having more of these induces an extra CPU cost to flushing the WAL where scans across more locks would need to happen. All the details around that are mostly documented within XLogInsertRecord(), where the flow behind a WAL insertion is explained (space reserved within a page, page boundary crossed, etc.). The three routines mentioned above have a behavior clearly documented in lwlock.c: LWLockUpdateVar() requires first a lock to be acquired with LWLockAcquire(). This is in charge of updating a variable pointer located in shared memory to a new value, waking up any processes waiting for an update. LWLockWaitForVar() can be used to wait for a variable to be updated. It should point to the same pointer as LWLockUpdateVar() so as a fresh value can be grabbed. LWLockReleaseClearVar() would happen after as a last cleanup phase, resetting the variable to wait on to a default value. As referring only to the PostgreSQL core code to get an idea of what these routines can do may be limited when put into action, so I have written a small module called lwlock_test that uses these APIs and is able to do the following, with two backends able to play an automated ping-pong game, each one of them waiting for variable update[...]