dist/html/LMDB_README.html

<!doctype html public "-//W3C//DTD HTML 4.01 Transitional//EN"
        "http://www.w3.org/TR/html4/loose.dtd">

<html>

<head>

<title>Postfix OpenLDAP LMDB Howto</title>

<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">

</head>

<body>

<h1><img src="postfix-logo.jpg" width="203" height="98" ALT="">Postfix OpenLDAP LMDB Howto</h1>

<hr>

<h2>Introduction</h2>

<p> Postfix uses databases of various kinds to store and look up
information. Postfix databases are specified as "type:name".  OpenLDAP
LMDB (called "LMDB" from here on) implements the Postfix database
type "lmdb".  The name of a Postfix LMDB database is the name of
the database file without the ".lmdb" suffix. </p>

<p> This document describes: </p>

<ul>

<li> <p> <a href="#with_lmdb">Building Postfix with LMDB support</a>.
</p>

<li> <p> <a href="#configure">Configuring LMDB settings</a>. </p>

<li> <p> <a href="#locking">Using LMDB maps with non-Postfix programs</a>. </p>

<li> <p> <a href="#supported"> Required minimum LMDB patchlevel</a>. </p>

<li> <p> <a href="#credits"> Credits</a>. </p>

</ul>

<h2><a name="with_lmdb">Building Postfix with LMDB support</a></h2>

<p> Postfix normally does not enable LMDB support.  To
build Postfix with LMDB support, use something like: </p>

<blockquote>
<pre>
% make makefiles CCARGS="-DHAS_LMDB -I/usr/local/include" \
    <a href="LMDB_README.html">AUXLIBS_LMDB</a>="-L/usr/local/lib -llmdb"
% make
</pre>
</blockquote>

<p> Postfix versions before 3.0 use AUXLIBS instead of <a href="LMDB_README.html">AUXLIBS_LMDB</a>.
With Postfix 3.0 and later, the old AUXLIBS variable still supports
building a statically-loaded LMDB database client, but only the new
<a href="LMDB_README.html">AUXLIBS_LMDB</a> variable supports building a dynamically-loaded or
statically-loaded LMDB database client.  </p>

<blockquote>

<p> Failure to use the <a href="LMDB_README.html">AUXLIBS_LMDB</a> variable will defeat the purpose
of dynamic database client loading. Every Postfix executable file
will have LMDB database library dependencies. And that was exactly
what dynamic database client loading was meant to avoid. </p>

</blockquote>


<p> Solaris may need this: </p>

<blockquote>
<pre>
% make makefiles CCARGS="-DHAS_LMDB -I/usr/local/include" \
    <a href="LMDB_README.html">AUXLIBS_LMDB</a>="-R/usr/local/lib -L/usr/local/lib -llmdb"
% make
</pre>
</blockquote>

<p> The exact pathnames depend on how LMDB was installed. </p>

<p> When building Postfix fails with: </p>

<blockquote>
<pre>
undefined reference to `pthread_mutexattr_destroy'
undefined reference to `pthread_mutexattr_init'
undefined reference to `pthread_mutex_lock'
</pre>
</blockquote>

<p> Add the "-lpthread" library to the "make makefiles" command. </p>

<blockquote>
<pre>
% make makefiles .... <a href="LMDB_README.html">AUXLIBS_LMDB</a>="... -lpthread"
</pre>
</blockquote>

<h2><a name="configure">Configuring LMDB settings</a></h2>

<p> Postfix provides one configuration parameter that controls
LMDB database behavior. </p>

<ul>

<li> <p> <a href="postconf.5.html#lmdb_map_size">lmdb_map_size</a> (default: 16777216).  This setting specifies
the initial LMDB database size limit in bytes.  Each time a database
becomes "full", its size limit is doubled. The maximum size is the
largest signed integer value of "long". </p>

</ul>

<h2> <a name="locking">Using LMDB maps with non-Postfix programs</a> </h2>

<p> Programs that use LMDB's built-in locking protocol will corrupt
a Postfix LMDB database or will read garbage. </p>

<p> Postfix does not use LMDB's built-in locking protocol, because
that would require world-writable lockfiles, and would violate
Postfix security policy.  Instead, Postfix uses external locks based
on fcntl(2) to prevent writers from corrupting the database, and
to prevent readers from receiving garbage.  </p>

<p> See <a href="lmdb_table.5.html">lmdb_table(5)</a> for a detailed description of the locking
protocol that all programs must use when they access a Postfix LMDB
database. </p>

<h2> <a name="supported"> Required minimum LMDB patchlevel </a> </h2>

<p> Currently, Postfix requires LMDB 0.9.11 or later. The required
minimum LMDB patchlevel has evolved over time, as the result of
Postfix deployment experience: </p>

<ul>

<li> <p> LMDB 0.9.11 allows Postfix daemons to log an LMDB error
message, instead of falling out of the sky without any notification.
</p>

<li> <p> LMDB 0.9.10 closes an information leak where LMDB was
writing up to 4-kbyte chunks of uninitialized heap memory to the
database. This would persist information that was not meant to be
persisted, or share information that was not meant to be shared.
</p>

<li> <p> LMDB 0.9.9 allows Postfix to use external (fcntl()-based)
locks, instead of having to use world-writable LMDB lock files,
violating the Postfix security model in multiple ways. </p>

<li> <p> LMDB 0.9.8 allows Postfix to recover from a "database full"
error without having to close the database. This version adds support
to update the database size limit on-the-fly.  This is necessary
because Postfix database sizes vary with mail server load. </p>

<li> <p> LMDB 0.9.7 allows the <a href="postmap.1.html">postmap(1)</a> and <a href="postalias.1.html">postalias(1)</a> commands
to use a bulk-mode transaction larger than the amount of physical
memory. This is necessary because LMDB supports databases larger
than physical memory. </p>

</ul>

<h2> <a name="credits"> Credits</a> </h2>

<ul>

<li> <p> Howard Chu contributed the initial Postfix dict_lmdb driver.
</p>

<li> <p> Wietse Venema wrote an abstraction layer (slmdb) that
behaves more like Berkeley DB, NDBM, etc.  This layer automatically
retries an LMDB request when a database needs to be resized, or
after a database was resized by a different process.  </p>

<li> <p> Howard and Wietse went through many iterations with changes
to both LMDB and Postfix, with input from Viktor Dukhovni. </p>

</ul>

<!--

<h2><a name="limitations">Unexpected failure modes of Postfix LMDB
databases.  </a> </h2>

<p> As documented below, conversion to LMDB introduces a number of
failure modes that don't exist with other Postfix databases.  Some
failure modes have been eliminated in the course of time.
The writeup below reflects the status as of LMDB 0.9.9. </p>

-->

<!--

<p> <strong>Unexpected "Permission denied" errors. </strong></p>

<dl>

<dt> Problem: </dt> <dd> <p> A world-readable LMDB database cannot
be opened by a process with a UID that differs from the database
file owner, even when an attempt is made to open the database
read-only.  This problem does not exist with other Postfix databases.
</p> </dd>

<dt> Background: </dt> <dd> <p> The LMDB implementation requires
write access to maintain read locks, and perhaps for other purposes.
</p> </dd>

<dt> Solution: </dt> <dd> <p> Consider using <a href="CDB_README.html">cdb</a>: to manage root-owned
databases under the root-owned <tt>/etc</tt> or <a href="postconf.5.html#config_directory">config_directory</a>
(default: <tt>/etc/postfix</tt>) such as <a href="access.5.html">access(5)</a>, <a href="virtual.5.html">virtual(5)</a>,
<a href="transport.5.html">transport(5)</a>. Support to create LMDB databases is available only
for unprivileged Postfix daemon processes such as <a href="postscreen.8.html">postscreen(8)</a>,
<a href="tlsmgr.8.html">tlsmgr(8)</a> and <a href="verify.8.html">verify(8)</a> that manage postfix-owned databases under
the postfix-owned <a href="postconf.5.html#data_directory">data_directory</a> (default: <tt>/var/lib/postfix</tt>).
</p> </dd>

</dl>

-->

<!--

<p> <strong>Unexpected "readers full" errors. </strong></p>

<dl>

<dt> Problem: </dt> <dd> <p> Under heavy load, database read
operations fail with MDB_READERS_FULL errors. This problem does not
exist with other Postfix databases. </p> </dd>

<dt> Background: </dt> <dd> <p> The LMDB implementation enforces a
hard limit on the number of simultaneous read requests for the same
database environment. This limit must be specified in advance with
the lmdb_max_readers configuration parameter. </p> </dd>

<dt> Mitigation: </dt> <dd> <p> Postfix logs a warning suggesting
that the lmdb_max_readers parameter value be increased, and retries
the failed operation for a limited number of times while running
with reduced performance.  </p> </dd>

<dt> Prevention: </dt> <dd> <p> Monitor your LMDB files for
MDB_READERS_FULL errors. After making the necessary adjustments,
restart Postfix. </p> </dd>

</dl>

-->

<!--

<p> <strong>Unexpected <a href="postmap.1.html">postmap(1)</a>/<a href="postalias.1.html">postalias(1)</a> "database full"
errors.  </strong></p>

<dl>

<dt> Problem: </dt> <dd> <p> The "postmap <a href="lmdb_table.5.html">lmdb</a>:filename" command
fails with an MDB_TXN_FULL error.  This problem does not exist with
other Postfix databases. </p> </dd>

<dt> Background: </dt>

<dd>

<p> The LMDB implementation has a hard limit on the total transaction
size. This limit is independent of the LMDB database size. Therefore,
the problem cannot be resolved by increasing the <a href="postconf.5.html#lmdb_map_size">lmdb_map_size</a>
value. </p>

<p> This symptom is indicative of a flawed design.  All LMDB data
structures should share the same storage pool so that they can scale
with the database size, and so that all "out of storage" errors are
resolved by increasing the database size. </p> </dd>

-->

<!--

<p> Problem: </dt> <dd> <p> The "postmap <a href="lmdb_table.5.html">lmdb</a>:filename" command
fails with an MDB_MAP_FULL error.  This problem does not exist with
other Postfix databases. </p> </dd>

<dl>

<dt> Background: </dt>

<dd>

<p> LMDB databases have a hard size limit (configured with the
<a href="postconf.5.html#lmdb_map_size">lmdb_map_size</a> configuration parameter). </p>

<p> When executing "postmap <a href="lmdb_table.5.html">lmdb</a>:filename", the Postfix LMDB database
client stores the new data in a transaction which takes up space
in addition to the existing data, and commits the transaction when
it closes the database.  Only then can the space for old data be
reused.  </p>

</dd>

<dt> Impact: </dt> <dd> <p> This failure does not affect Postfix
availability, because the old data still exists in the database.
</p> </dd>

<dt> Mitigation: </dt> <dd>

<p> When the <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> command fails with an
MDB_MAP_FULL error, it expands the database file size to the current
LMDB map size limit before terminating.  </p>

<p> Next, when you re-run the <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> command,
it discovers that the LMDB file is larger than <a href="postconf.5.html#lmdb_map_size">lmdb_map_size</a>/3,
logs a warning, and uses a larger LMDB map size limit instead: </p>

<p> <tt> warning: <i>filename</i>.<a href="lmdb_table.5.html">lmdb</a>: file size 15024128 &ge;
(lmdb map size limit 16777216)/3<br> warning: <i>filename</i>.<a href="lmdb_table.5.html">lmdb</a>:
using map size limit 45072384</tt> </p>

<p> By repeating the two steps above you can automate recovery and
avoid the need for human intervention. Just repeat "postmap
<a href="lmdb_table.5.html">lmdb</a>:filename" (up to some limit).  After each failure it will use
a 3x larger size limit, and eventually the "database full" error
should disappear. This fails only when the disk is full or when
the LMDB map size limit would exceed the memory address space size
limit.  </p>

<dt> Prevention: </dt> <dd> <p> Monitor your LMDB files and make
sure that in <a href="postconf.5.html">main.cf</a>, <a href="postconf.5.html#lmdb_map_size">lmdb_map_size</a> &gt; 3x the largest LMDB file
size. </p> </dd> </dl>

</dl>

-->

<!--

<p> <strong>Unexpected Postfix daemon "database full" errors.
</strong></p>

<dl>

<dt> Problem: </dt> <dd> <p> Postfix daemon programs fail with
"database full" errors, such as <a href="postscreen.8.html">postscreen(8)</a>, <a href="tlsmgr.8.html">tlsmgr(8)</a> or <a href="verify.8.html">verify(8)</a>.
This problem does not exist with other Postfix databases.  </p>
</dd>

<dt> Impact: </dt> <dd> <p> This failure temporarily affects Postfix
availability. The daemon restarts automatically and tries to open
the database again as described next.  </p> </dd>

<dt> Mitigation: </dt> <dd> <p> When a Postfix daemon opens an LMDB
file larger than <a href="postconf.5.html#lmdb_map_size">lmdb_map_size</a>/3, it logs a warning and uses a
larger size limit instead: </p>

<p> <tt> warning: <i>filename</i>.<a href="lmdb_table.5.html">lmdb</a>: file size 15024128 &ge;
(lmdb map size limit 16777216)/3 <br>warning: <i>filename</i>.<a href="lmdb_table.5.html">lmdb</a>:
using map size limit 45072384</tt> </p>

<p> This can be used to automate recovery and avoid the need for
human intervention. Each time the daemon runs into a "database full"
error, it restarts and uses a 3x larger size limit. The "database
full" error will disappear, at least for a while.  </p>

<dt> Prevention: </dt> <dd> <p> Monitor your LMDB files and make
sure that <a href="postconf.5.html#lmdb_map_size">lmdb_map_size</a> &gt; 3x the largest LMDB file size. </p>
</dd> </dl>

-->

<!--

<p> <strong>Non-obvious recovery with <a href="postmap.1.html">postmap(1)</a>, <a href="postalias.1.html">postalias(1)</a>, or
<a href="tlsmgr.8.html">tlsmgr(8)</a> from a corrupted database.  </strong></p>

<dl>

<dt> Problem: </dt> <dd> <p> A corrupted LMDB database can't be
rebuilt simply by re-running <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a>, or by
waiting until a <a href="tlsmgr.8.html">tlsmgr(8)</a> daemon restarts.  This problem does not
exist with other Postfix databases.  </p> </dd>

<dt> Background: </dt> <dd> <p> The Postfix LMDB database client
does not truncate the database file.  Instead it attempts to create
a transaction for a "drop" request plus subsequent "store" requests.
That is obviously not possible with a corrupted database file. </p>
</dd>

<dt> Impact: </dt> <dd> <p> Postfix does not process mail until
someone fixes the problem.  </p> </dd>

<dt> Recovery: </dt> <dd> <p> First delete the ".lmdb" file by hand.
Then rebuild the file with the <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a>
command if the file was created with those commands, or restart
postfix daemons if the file is maintained by <a href="tlsmgr.8.html">tlsmgr(8)</a>.
</p> </dd>

<dt> Prevention: </dt> <dd>

<p> Arrange your file systems such that they never run out of free
space. </p>

<p> Use ECC memory to detect and correct silent corruption of
in-memory file system data and metadata. </p>

<p> Use a file system such as ZFS to detect and correct silent
corruption of on-disk file system data and metadata. DO NOT
use ZFS on systems without ECC memory error correction. </p>

</dd> </dl>

-->

</body>

</html>