1# $OpenLDAP$ 2# Copyright 1999-2021 The OpenLDAP Foundation, All Rights Reserved. 3# COPYING RESTRICTIONS APPLY, see COPYRIGHT. 4 5H1: Tuning 6 7This is perhaps one of the most important chapters in the guide, because if 8you have not tuned {{slapd}}(8) correctly or grasped how to design your 9directory and environment, you can expect very poor performance. 10 11Reading, understanding and experimenting using the instructions and information 12in the following sections, will enable you to fully understand how to tailor 13your directory server to your specific requirements. 14 15It should be noted that the following information has been collected over time 16from our community based FAQ. So obviously the benefit of this real world experience 17and advice should be of great value to the reader. 18 19 20H2: Performance Factors 21 22Various factors can play a part in how your directory performs on your chosen 23hardware and environment. We will attempt to discuss these here. 24 25 26H3: Memory 27 28Scale your cache to use available memory and increase system memory if you can. 29 30 31H3: Disks 32 33Use fast filesystems, and conduct your own testing to see which filesystem 34types perform best with your workload. (On our own Linux testing, EXT2 and JFS 35tend to provide better write performance than everything else, including 36newer filesystems like EXT4, BTRFS, etc.) 37 38Use fast subsystems. Put each database on separate disks. 39 40H3: Network Topology 41 42http://www.openldap.org/faq/data/cache/363.html 43 44Drawing here. 45 46 47H3: Directory Layout Design 48 49Reference to other sections and good/bad drawing here. 50 51 52H3: Expected Usage 53 54Discussion. 55 56 57H2: Indexes 58 59H3: Understanding how a search works 60 61If you're searching on a filter that has been indexed, then the search reads 62the index and pulls exactly the entries that are referenced by the index. 63If the filter term has not been indexed, then the search must read every single 64 entry in the target scope and test to see if each entry matches the filter. 65Obviously indexing can save a lot of work when it's used correctly. 66 67In back-mdb, indexes can only track a certain number of entries per key (by 68default that number is 2^16 = 65536). If more entries' values hash to this 69key, some/all of them will have to be represented by a range of candidates, 70making the index less useful over time as deletions cannot usually be tracked 71accurately. 72 73H3: What to index 74 75As a general rule, to make any use of indexes, you must set up an equality 76index on objectClass: 77 78> index objectClass eq 79 80Then you should create indices to match the actual filter terms used in 81search queries. 82 83> index cn,sn,givenname,mail eq 84 85Each attribute index can be tuned further by selecting the set of index types to generate. For example, substring and approximate search for organizations (o) may make little sense (and isn't like done very often). And searching for {{userPassword}} likely makes no sense what so ever. 86 87General rule: don't go overboard with indexes. Unused indexes must be maintained and hence can only slow things down. 88 89See {{slapd.conf}}(5) and {{slapdindex}}(8) for more information 90 91 92H3: Presence indexing 93 94If your client application uses presence filters and if the 95target attribute exists on the majority of entries in your target scope, then 96all of those entries are going to be read anyway, because they are valid 97members of the result set. In a subtree where 100% of the 98entries are going to contain the same attributes, the presence index does 99absolutely NOTHING to benefit the search, because 100% of the entries match 100that presence filter. As an example, setting a presence index on objectClass 101provides no benefit since it is present on every entry. 102 103So the resource cost of generating the index is a 104complete waste of CPU time, disk, and memory. Don't do it unless you know 105that it will be used, and that the attribute in question occurs very 106infrequently in the target data. 107 108Almost no applications use presence filters in their search queries. Presence 109indexing is pointless when the target attribute exists on the majority of 110entries in the database. In most LDAP deployments, presence indexing should 111not be done, it's just wasted overhead. 112 113See the {{Logging}} section below on what to watch out for if you have a frequently searched 114for attribute that is unindexed. 115 116H3: Equality indexing 117 118Similarly to presence indexes, equality indexes are most useful if the 119values searched for are uncommon. Most OpenLDAP indexes work by hashing 120the normalised value and using the hash as the key. Hashing behaviour 121depends on the matching rule syntax, some matching rules also implement 122indexers that help speed up inequality (lower than, ...) queries. 123 124Check the documentation and other parts of this guide if some indexes are 125mandatory - e.g. to enable replication, it is expected you index certain 126operational attributes, likewise if you rely on filters in ACL processing. 127 128Approximate indexes are usually identical to equality indexes unless 129a matching rule explicitly implements it. As of OpenLDAP 2.5, only 130directoryStringApproxMatch and IA5StringApproxMatch matchers 131and indexers are implemented, currently using soundex or metaphone, with 132metaphone being the default. 133 134H3: Substring indexing 135 136Substring indexes work on splitting the value into short chunks and then 137indexing those in a similar way to how equality index does. The storage 138space needed to store all of this data is analogous to the amount of data 139being indexed, which makes the indexes extremely heavy-handed in most 140scenarios. 141 142 143H2: Logging 144 145H3: What log level to use 146 147The default of {{loglevel stats}} (256) is really the best bet. There's a corollary to 148this when problems *do* arise, don't try to trace them using syslog. 149Use the debug flag instead, and capture slapd's stderr output. syslog is too 150slow for debug tracing, and it's inherently lossy - it will throw away messages when it 151can't keep up. See {{slapd.conf}}(5) or {{slapd-config}}(5) for more information on 152how to configure the loglevel. 153 154Contrary to popular belief, {{loglevel 0}} is not ideal for production as you 155won't be able to track when problems first arise. 156 157H3: What to watch out for 158 159The most common message you'll see that you should pay attention to is: 160 161> "<= mdb_equality_candidates: (foo) index_param failed (18)" 162 163That means that some application tried to use an equality filter ({{foo=<somevalue>}}) 164and attribute {{foo}} does not have an equality index. If you see a lot of these 165messages, you should add the index. If you see one every month or so, it may 166be acceptable to ignore it. 167 168The default syslog level is stats (256) which logs the basic parameters of each 169request; it usually produces 1-3 lines of output. On Solaris and systems that 170only provide synchronous syslog, you may want to turn it off completely, but 171usually you want to leave it enabled so that you'll be able to see index 172messages whenever they arise. On Linux you can configure syslogd to run 173asynchronously, in which case the performance hit for moderate syslog traffic 174pretty much disappears. 175 176H3: Improving throughput 177 178You can improve logging performance on some systems by configuring syslog not 179to sync the file system with every write ({{man syslogd/syslog.conf}}). In Linux, 180you can prepend the log file name with a "-" in {{syslog.conf}}. For example, 181if you are using the default LOCAL4 logging you could try: 182 183> # LDAP logs 184> LOCAL4.* -/var/log/ldap 185 186For syslog-ng, add or modify the following line in {{syslog-ng.conf}}: 187 188> options { sync(n); }; 189 190where n is the number of lines which will be buffered before a write. 191 192 193H2: {{slapd}}(8) Threads 194 195{{slapd}}(8) can process requests via a configurable number of threads, which 196in turn affects the in/out rate of connections. 197 198This value should generally be a function of the number of "real" cores on 199the system, for example on a server with 2 CPUs with one core each, set this 200to 8, or 4 threads per real core. This is a "read" maximized value. The more 201threads that are configured per core, the slower {{slapd}}(8) responds for 202"read" operations. On the flip side, it appears to handle write operations 203faster in a heavy write/low read scenario. 204 205The upper bound for good read performance appears to be 16 threads (which 206also happens to be the default setting). 207