1<!doctype html public "-//W3C//DTD HTML 4.01 Transitional//EN" 2 "http://www.w3.org/TR/html4/loose.dtd"> 3 4<html> 5 6<head> 7 8<title>Postfix Before-Queue Content Filter </title> 9 10<meta http-equiv="Content-Type" content="text/html; charset=us-ascii"> 11 12</head> 13 14<body> 15 16<h1><img src="postfix-logo.jpg" width="203" height="98" ALT="">Postfix Before-Queue Content Filter </h1> 17 18<hr> 19 20<h2>WARNING </h2> 21 22<p> The before-queue content filtering feature described in this 23document limits the amount of mail that a site can handle. See the 24"<a href="#pros_cons">Pros and Cons</a>" section below for details. 25</p> 26 27<h2>The Postfix before-queue content filter feature</h2> 28 29<p> As of version 2.1, the Postfix SMTP server can forward all 30incoming mail to a content filtering proxy server that inspects all 31mail BEFORE it is stored in the Postfix mail queue. It is roughly 32equivalent in capabilities to the approach described in <a href="MILTER_README.html">MILTER_README</a>, 33except that the latter uses a dedicated protocol instead of SMTP. 34 35<p> The before-queue content filter is meant to be used as follows: </p> 36 37<blockquote> 38 39<table> 40 41<tr> 42 43 <td bgcolor="#f0f0ff" align="center" valign="middle" 44 width="10%"> Internet </td> 45 46 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 47 48 <td bgcolor="#f0f0ff" align="center" valign="middle" 49 width="10%"> <a href="smtpd.8.html">Postfix SMTP server</a> 50 </td> 51 52 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 53 54 <td bgcolor="#f0f0ff" align="center" valign="middle" 55 width="10%"> <b>Before</b> <b>queue</b> <b>filter</b> </td> 56 57 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 58 59 <td bgcolor="#f0f0ff" align="center" valign="middle" 60 width="10%"> <a href="smtpd.8.html">Postfix SMTP server</a> 61 </td> 62 63 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 64 65 <td bgcolor="#f0f0ff" align="center" valign="middle" 66 width="10%"> <a href="cleanup.8.html">Postfix cleanup 67 server</a> </td> 68 69 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 70 71 <td bgcolor="#f0f0ff" align="center" valign="middle" 72 width="10%"> Postfix queue </td> 73 74 <td align="center" valign="middle" width="5%"> <tt> -< </tt> </td> 75 76 <td bgcolor="#f0f0ff" align="center" valign="middle" 77 width="10%"> <a href="smtp.8.html">smtp</a><br> <a 78 href="local.8.html">local</a><br> <a 79 href="virtual.8.html">virtual</a> </td> 80 81</tr> 82 83</table> 84 85</blockquote> 86 87<p> The before-queue content filter is not to be confused with the 88approach described in the <a href="FILTER_README.html">FILTER_README</a> document, where mail is 89filtered AFTER it is stored in the Postfix mail queue. </p> 90 91<p> This document describes the following topics: </p> 92 93<ul> 94 95<li><a href="#principles">Principles of operation</a> 96 97<li><a href="#pros_cons">Pros and cons of before-queue content filtering</a> 98 99<li><a href="#config">Configuring the Postfix SMTP pass-through 100proxy feature</a> 101 102<li><a href="#parameters">Configuration parameters</a> 103 104<li><a href="#protocol">How Postfix talks to the before-queue content 105filter</a> 106 107</ul> 108 109<h2><a name="principles">Principles of operation</a></h2> 110 111<p> As shown in the diagram above, the before-queue filter sits 112between two Postfix SMTP server processes. </p> 113 114<ul> 115 116<li> <p> The before-filter Postfix SMTP server accepts connections from the 117Internet and does the usual relay access control, SASL authentication, 118TLS negotiation, 119RBL lookups, rejecting non-existent sender or recipient addresses, 120etc. </p> 121 122<li> <p> The before-queue filter receives unfiltered mail content from 123Postfix and does one of the following: </p> 124 125<ol> 126 127 <li> <p> Re-inject the mail back into Postfix via SMTP, perhaps 128 after changing its content and/or destination. </p> 129 130 <li> <p> Discard or quarantine the mail. </p> 131 132 <li> <p> Reject the mail by sending a suitable SMTP status code 133 back to Postfix. Postfix passes the status back to the remote 134 SMTP client. This way, Postfix does not have to send a bounce 135 message. </p> 136 137</ol> 138 139<li> <p>The after-filter Postfix SMTP server receives mail from the 140content filter. From then on Postfix processes the mail as usual. </p> 141 142</ul> 143 144<p> The before-queue content filter described here works just like 145the after-queue content filter described in the <a href="FILTER_README.html">FILTER_README</a> 146document. In many cases you can use the same software, within the 147limitations as discussed in the "<a href="#pros_cons">Pros and 148Cons</a>" section below. </p> 149 150<h2><a name="pros_cons">Pros and cons of before-queue content 151filtering</a></h2> 152 153<ul> 154 155<li> <p> Pro: Postfix can reject mail before the incoming SMTP mail 156transfer completes, so that Postfix does not have to send rejected 157mail back to the sender (which is usually forged anyway). Mail 158that is not accepted remains the responsibility of the remote SMTP 159client. </p> 160 161<li> <p> Con: The remote SMTP client expects an SMTP reply within 162a deadline. As the system load increases, fewer and fewer CPU 163cycles remain available to answer within the deadline, and eventually 164you either have to stop accepting mail or you have to stop filtering 165mail. It is for this reason that the before-queue content filter 166limits the amount of mail that a site can handle. </p> 167 168<li> <p> Con: Content filtering software can use lots of memory 169resources. You have to reduce the number of simultaneous content 170filter processes so that a burst of mail will not drive your system 171into the ground. </p> 172 173<ul> 174 175<li> <p> With Postfix versions 2.7 and later, SMTP clients will 176experience an increase in the delay between the time the client 177sends "end-of-message" and the time the Postfix SMTP server replies 178(here, the number of before-filter SMTP server processes can be 179larger than the number of filter processes). </p> 180 181<li> <p> With Postfix versions before 2.7, SMTP clients will 182experience an increase in the delay before they can receive service 183(here, the number of before-filter SMTP server processes is always 184equal to the number of filter processes). </p> 185 186</ul> 187 188</ul> 189 190<h2><a name="config">Configuring the Postfix SMTP pass-through 191proxy feature</a></h2> 192 193<p> In the following example, the before-filter Postfix SMTP server 194gives mail to a content filter that listens on localhost port 10025. 195The after-filter Postfix SMTP server receives mail from the content 196filter via localhost port 10026. From then on mail is processed as 197usual. </p> 198 199<p> The content filter itself is not described here. You can use 200any filter that is SMTP enabled. For non-SMTP capable content 201filtering software, Bennett Todd's SMTP proxy implements a nice 202Perl-based framework. See: <a href="http://bent.latency.net/smtpprox/">http://bent.latency.net/smtpprox/</a> or 203https://github.com/jnorell/smtpprox.</p> 204 205<blockquote> 206 207<table border="0"> 208 209<tr> 210 211 <td bgcolor="#f0f0ff" align="center" valign="middle" 212 width="10%"> Internet </td> 213 214 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 215 216 <td bgcolor="#f0f0ff" align="center" valign="middle" 217 width="10%"> <a href="smtpd.8.html">Postfix SMTP server on 218 port 25</a> </td> 219 220 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 221 222 <td bgcolor="#f0f0ff" align="center" valign="middle" 223 width="10%"> filter on localhost port 10025 </td> 224 225 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 226 227 <td bgcolor="#f0f0ff" align="center" valign="middle" 228 width="10%"> <a href="smtpd.8.html">Postfix SMTP server on 229 localhost port 10026</a> </td> 230 231 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 232 233 <td bgcolor="#f0f0ff" align="center" valign="middle" 234 width="10%"> <a href="cleanup.8.html">Postfix cleanup 235 server</a> </td> 236 237 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 238 239 <td bgcolor="#f0f0ff" align="center" valign="middle" 240 width="10%"> Postfix <a href="QSHAPE_README.html#incoming_queue">incoming queue</a> </td> 241 242</tr> 243 244</table> 245 246</blockquote> 247 248<p> This is configured by editing the <a href="master.5.html">master.cf</a> file: </p> 249 250<blockquote> 251<pre> 252/etc/postfix/<a href="master.5.html">master.cf</a>: 253 # ============================================================= 254 # service type private unpriv chroot wakeup maxproc command 255 # (yes) (yes) (yes) (never) (100) 256 # ============================================================= 257 # 258 # Before-filter SMTP server. Receive mail from the network and 259 # pass it to the content filter on localhost port 10025. 260 # 261 smtp inet n - n - 20 smtpd 262 -o <a href="postconf.5.html#smtpd_proxy_filter">smtpd_proxy_filter</a>=127.0.0.1:10025 263 -o <a href="postconf.5.html#smtpd_client_connection_count_limit">smtpd_client_connection_count_limit</a>=10 264 # Postfix 2.7 and later performance feature. 265 # -o <a href="postconf.5.html#smtpd_proxy_options">smtpd_proxy_options</a>=speed_adjust 266 # 267 # After-filter SMTP server. Receive mail from the content filter 268 # on localhost port 10026. 269 # 270 127.0.0.1:10026 inet n - n - - smtpd 271 -o <a href="postconf.5.html#smtpd_authorized_xforward_hosts">smtpd_authorized_xforward_hosts</a>=127.0.0.0/8 272 -o <a href="postconf.5.html#smtpd_client_restrictions">smtpd_client_restrictions</a>= 273 -o <a href="postconf.5.html#smtpd_helo_restrictions">smtpd_helo_restrictions</a>= 274 -o <a href="postconf.5.html#smtpd_sender_restrictions">smtpd_sender_restrictions</a>= 275 # Postfix 2.10 and later: specify empty <a href="postconf.5.html#smtpd_relay_restrictions">smtpd_relay_restrictions</a>. 276 -o <a href="postconf.5.html#smtpd_relay_restrictions">smtpd_relay_restrictions</a>= 277 -o <a href="postconf.5.html#smtpd_recipient_restrictions">smtpd_recipient_restrictions</a>=<a href="postconf.5.html#permit_mynetworks">permit_mynetworks</a>,reject 278 -o <a href="postconf.5.html#smtpd_data_restrictions">smtpd_data_restrictions</a>= 279 -o <a href="postconf.5.html#mynetworks">mynetworks</a>=127.0.0.0/8 280 -o <a href="postconf.5.html#receive_override_options">receive_override_options</a>=<a href="postconf.5.html#no_unknown_recipient_checks">no_unknown_recipient_checks</a> 281</pre> 282</blockquote> 283 284<p> Note: do not specify spaces around the "=" or "," characters. </p> 285 286<p> The before-filter SMTP server entry is a modified version of the 287default Postfix SMTP server entry that is normally configured at 288the top of the <a href="master.5.html">master.cf</a> file: </p> 289 290<ul> 291 292 <li> <p> The number of SMTP sessions is reduced from the default 293 100 to only 20. This prevents a burst of mail from running your 294 system into the ground with too many content filter processes. </p> 295 296 <li> <p> The "-o <a href="postconf.5.html#smtpd_client_connection_count_limit">smtpd_client_connection_count_limit</a>=10" prevents 297 one SMTP client from using up all 20 SMTP server processes. 298 This limit is not necessary if you receive all mail from a 299 trusted <a href="postconf.5.html#relayhost">relay host</a>. </p> 300 301 <p> Note: this setting is available in Postfix version 2.2 and 302 later. Earlier Postfix versions will ignore it. </p> 303 304 <li> <p> The "-o <a href="postconf.5.html#smtpd_proxy_filter">smtpd_proxy_filter</a>=127.0.0.1:10025" tells the 305 before-filter SMTP server that it should give incoming mail to 306 the content filter that listens on localhost TCP port 10025. 307 308 <li> <p> The "-o <a href="postconf.5.html#smtpd_proxy_options">smtpd_proxy_options</a>=speed_adjust" tells the 309 before-filter SMTP server that it should receive an entire email 310 message before it connects to a content filter. This reduces 311 the number of simultaneous filter processes. </p> 312 313 <p> NOTE 1: When this option is turned on, a content filter must 314 not <i>selectively</i> reject recipients of a multi-recipient 315 message. Rejecting all recipients is OK, as is accepting all 316 recipients. </p> 317 318 <p> NOTE 2: This feature increases the minimum amount of free 319 queue space by $<a href="postconf.5.html#message_size_limit">message_size_limit</a>. The extra space is needed 320 to save the message to a temporary file. </p> 321 322 <li> <p> Postfix ≥ 2.3 supports both TCP and UNIX-domain filters. 323 The above filter could be specified as "inet:127.0.0.1:10025". 324 To specify a UNIX-domain filter, specify "unix:<i>pathname</i>". 325 A relative pathname is interpreted relative to the Postfix queue 326 directory. </p> 327 328</ul> 329 330<p> The after-filter SMTP server is a new <a href="master.5.html">master.cf</a> entry: </p> 331 332<ul> 333 334 <li> <p> The "127.0.0.1:10026" makes the after-filter SMTP 335 server listen 336 on the localhost address only, without exposing it to the 337 network. NEVER expose the after-filter SMTP server to the 338 Internet :-) </p> 339 340 <li> <p> The "-o <a href="postconf.5.html#smtpd_authorized_xforward_hosts">smtpd_authorized_xforward_hosts</a>=127.0.0.0/8" 341 allows the after-filter SMTP server to receive remote SMTP 342 client information from the before-filter SMTP server, so that 343 the after-filter Postfix daemons log the remote SMTP client 344 information instead of logging localhost[127.0.0.1]. </p> 345 346 <li> <p> The other after-filter SMTP server settings avoid 347 duplication of work that is already done in the "before filter" 348 SMTP server. </p> 349 350</ul> 351 352<p> By default, the filter has 100 seconds to do its work. If it 353takes longer then Postfix gives up and reports an error to the 354remote SMTP client. You can increase this time limit (see configuration 355parameter section below) but doing so is pointless because you 356can't control when the remote SMTP client times out. </p> 357 358<h2><a name="parameters">Configuration parameters</a></h2> 359 360<p> Parameters that control proxying: </p> 361 362<ul> 363 364<li> <p> <a href="postconf.5.html#smtpd_proxy_filter">smtpd_proxy_filter</a> (syntax: host:port): The host and TCP 365port of the before-queue content filter. When no host or host: 366is specified here, localhost is assumed. </p> 367 368<li> <p> <a href="postconf.5.html#smtpd_proxy_timeout">smtpd_proxy_timeout</a> (default: 100s): Timeout for connecting 369to the before-queue content filter and for sending and receiving 370commands and data. All proxy errors are logged to the maillog 371file. For privacy reasons, all the remote SMTP client sees is "451 372Error: queue file write error". It would not be right to disclose 373internal details to strangers. </p> 374 375<li> <p> <a href="postconf.5.html#smtpd_proxy_ehlo">smtpd_proxy_ehlo</a> (default: $<a href="postconf.5.html#myhostname">myhostname</a>): The hostname to 376use when sending an EHLO command to the before-queue content filter. 377</p> 378 379</ul> 380 381<h2><a name="protocol">How Postfix talks to the before-queue content 382filter</a></h2> 383 384<p> The before-filter Postfix SMTP server connects to the content 385filter, delivers one message, and disconnects. While sending mail 386into the content filter, Postfix speaks ESMTP but uses no command 387pipelining. Postfix generates its own EHLO, XFORWARD (for logging 388the remote client IP address instead of localhost[127.0.0.1]), DATA 389and QUIT commands, and forwards unmodified copies of all the MAIL 390FROM and RCPT TO commands that the before-filter Postfix SMTP server 391didn't reject itself. 392Postfix sends no other SMTP commands. </p> 393 394<p> The content filter should accept the same MAIL FROM and RCPT 395TO command syntax as the before-filter Postfix SMTP server, and 396should forward the commands without modification to the after-filter 397SMTP server. If the content filter or after-filter SMTP server 398does not support all the ESMTP features that the before-filter 399Postfix SMTP server supports, then the missing features must be 400turned off in the before-filter Postfix SMTP server with the 401<a href="postconf.5.html#smtpd_discard_ehlo_keywords">smtpd_discard_ehlo_keywords</a> parameter. </p> 402 403<p> When the filter rejects content, it should send a negative SMTP 404response back to the before-filter Postfix SMTP server, and it 405should abort the connection with the after-filter Postfix SMTP 406server without completing the SMTP conversation with the after-filter 407Postfix SMTP server. </p> 408 409</body> 410 411</html> 412
