1<!doctype html public "-//W3C//DTD HTML 4.01 Transitional//EN" 2 "http://www.w3.org/TR/html4/loose.dtd"> 3 4<html> 5 6<head> 7 8<title>Postfix Before-Queue Content Filter </title> 9 10<meta http-equiv="Content-Type" content="text/html; charset=utf-8"> 11<link rel='stylesheet' type='text/css' href='postfix-doc.css'> 12 13</head> 14 15<body> 16 17<h1><img src="postfix-logo.jpg" width="203" height="98" ALT="">Postfix Before-Queue Content Filter </h1> 18 19<hr> 20 21<h2>WARNING </h2> 22 23<p> The before-queue content filtering feature described in this 24document limits the amount of mail that a site can handle. See the 25"<a href="#pros_cons">Pros and Cons</a>" section below for details. 26</p> 27 28<h2>The Postfix before-queue content filter feature</h2> 29 30<p> As of version 2.1, the Postfix SMTP server can forward all 31incoming mail to a content filtering proxy server that inspects all 32mail BEFORE it is stored in the Postfix mail queue. It is roughly 33equivalent in capabilities to the approach described in MILTER_README, 34except that the latter uses a dedicated protocol instead of SMTP. 35 36<p> The before-queue content filter is meant to be used as follows: </p> 37 38<blockquote> 39 40<table> 41 42<tr> 43 44 <td bgcolor="#f0f0ff" align="center" valign="middle" 45 width="10%"> Internet </td> 46 47 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 48 49 <td bgcolor="#f0f0ff" align="center" valign="middle" 50 width="10%"> <a href="smtpd.8.html">Postfix SMTP server</a> 51 </td> 52 53 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 54 55 <td bgcolor="#f0f0ff" align="center" valign="middle" 56 width="10%"> <b>Before</b> <b>queue</b> <b>filter</b> </td> 57 58 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 59 60 <td bgcolor="#f0f0ff" align="center" valign="middle" 61 width="10%"> <a href="smtpd.8.html">Postfix SMTP server</a> 62 </td> 63 64 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 65 66 <td bgcolor="#f0f0ff" align="center" valign="middle" 67 width="10%"> <a href="cleanup.8.html">Postfix cleanup 68 server</a> </td> 69 70 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 71 72 <td bgcolor="#f0f0ff" align="center" valign="middle" 73 width="10%"> Postfix queue </td> 74 75 <td align="center" valign="middle" width="5%"> <tt> -< </tt> </td> 76 77 <td bgcolor="#f0f0ff" align="center" valign="middle" 78 width="10%"> <a href="smtp.8.html">smtp</a><br> <a 79 href="local.8.html">local</a><br> <a 80 href="virtual.8.html">virtual</a> </td> 81 82</tr> 83 84</table> 85 86</blockquote> 87 88<p> The before-queue content filter is not to be confused with the 89approach described in the FILTER_README document, where mail is 90filtered AFTER it is stored in the Postfix mail queue. </p> 91 92<p> This document describes the following topics: </p> 93 94<ul> 95 96<li><a href="#principles">Principles of operation</a> 97 98<li><a href="#pros_cons">Pros and cons of before-queue content filtering</a> 99 100<li><a href="#config">Configuring the Postfix SMTP pass-through 101proxy feature</a> 102 103<li><a href="#parameters">Configuration parameters</a> 104 105<li><a href="#protocol">How Postfix talks to the before-queue content 106filter</a> 107 108</ul> 109 110<h2><a name="principles">Principles of operation</a></h2> 111 112<p> As shown in the diagram above, the before-queue filter sits 113between two Postfix SMTP server processes. </p> 114 115<ul> 116 117<li> <p> The before-filter Postfix SMTP server accepts connections from the 118Internet and does the usual relay access control, SASL authentication, 119TLS negotiation, 120RBL lookups, rejecting non-existent sender or recipient addresses, 121etc. </p> 122 123<li> <p> The before-queue filter receives unfiltered mail content from 124Postfix and does one of the following: </p> 125 126<ol> 127 128 <li> <p> Re-inject the mail back into Postfix via SMTP, perhaps 129 after changing its content and/or destination. </p> 130 131 <li> <p> Discard or quarantine the mail. </p> 132 133 <li> <p> Reject the mail by sending a suitable SMTP status code 134 back to Postfix. Postfix passes the status back to the remote 135 SMTP client. This way, Postfix does not have to send a bounce 136 message. </p> 137 138</ol> 139 140<li> <p>The after-filter Postfix SMTP server receives mail from the 141content filter. From then on Postfix processes the mail as usual. </p> 142 143</ul> 144 145<p> The before-queue content filter described here works just like 146the after-queue content filter described in the FILTER_README 147document. In many cases you can use the same software, within the 148limitations as discussed in the "<a href="#pros_cons">Pros and 149Cons</a>" section below. </p> 150 151<h2><a name="pros_cons">Pros and cons of before-queue content 152filtering</a></h2> 153 154<ul> 155 156<li> <p> Pro: Postfix can reject mail before the incoming SMTP mail 157transfer completes, so that Postfix does not have to send rejected 158mail back to the sender (which is usually forged anyway). Mail 159that is not accepted remains the responsibility of the remote SMTP 160client. </p> 161 162<li> <p> Con: The smtpd(8) service before the smtpd_proxy_filter 163cannot support features that involve header or body access, or that 164involve queue file manipulation (i.e., anything that involves 165processing by the cleanup(8) service). </p> 166 167<ul> 168 169<li> <p> No support for HOLD actions in Postfix smtpd access(5) 170restrictions. </p> 171 172<li> <p> No support for smtpd_milters features that involve message 173header or body content. </p> 174 175<li> <p> No support for receive_override_options. 176 177</ul> 178 179<p> Instead, specify those features with the smtpd(8) service behind 180the smtpd_proxy_filter. In some cases, it may be possible to combine 181a before-filter PREPEND action that emits a unique pattern (for 182example containing the MTA domain name), with an after-filter 183header_checks action that does what you want, and with an 184smtp_header_checks IGNORE action that deletes the prepended header 185from transit mail. </p> 186 187<li> <p> Con: The remote SMTP client expects an SMTP reply within 188a deadline. As the system load increases, fewer and fewer CPU 189cycles remain available to answer within the deadline, and eventually 190you either have to stop accepting mail or you have to stop filtering 191mail. It is for this reason that the before-queue content filter 192limits the amount of mail that a site can handle. </p> 193 194<li> <p> Con: Content filtering software can use lots of memory 195resources. You have to reduce the number of simultaneous content 196filter processes so that a burst of mail will not drive your system 197into the ground. </p> 198 199<ul> 200 201<li> <p> With Postfix versions 2.7 and later, SMTP clients will 202experience an increase in the delay between the time the client 203sends "end-of-message" and the time the Postfix SMTP server replies 204(here, the number of before-filter SMTP server processes can be 205larger than the number of filter processes). </p> 206 207<li> <p> With Postfix versions before 2.7, SMTP clients will 208experience an increase in the delay before they can receive service 209(here, the number of before-filter SMTP server processes is always 210equal to the number of filter processes). </p> 211 212</ul> 213 214</ul> 215 216<h2><a name="config">Configuring the Postfix SMTP pass-through 217proxy feature</a></h2> 218 219<p> In the following example, the before-filter Postfix SMTP server 220gives mail to a content filter that listens on localhost port 10025. 221The after-filter Postfix SMTP server receives mail from the content 222filter via localhost port 10026. From then on mail is processed as 223usual. </p> 224 225<p> The content filter itself is not described here. You can use 226any filter that is SMTP enabled. For non-SMTP capable content 227filtering software, Bennett Todd's SMTP proxy implements a nice 228Perl-based framework. See: 229https://web.archive.org/web/20151022025756/http://bent.latency.net/smtpprox/ 230or https://github.com/jnorell/smtpprox/ </p> 231 232<blockquote> 233 234<table border="0"> 235 236<tr> 237 238 <td bgcolor="#f0f0ff" align="center" valign="middle" 239 width="10%"> Internet </td> 240 241 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 242 243 <td bgcolor="#f0f0ff" align="center" valign="middle" 244 width="10%"> <a href="smtpd.8.html">Postfix SMTP server on 245 port 25</a> </td> 246 247 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 248 249 <td bgcolor="#f0f0ff" align="center" valign="middle" 250 width="10%"> filter on localhost port 10025 </td> 251 252 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 253 254 <td bgcolor="#f0f0ff" align="center" valign="middle" 255 width="10%"> <a href="smtpd.8.html">Postfix SMTP server on 256 localhost port 10026</a> </td> 257 258 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 259 260 <td bgcolor="#f0f0ff" align="center" valign="middle" 261 width="10%"> <a href="cleanup.8.html">Postfix cleanup 262 server</a> </td> 263 264 <td align="center" valign="middle" width="5%"> <tt> -> </tt> </td> 265 266 <td bgcolor="#f0f0ff" align="center" valign="middle" 267 width="10%"> Postfix incoming queue </td> 268 269</tr> 270 271</table> 272 273</blockquote> 274 275<p> This is configured by editing the master.cf file: </p> 276 277<blockquote> 278<pre> 279/etc/postfix/master.cf: 280 # ============================================================= 281 # service type private unpriv chroot wakeup maxproc command 282 # (yes) (yes) (yes) (never) (100) 283 # ============================================================= 284 # 285 # Before-filter SMTP server. Receive mail from the network and 286 # pass it to the content filter on localhost port 10025. 287 # 288 smtp inet n - n - 20 smtpd 289 -o smtpd_proxy_filter=127.0.0.1:10025 290 -o smtpd_client_connection_count_limit=10 291 # Postfix 2.7 and later performance feature. 292 # -o smtpd_proxy_options=speed_adjust 293 # 294 # After-filter SMTP server. Receive mail from the content filter 295 # on localhost port 10026. 296 # 297 127.0.0.1:10026 inet n - n - - smtpd 298 -o smtpd_authorized_xforward_hosts=127.0.0.0/8 299 -o smtpd_client_restrictions= 300 -o smtpd_helo_restrictions= 301 -o smtpd_sender_restrictions= 302 # Postfix 2.10 and later: specify empty smtpd_relay_restrictions. 303 -o smtpd_relay_restrictions= 304 -o smtpd_recipient_restrictions=permit_mynetworks,reject 305 -o smtpd_data_restrictions= 306 -o mynetworks=127.0.0.0/8 307 -o receive_override_options=no_unknown_recipient_checks 308</pre> 309</blockquote> 310 311<p> Note: do not specify spaces around the "=" or "," characters. </p> 312 313<p> The before-filter SMTP server entry is a modified version of the 314default Postfix SMTP server entry that is normally configured at 315the top of the master.cf file: </p> 316 317<ul> 318 319 <li> <p> The number of SMTP sessions is reduced from the default 320 100 to only 20. This prevents a burst of mail from running your 321 system into the ground with too many content filter processes. </p> 322 323 <li> <p> The "-o smtpd_client_connection_count_limit=10" prevents 324 one SMTP client from using up all 20 SMTP server processes. 325 This limit is not necessary if you receive all mail from a 326 trusted relay host. </p> 327 328 <p> Note: this setting is available in Postfix version 2.2 and 329 later. Earlier Postfix versions will ignore it. </p> 330 331 <li> <p> The "-o smtpd_proxy_filter=127.0.0.1:10025" tells the 332 before-filter SMTP server that it should give incoming mail to 333 the content filter that listens on localhost TCP port 10025. 334 335 <li> <p> The "-o smtpd_proxy_options=speed_adjust" tells the 336 before-filter SMTP server that it should receive an entire email 337 message before it connects to a content filter. This reduces 338 the number of simultaneous filter processes. </p> 339 340 <p> NOTE 1: When this option is turned on, a content filter must 341 not <i>selectively</i> reject recipients of a multi-recipient 342 message. Rejecting all recipients is OK, as is accepting all 343 recipients. </p> 344 345 <p> NOTE 2: This feature increases the minimum amount of free 346 queue space by $message_size_limit. The extra space is needed 347 to save the message to a temporary file. </p> 348 349 <li> <p> Postfix ≥ 2.3 supports both TCP and UNIX-domain filters. 350 The above filter could be specified as "inet:127.0.0.1:10025". 351 To specify a UNIX-domain filter, specify "unix:<i>pathname</i>". 352 A relative pathname is interpreted relative to the Postfix queue 353 directory. </p> 354 355</ul> 356 357<p> The after-filter SMTP server is a new master.cf entry: </p> 358 359<ul> 360 361 <li> <p> The "127.0.0.1:10026" makes the after-filter SMTP 362 server listen 363 on the localhost address only, without exposing it to the 364 network. NEVER expose the after-filter SMTP server to the 365 Internet :-) </p> 366 367 <li> <p> The "-o smtpd_authorized_xforward_hosts=127.0.0.0/8" 368 allows the after-filter SMTP server to receive remote SMTP 369 client information from the before-filter SMTP server, so that 370 the after-filter Postfix daemons log the remote SMTP client 371 information instead of logging localhost[127.0.0.1]. </p> 372 373 <li> <p> The other after-filter SMTP server settings avoid 374 duplication of work that is already done in the "before filter" 375 SMTP server. </p> 376 377</ul> 378 379<p> By default, the filter has 100 seconds to do its work. If it 380takes longer then Postfix gives up and reports an error to the 381remote SMTP client. You can increase this time limit (see the <a href="#parameters">"Configuration 382parameters"</a> section below) but doing so is pointless because you 383can't control when the remote SMTP client times out. </p> 384 385<h2><a name="parameters">Configuration parameters</a></h2> 386 387<p> Parameters that control proxying: </p> 388 389<ul> 390 391<li> <p> smtpd_proxy_filter (syntax: host:port): The host and TCP 392port of the before-queue content filter. When no host or host: 393is specified here, localhost is assumed. </p> 394 395<li> <p> smtpd_proxy_timeout (default: 100s): Timeout for connecting 396to the before-queue content filter and for sending and receiving 397commands and data. All proxy errors are logged to the maillog 398file. For privacy reasons, all the remote SMTP client sees is "451 399Error: queue file write error". It would not be right to disclose 400internal details to strangers. </p> 401 402<li> <p> smtpd_proxy_ehlo (default: $myhostname): The hostname to 403use when sending an EHLO command to the before-queue content filter. 404</p> 405 406</ul> 407 408<h2><a name="protocol">How Postfix talks to the before-queue content 409filter</a></h2> 410 411<p> The before-filter Postfix SMTP server connects to the content 412filter, delivers one message, and disconnects. While sending mail 413into the content filter, Postfix speaks ESMTP but uses no command 414pipelining. Postfix generates its own EHLO, XFORWARD (for logging 415the remote client IP address instead of localhost[127.0.0.1]), DATA 416and QUIT commands, and forwards unmodified copies of all the MAIL 417FROM and RCPT TO commands that the before-filter Postfix SMTP server 418didn't reject itself. 419Postfix sends no other SMTP commands. </p> 420 421<p> The content filter should accept the same MAIL FROM and RCPT 422TO command syntax as the before-filter Postfix SMTP server, and 423should forward the commands without modification to the after-filter 424SMTP server. If the content filter or after-filter SMTP server 425does not support all the ESMTP features that the before-filter 426Postfix SMTP server supports, then the missing features must be 427turned off in the before-filter Postfix SMTP server with the 428smtpd_discard_ehlo_keywords parameter. </p> 429 430<p> When the filter rejects content, it should send a negative SMTP 431response back to the before-filter Postfix SMTP server, and it 432should abort the connection with the after-filter Postfix SMTP 433server without completing the SMTP conversation with the after-filter 434Postfix SMTP server. </p> 435 436</body> 437 438</html> 439
