1Wish list: 2 3 Things to do before the stable release: 4 5 make pre-release-check, HTML validator check. 6 7 Disable -DSNAPSHOT and -DNONPROD in makedefs. 8 9 Scan Postfix code with github.com/googleprojectzero/weggli 10 (depends on "rust"). 11 12 Investigate clang-format compatibility as a possible migration 13 away from indent. This requires that the output is stable. 14 15 Check out https://github.com/milter-manager/milter-manager/ 16 17 Check out https://cutter.osdn.jp/ (C/C++ unit tests). 18 19 Follow https://github.com/vdukhovni/postfix/commits/rpk 20 21 Multi-recipient support in sender/recipient_bcc_maps and 22 always_bcc. 23 24 Test for dns_rr_copy() + dns_rr_free(). 25 26 27 mail_conf_xxx supprt for non-negative numbers (i.e. 28 numbers with a lower bound of zero). 29 30 Log anvil transgressions with their address range (in 31 addition to the offending IP address. We should not disclose 32 to random clients how we aggregate anvil event counters. 33 34 Should "postconf -f" pretty-print text inside {}? 35 36 Is there any code that calls attr_scan*() and that works 37 when the number of attributes received < the expected number? 38 If there is no such code, then we can simplify a few things. 39 40 Update TLS_README diagram, tlsmgr no longer manages cert 41 info. 42 43 Consider renaming local_header_rewrite_clients to 44 local_header_canonicalize_clients, as a more accurate name. 45 Optionally support "local_header_canonicalize_classes = 46 rewrite_addresses, add_missing_headers" (default setting). 47 48 And ditto for remote_header_rewrite_domain, whether it 49 should rewrite address, add missing headers, or both. 50 51 Add weight factors to randmap, for example randmap:{{result1}*99, 52 {result2}*1}. To parse out weights, see postscreen. 53 54 randmap already allows randmap:{{result}, ...}, to support 55 whitespace and comma in result values, but it should also 56 extract the value from {}. 57 58 Migrate masquerade_domains from ARGV to STRING_LIST, or 59 deprecate this feature because it breaks table-driven address 60 validation. 61 62 Enforce var_line_limit in util/attr_scan*c. This is needed if 63 we want to expose Postfix RPC protocols externally. 64 65 Can tests use LD_PRELOAD to inject fake modules such as 66 fake_dns(3), fake_msg(3), fake_myaddrinfo(3) and so on? 67 One limitation is that functions etc. in a preloaded object 68 always take precedence, even in code that is not being 69 tested. 70 71 '%l' support, similar to %D in the Dovecot LDAP driver. 72 Subject: Feature request: '%l' expansion for ldap_table, 73 Date: Tue, 5 Apr 2022. Message-ID: 74 <ef7c661c-d86a-2366-6a73-ec8d51d75012@dev.snart.me> 75 76 WARN_IF_REJECT like prefix that disables the error counter increment. 77 78 Consider migrating Postfix server sockets from directory 79 $queue_directory/public to $queue_directory/protected. The 80 directory $queue_directory/public can then be used for 81 non-Postfix listeners (one subdirectory per application). 82 83 FILTER_README needs some text on multi-instance implementations, 84 and existing multi-instance references need to be updated. 85 86 Fix code that still uses "long" for data_size and data_offset, 87 and sscanf("%ld or strtou?l()). This seems relevant for 32-bit 88 systems. This would use a new REC_TYPE_OFFS with a corresponding 89 data type of off_t, using off_cvt() for conversion from string, 90 and new code to convert off_t to string. 91 92 A smart query service for live Postfix tables that outputs JSON? 93 If the idea is to introspect on a running Postfix system, this 94 involves adding an RPC endpoint to specific Postfix services. 95 That could work for single-instance services like qmgr, verify, 96 postscreen. 97 98 JSON logging? 99 100 default_transport_maps? This would simplify configuration. 101 102 Add a pointer to 103 https://fabianlee.org/2019/10/23/docker-running-a-postfix-container-for-testing-mail-during-development/ 104 and https://github.com/docker-mailserver/docker-mailserver 105 106 Add a pointer to 107 https://github.com/tarickb/sasl-xoauth2 and/or 108 http://mmogilvi.users.sourceforge.net/software/oauthbearer.html 109 in documentation or on-line howtos. 110 111 Read the above links and see how we can improve usability on 112 the Postfix side. 113 114 Add verp=+= to the qmgr "from=" logging. This is already 115 implemented but not yet integrated. 116 117 Need canonical Dovecot example that has virtual_mailbox_domains, 118 (virtual_mailbox_maps or reject unverified_recipient), and 119 virtual_transport. 120 121 Make smtpd_relay_before_recipient_restrictions settable 122 in smtpd_checks tests. 123 124 Make the DNS resolver library pluggable, so that we can a) 125 plug in a fake resolver library for DNS-related regression 126 tests and make DNS tests hermetic (no external dependency; 127 b) add support for non-libbind resolvers. Gracefully handle 128 requests for unsupported functionality; return an error status, 129 instead of terminating. 130 131 Add a robust dnssec_probe regression test (success and fail) 132 that does not break existing regression tests. 133 134 smtp_sasl_tls_security_options = noanonymous, and make 135 smtp_sasl_security_options the default dependent on the 136 smtp_sasl_tls_security_options default (i.e. reverse the 137 dependency). Or make them independent. 138 139 Try to make the master throttle more distrusting. Currently, 140 the master throttles a service after a child process cannot be 141 created (fork() fails), or if a child process fails upon its 142 first use. The master always unthrottles the service if a process 143 handles a client successfully. This is sufficient to mitigate 144 local errors that break all attempts to use a service. It also 145 slows down stupid remote attacks as long as malicious traffic 146 dominates benign traffic. Perhaps monitor a crashing percentage? 147 If 50% of all connections to a service result in abnormal 148 termination, that would be bad even under a non-attack scenario. 149 150 More accurate address verification: do a quota check before 151 reporting that a local(8) or virtual(8) recipient is deliverable. 152 153 Eliminate duplicate mail submission permission checks from 154 sendmail, so that they happen in postdrop only. Then, pass the 155 result through the postdrop-to-sendmail protocol. This requires 156 that postdrop reads all inputs before responding (the 157 local_login_sender_maps check depends on the envelope 158 sender). Then sendmail can save input to dead.letter (no setgid 159 privilege, but it would still have to use safe_open() to avoid 160 clobbering files). 161 162 Consider removing compat_level_from_numbers() and aliases, 163 because they are no longer used anywhere. 164 165 Allow '}' at the beginning of a line. This would make multi-line 166 configuration settings easier to enter. This may be true 167 for main.cf, master.cf and similar files (such as database 168 configuration files, but not necessarily elsewhere). So it 169 may have to be a readlline flag. 170 171 Understand what happens with DNSSEC related status fields 172 in posttls-finger when resolv.conf points to a host that 173 runs no DNS server. 174 175 Hardening the half-dane behavior: some sites may rely on 176 current behavior which allows original MX domain name for 177 certificate matches. Requires a new (compatibility) parameter 178 setting? 179 180 Code deduplication: migrate multi_server applications to 181 event_server, because the multi_server and event_server 182 skeletons are much more similar than other skeletons. In 183 addition to the default event_server accept() handler, also 184 register a read event callback for handling post_accept 185 events. But the currrent multi_server API fits typical usage 186 better. 187 188 When a secondary instance has no multi_instance_name set, 189 postmulti -i won't be able to find it. 190 191 nbbio: exercise the sanity checks with fake msg(3) functions. 192 193 optreset (bsd-ism) how badly do we need it? 194 195 transport policy protocol (clone of check_policy). 196 197 See also postscreen event-driven client for policy delegation 198 below. 199 200 smtp_line_length_limit can insert a line break in the middle 201 of a multi-byte character (which is not necessarily UTF-8, 202 so we can't simply look at the 8th bit). Also, note that a 203 multi-byte character may span queue file record boundaries, 204 for example if line_length_limit == smtp_line_length_limit. 205 The only way to fix this is to make the smtp_text_out() 206 routine aware of every possible multi-byte encoding. 207 208 Replace ad-hoc code for pipe(8) flags handling, with 209 infrastructure that was built for smtp(8). 210 211 Move map descriptions from postconf(1) to DATABASE_README 212 and point there. The text in DATABASE_README is less complete 213 than that in postconf(1). 214 215 make tls_pre_jail_init() safe by design for use in programs 216 that implement both clients and servers. 217 218 In smtpd(8) and postscreen(8), set the ehlo_discard_mask 219 to ~0 so that STARTTLS, BDAT, DSN, etc. work only for clients 220 that send EHLO. 221 222 Wordsmithing: "replace by X" -> "replace with X" unless X 223 is "responsible" for making the substitution. 224 225 In postscreen, don't fork after 'postfix reload' when 226 psc_check_queue_length (and psc_post_queue_length?) is zero. 227 228 After I/O error, store errno in VSTREAM object before errno 229 may be overwritten. 230 231 Add some tips for logging from container: 232 https://www.projectatomic.io/blog/2016/10/playing-with-docker-logging/; 233 syslog_name = $myhostname/postfix; mkdir queue and data 234 dir; postfix check to create queue subdirectories. 235 236 Add postwhite as a postscreen-related project. 237 https://github.com/stevejenkins/postwhite/blob/master/README.md 238 239 XFORWARD attributes in policy protocol? 240 241 Document postsrsd and postforward for srs-ifying. Would 242 more fine-grained smtp_generic_maps support help? 243 244 Decide whether to deprecate database configuration pathnames 245 that start with ".", for example, ldap:./file/name. These forms 246 are documented for ldap:, memcache:, mysql:, pgsql:, and sqlite: 247 maps. Postfix daemon processes will look up files relative to the 248 queue directory, but with postmap command-line processes it would 249 be more natural to interpret relative pathnames relative to the 250 current directory of the calling process (it would be a surprise 251 if "postmap hash:./foo" would access "/var/spool/postfix/foo", 252 or if "postmap hash:foo" and or "postmap hash:./foo" would access 253 different files). 254 255 Convert postalias(1) to store external-form keys, and convert 256 aliases(5) to perform external-first lookup with fallback to 257 internal form, to make it consistent with the rest of Postfix. 258 In several years we may remove the internal-form fallbacks 259 with a compatibility_level safety net. 260 261 In the bounce daemon, set util_utf8_enable if returning an 262 SMTPUTF8 message. This is wrong; if SMTPUTF8 is disabled, 263 then Postfix must not turn it on. 264 265 Add a header_body_checks extension callback in smtp_proto.c 266 that implements the PASS action. 267 268 Propagate SMTPD_PEER_CODE_XXX from smtpd(8) to cleanup(8), 269 so that {client_resolve} and {_} produce consistent results. 270 271 NO_IP_CYRUS_SASL_AUTH should be a main.cf parameter. 272 273 Modeline support in config files to enable/disable trailing 274 #comment, and to give hints about how to handle an LHS or 275 RHS. This will not preserve trailing comments in lines that 276 are modified with "postconf -e" and the like. 277 278 Maintainability: replace lengthy libmilter-API argument lists 279 with named parameters, as with the libtls API. 280 281 Fix buflen integer overflow detection in dict*sql.c. 282 283 Fix "make test" bitrot. 284 285 Move DNS-based tests from porcupine.org to postfix.org, or use 286 a mock DNS library (a library that presents the same API as the 287 real library, but that produces canned responses). 288 289 Document dns_ncache_ttl_fix_enable use case in POSTSCREEN_README 290 and RELEASE_NOTES. 291 292 Remove this file from the stable release. 293 294 Things to do after the stable release: 295 296 Specify WARN_UNUSED_RESULT for all library functions that 297 pass, deliver, bounce or defer a delivery request. 298 299 Invent some kind of type-checking wrappers for htable(3), 300 ctable(3) and other modules that take and return a void* 301 pointer. We already did that for variadic functions. 302 303 TLS certificate provenance: indicate whether a subject 304 name/issuer are verified or not (for example, change the 305 attribute name to unverified_ccert_subject etc.). This is 306 relevant only for fingerprint-based authentication including 307 DANE, and affects logging, SMTPD policy, and Milters. 308 309 Generalize the daemon '-S' stand-alone mode, so that it can 310 be used with custom configuration settings for request/reply 311 regression testing. This would use the existing "-o name=value" 312 support to override parameters. For example, queue_directory 313 would point to a directory with sockets for fake versions of 314 Postfix-internal services. 315 316 Update the list of Sendmail macros that Postfix can send 317 to Milters (auth_ssf and TLS-related). 318 319 Update smtpd command count when rejecting or skipping input 320 before command-table lookup. But then we need to count 321 commands that are rejected (malformed UTF-8, tokenizer 322 error, forbidden command), or skipped (noop). 323 324 What is the best place to detect spaces in pathnames during 325 installation/upgrade/packaging? postfix-install for early 326 warning, and post-install as a safety net? 327 328 When the service basename differs from the program file 329 basename, either prepend the service name to the syslogname (as 330 if syslog_name=postfix/service/program), or prepend the service 331 name to the process name (perhaps too confusing). The service 332 indication is desirable for mail delivery transports (smtp 333 versus relay) as it identifies what scheduler parameters are 334 in effect, but it is also desirable for mail receiving services 335 (smtp versus submission verus smtps as configured in the stock 336 master.cf file). This requires exceptions for some program names 337 (exclude smtpd to avoid logging postfix/smtp/smtpd which could 338 result in more confusion, and maybe other program names). 339 340 UTF8 DNS[BW]L domain name. 341 342 Consolidate maps flags in mail_params.h instead of having 343 multiple copies scattered across programs. 344 345 Try to allow UTF-8 myhostname/mydomain, at least in bounce 346 template expansion. 347 348 In the SMTP server, do not issue an enhanced status code when 349 rejecting a connection before the HELO handshake is completed. 350 351 Maybe don't whitelist a client that has maxed out its 352 per-MTA connection count limit. 353 354 Log command=good/bad statistics in postscreen? 355 356 smtpd_checks tests either must use a DNS dummy resolver 357 (override the res_search API) or all names must be under 358 test.postfix.org (but that does not work for address->name 359 lookups, and cannot simulate some errors). 360 361 Reporting the original Message-ID in a bounce message 362 In-Reply-To: or References: header. In the cleanup daemon, 363 grab a copy of the Message-ID and export it along with other 364 header-extracted information at the top of the "extracted" 365 queue file segment. In the queue manager, extract this 366 along with other header-extracted information, and forward 367 the Message-ID in the bounce server notification request. 368 369 Clobber ORCPT when sender is owner-mumble? 370 371 Add milter_mumble_macros to the list of per-macro features. 372 373 The pickup daemon logs warnings only when the cleanup daemon 374 dit not provide a "reason" attribute. Is this logic right? 375 376 up-convert myhostname to UTF-8 in MIME boundary strings? 377 378 Eliminate code duplication between pcf_print_master_field() 379 and pcf_print_master_entry(). 380 381 Error reporting: see if pcf_check_master_entry() and children 382 can return error descriptions instead of terminating with 383 a fatal error. 384 385 Add a switch to consider postscreen deep protocol tests as 386 "completed" when receiving "RSET" after "RCPT TO" and the 387 session has passed all tests up to that point. RSET becomes 388 like QUIT except perhaps that it does not hang up. 389 390 apipe: map, splits results into address lists and performs 391 lookups for the invidual addresses, converting back and 392 forth between external and internal forms. 393 394 Clarify that receive_override_options have no effect with 395 smtpd_proxy_filter. 396 397 Document the relative order of header_checks, address 398 rewriting, milters. 399 400 NOT: Table-driven case folding and case-insensitive string 401 comparison specifically for UTF-8. Use libicu functions 402 instead. 403 404 When downgrading message/global to 7bit, is quoted-printable 405 the appropriate encoding? Should it be base64? 406 407 Should we encode headers with RFC 2047, when that is the 408 only reason that Postfix cannot deliver to a non-UTF8SMTP 409 server? Probably not in the general case. What about 410 Postfix as a gateway server that converts UTF8SMTP 411 for delivery to non-UTF8SMTP environments? 412 413 Document and test restriction_classes example for 414 smtpd_policy_service_default_action. 415 416 Don't accept AUTH or other features that are not announced 417 in the EHLO response. 418 419 Suggested at Mailserver conference: Postscreen RDNS-based 420 reputation (but this makes postscreen performance highly 421 unpredicable because it introduces a dependency on random 422 DNS servers). 423 424 Suggested at Mailserver conference: a way to select a 425 specific field in a table, presumably as the result value. 426 This may be done with a filtermap{i,j,...}: table that propagates 427 only the specified field(s). 428 429 Discourage the use of "after 220" tests in POSTSCREEN_README 430 and the documentation of individual parameter settings. 431 432 To un-break "make tests" under src/smtpd, make tests 433 independent from the DNS and native routines for host 434 name/address lookup. 435 436 Make been_here flag BH_FLAG_FOLD configurable for masochists. 437 438 Replace some redundant TLS_README sections with pointers 439 to FORWARD_SECRECY_README. 440 441 Move html/index.html source to proto/. 442 443 How hard is it to follow canonical or virtual mapping 444 for the purpose of address validation? We must never 445 reject a valid address. 446 447 Preserve case in smtpd_resolve_addr() and add a structure 448 member for the case-folded address. IIRC some Milter macro 449 needs to show the unfolded address. 450 451 Per SASL account rate limits. This requires new infrastructure 452 that maintains stats by SASL account instead of client IP 453 address. 454 455 Watchdog timer in postmap/postalias. 456 457 Begin code revision, after DANE support stabilizes. This 458 should be one pass that changes only names and no code. 459 460 recipient_delimiters = $recipient_delimiter for BC 461 462 All source code must specify its original author and 463 license statement. Some code modules specify Lutz Jaenicke 464 as the original author and fall under his liberal license. 465 Code that is added to such a module has the same license 466 (or at least something that is not more restrictive). Code 467 modules without input from Lutz Jaenicke must state its 468 original author and license (preferably no more restrictive 469 than Postfix's own license). Currently, too many files list 470 Wietse as the original author, and Lutz Jaenicke's license, 471 which is wrong. 472 473 We have smtp_host_lookup, smtp_dns_resolver_options, and 474 now smtp_dns_support_level. Of these, smtp_dns_resolver_options 475 is orthogonal but the rest has overlap. 476 477 There needs to be support for automatic migration from the 478 deprecated disable_dns_lookups feature to the preferred 479 smtp_dns_support_level feature. This support needs to exist 480 for several releases before the deprecated feature can be 481 removed. 482 483 End code revision, after DANE support stabilizes. 484 485 It would be nice if "bare username" lookup is not hard-coded 486 for domains in the local address class. 487 488 Don't forget Apple's code donation for fetching mail from 489 IMAP server. 490 491 Should postconf -o refuse to work without the -x option? 492 493 Make 30s caching (feature 20070414) configurable, such that 494 0 means no caching. 495 496 Make errno white/blacklist for getpwnam_r etc. and mailbox 497 write errors. 498 499 smtpd_muble_restrictions rule names are case-insensitive. 500 restriction_classes values are case-sensitive but should 501 be case-insensitive for consistency with smtpd_muble_restrictions. 502 503 Make "rename" the default when postmapping a DB file 504 (later: use copy+rename for postmap -i, postmap -d). 505 506 Service-name parameters aren't documented in daemon manpages. 507 508 When faking up the DSN ORCPT, don't send bare usernames 509 from local command-line submission. 510 511 lmtp_assume_final is broken. A 2XX response does not imply 512 final delivery. The Sieve language implements accept-then-bounce. 513 514 postscreen event-driven plug-in interface to send out a 515 query in parallel with the Pregreet and DNSBL tests, using 516 a simplified version of the policy delegation protocol. 517 518 Parallelized queue preprocessing: rip out the queue manager 519 code to read queue files and resolve recipients, and run 520 it in parallel processes. The queue manager then processes 521 their results as they become available. This would eliminate 522 the qmgr<->trivial-rewrite bottleneck. This can also eliminate 523 much of the scheduling disadvantage of a single queue manager 524 compared to hundreds of mail receiving or sending processes 525 (especially if there is a way to scan the queue in parallel). 526 527 Memory pools for same-type memory objects. This can be 528 used to either increase memory locality for frequently-allocated 529 objects (MRU allocation) or to make use-after-free bugs 530 more detectable (use LRU allocation and wipe the object 531 immediately after free(). Finally, same-type memory pools 532 prevent object type errors with use-after-free bugs. 533 534 "no-cache" option for selected postscreen tests? 535 536 Need a new DICT flag to indicate that a map handle supports 537 locking. If it doesn't (as with memcache or proxymap 538 handles), then postscreen etc. don't need to close a cache 539 file after "postfix reload". After a fork() it is OK to 540 keep using a memcache or proxymap handle, because the parent 541 exits immediately. For this to work, the memcache client 542 needs to propagate the flag from a persistent backup map, 543 but the proxymap protocol should not propagate this to the 544 client. 545 546 Different TTL values for different DNSBL sources? 547 548 Replace master(8) SIGHUP by very simple socket protocol to 549 allow reload of a specific service. 550 551 postscreen: in the dummy SMTP engine, log the protocol state 552 at time of violation (like smtpd, set state->where initially 553 to CONNECT, then update it with the name of the last "known" 554 command, or set it to "unimplemented"). 555 556 The discussion of postscreen cache configuration is in the 557 wrong place (how whitelisting works). Move it to the section 558 about configuring postscreen. 559 560 Before proxymap can be exposed to the network (primarily 561 to share postscreen or verify caches), need to enforce 562 limits on attribute string name and value length in IPC 563 protocols. 10-20KB seems OK. We need to enforce content 564 sanity checks (for example, no control characters; Postfix 565 does not pass around multi-line data in table lookups). The 566 VSTREAM library already supports read/write deadlines. We 567 need to use attack-resistant code for numeric conversion. 568 569 move flush_init() etc. from defer service clients to the 570 bounce daemon? Postfix works best when work can be spread 571 out over many clients, instead of over a few servers. 572 573 multi_connect() function that takes a list of inet:host:port 574 and/or unix:pathname specs, with an explicit "inet" prefix 575 argument to handle applications that use host:port only. 576 This will simplify multi-host implementation for memcache 577 client, dovecot client, and other. 578 579 dict_memcache: treat "bad" key as cache miss, i.e. read/write 580 the backup database as if the cache did not exist. This 581 does not help because most Postfix maps (virtual, canonical, 582 access, transport, ...) also don't support spaces in keys. 583 584 postscreen: keep the cache open after "postfix reload" when 585 it is remote (type memcache: or proxy:). This does not work 586 because memcache can use a non-proxied file as backup). 587 588 What is the feasibility of adding an mta_name (personality) 589 attribute that is propagated via queue files and delivery 590 agent requests? It would default to myhostname. 591 592 Major performance improvement opportunity (that is until 593 everyone runs Postfix queues on SSDs). Investigate the 594 viability of a daemon that produces incoming and postdrop 595 queue files on request (in reality it would maintain a 596 limited queue of "spare" files). Central queue file allocation 597 reduces the I/O performance disadvantage that qmgr has when 598 100 smtpd processes are receiving mail, or when lots of 599 mail is submitted with the sendmail command line. When an 600 smtpd process accepts MAIL FROM, a cleanup daemon requests 601 a queue file and receives a queue ID + file handle from the 602 queue file daemon. If the queue file daemon is down, the 603 cleanup daemon creates the file itself like it does now; 604 this can be hidden in the mail_stream library module. If 605 the mail transaction is aborted, then the cleanup daemon 606 gives the queue file back to the queue file daemon's "spare" 607 file pool, saving most of the overhead of creating and 608 deleting a queue file (the file would still need to be 609 renamed at the start of the next mail transaction). If the 610 cleanup daemon is unable to give a file back, then it can 611 delete the file like it does now; this can be hidden in the 612 mail_stream library module. The whole thing can be 613 transparently added to Postfix by adding calls to a 614 queue-file-service client to the mail_queue_enter() and 615 mail_queue_remove() library routines. Other advantages: 616 1) negligible performance hit when queue file allocation 617 happens earlier, so that logging and milters have a queue 618 ID for the whole transaction not just the first valid 619 recipient; 2) by not removing every queue files we get most 620 of the performance gain of a queue based on append/truncate 621 instead of the much more expensive create/delete. 622 623 Investigate viability of Sendmail dns maps. 624 625 Make the rules for how to use close-on-exec more explicit. 626 627 Provide separate timeout control for dict_proxy client, 628 rewrite client, resolve client, cleanup client, and so on. 629 Perhaps a timeout argument to the mail_connect() routines. 630 631 Trick from amavisd: save listen socket/fifo/etc state, clear 632 their close-on-exec flags, exec the same program file to 633 re-initialize (with saved socket state on command line or 634 in environment), then restore the listen socket/fifo/etc 635 close-on-exec flags. This could be a way to mitigate the 636 impact of memory/file leaks, and to implement "postfix 637 reload" support for master(8) features that currently don't 638 support this. 639 640 Sub-second time resolution. The first benefit is to make 641 per-destination rate delays more usable. Other applications 642 will come up once the support exists. The straightforward 643 approach is to represent all time intervals in milliseconds, 644 and to update all code that makes system calls with a time 645 argument (as well as the compiled-in upper and lower time 646 parameter bounds, which are currently in seconds). 647 Unfortunately, that limits he maximum time interval to less 648 than 25 days on 32-bit systems, and is likely to break 649 compatibility (for starters, it cannot even deal with the 650 compiled-in 100d upper bound on the queue file lifetime). 651 A second option is to have a "compatibility" time base 652 switch between milliseconds and seconds; this means extra 653 changes to all code that makes system calls with a time 654 argument, and the way that the compiled-in upper and lower 655 bounds are specified. Some of this can be encapsulated in 656 macros like time_to_sec(t), time_to_msec(t) and sec_to_time(t). 657 Finally, it is relatively easy to replace the events(3) 658 interface to use "double" for the time delay arguments, but 659 it is a major pain to convert all main.cf time parameters 660 into doubles (converting only some leads to a documentation 661 nightmare). 662 663 Address verify cache: allow a negative cache "refresh" 664 result to purge a "positive" cache entry in some safe manner. 665 Currently, the negative cache "refresh" result is discarded, 666 address verify cache lookup returns OK, and each lookup 667 forces a "refresh" probe until the entry expires. 668 669 Some Sendmail configurations trigger sub-optimal behavior 670 when the postscreen_whitelist_interfaces parameter lists 671 primary MX addresses only. When postscreen's "deep protocol 672 tests" are successful on the primary MX address (i.e. they 673 result in 4XX responses to RCPT TO), some Sendmail 674 configurations keep the primary MX connection open until 675 AFTER they finish talking to the backup MX address. The 676 problem is that the backup connection runs into a WHITELIST 677 VETO condition because the whitelisting database has not 678 yet been updated with the PASS NEW result for the primary 679 MX connection. Unfortunately postscreen can't update the 680 whitelisting database before the primary MX connection is 681 closed, because a client may still make a mistake. 682 683 In the SMTP server, check if the connection is closed before 684 replying to ".", and discard the message if the reply can't 685 be sent. This reduces the time window for RFC 1047 message 686 duplication, and may even prevent the delivery of some spam. 687 http://www.exim.org/lurker/message/20070416.103159.9d5ff0ce.en.html 688 This requires splitting the SMTP server's commit operation 689 into two operations: first, a tentative commit operation 690 that performs most of the I/O and processing in milters and 691 in the cleanup server; second, a final commit operation 692 that is executed only if the remote SMTP client hasn't hung 693 up in the mean time. Unfortunately, SMTP-based before-queue 694 content filters don't support a tentative commit operation. 695 696 Find out how to reproduce Berkeley DB bogus ENOENT errors. 697 postscreen does not log this with Berkeley DB 1 (FreeBSD 698 4..8), 4.7.25 (Ubuntu 9.04) and 4.8.24 (Ubuntu 10.04). 699 700 postconf command-line option to show the compile-time 701 settings (CCARGS, AUXLIBS) in case binary packages 702 don't install the makedefs.out file. 703 704 events.c: cache the side effects of file descriptor event 705 enable/disable operations in user space, and do bulk kernel 706 updates at event_loop() time. This can eliminate costly 707 system calls with successive event disable/enable operations 708 on the same file descriptor. This can also eliminate the 709 need for tricky code that tries to avoid the expense of 710 successive disable/enable operations. Such code is likely 711 to introduce bugs. 712 713 When does it pay off to send domains in the active queue 714 to a DNS prefetch daemon? Could this generalize to a dynamic 715 transport map that piggy-backs domains with the same MX 716 host into the same mail delivery transaction? 717 718 tlsproxy(8) should receive TLS preferences from postscreen(8) 719 and smtpd(8), instead of reading them from main.cf. This 720 means that many tlsproxy_ parameters become postscreen_ 721 parameters, and that tls_server_init() parameters move to 722 to tls_server_start(). That is a significant API change. 723 It also means tlsproxy can't open all files before chroot(). 724 725 anvil rate limit for sasl_username. 726 727 Encapsulate nbbio buffer access and update by tlsproxy. 728 729 Full-duplex support for tlsproxy(8). This requires updating 730 events(3) and nbbio(3). 731 732 Register automagic destructor for object attached to VSTREAM. 733 734 Use different ipc time limits for email message transactions 735 (smtpd, pickup)->cleanup and for quick query/reply transactions 736 such as address rewriting/resolution. Beware of large time 737 limits for local or virtual alias expansion. 738 739 permit_tempfail_action (default: defer_if_reject) to be 740 used as the default value for dnswl_tempfail_action and 741 rhswl_tempfail_action. Steal liberally from the code that 742 implements unverified_recipient_tempfail_action etc. 743 744 Support filtering of messages that are generated by Postfix: 745 This would apply to postmaster notices and bounce messages 746 (DKIM), and address verification (BATV). 747 748 Consistency: in postconf.proto make <dt>..</dt> tags bold. 749 750 Would it help if there were different cleanup_service 751 parameter names for different message paths? smtpd(8) uses 752 the same cleanup_service value for receiving remote mail 753 and for submitting postmaster problem reports. Do we need 754 separate mumble_cleanup_service_name parameters for "inject", 755 "notify" and "forward" (with backwards compatible defaults)? 756 757 IF/ENDIF support for CIDR tables. 758 759 Need a regular expression table to translate address 760 verification responses into hard/soft/accept reply codes. 761 762 Is there a way to make sendmail -V work after local alias 763 expansion? Majordomo-like mailing lists would benefit from 764 this; the example in VERP_README does not work in the general 765 case. 766 767 When an alias is a member of an :include: list with owner- 768 alias, local(8) needs an option to deliver alias or alias->user 769 indirectly. What happens when an :include: list with owner- 770 alias includes another list? 771 772 Don't allow empty result values in pcre and regexp maps. 773 Postfix doesn't allow them anywhere else (check this). 774 775 Make PCRE_MAX_CAPTURE configurable. 776 777 Add some checks for tokens starting with #. A challenge 778 is to report sensible context from the guts of some low-level 779 parser, without introducing a great deal of clumsiness. 780 781 Add sendmail macros for {verify} and maybe other TLS info. 782 783 Find out if we are doing the correct thing by looking at 784 state->milter_reject_text when expanding {rcpt_addr} or 785 {rcpt_host}. 786 787 Find out why post_mail() etc. block when the qmgr fifo is 788 full (answer: trigger_timeout). How can this cause delays 789 in the queue manager? When a recipient bounces during 790 (transport, nexthop, address) resolution, it is redirected 791 to the error or retry mailer; and bounce-after-delivery is 792 asynchrounous so it can't block the queue manager, either. 793 794 How to ensure that proxy_read_maps is processed after all 795 its dependencies are initialized, or just bite the bullet 796 and rewrite the parameter initialization code. 797 798 The cleanup virtual alias expansion limit does not really 799 deliver on its promises. 1) It promises to truncate the 800 result without aborting delivery, which would be undesirable 801 anyway, but that is not what it does, so that is good. 2) 802 It keeps all the recipients from multi-recipient database 803 lookup, then terminates further recursion when the result 804 exceeds the expansion limit. This behavior achieves the 805 original goal that all things shall have a finite size (even 806 though but we don'really care how large they are) but may 807 result in surprises when recipients are listed in virtual 808 alias domains or need expansion for other reasons. In a 809 phone call with Victor, a reasonable way out is to set the 810 limit to some large number (100000) and abort delivery when 811 the result exceeds the limit. 812 813 Should the postscreen save permanent white/black list lookup 814 results to the temporary cache, and query the temporary 815 cache first? Skipping white/black list lookups will speed 816 up the handling of "good" clients without a permanent 817 whitelist entry. Of course, this means that updates to the 818 white/black lists do not immediately take effect. Workarounds: 819 1) use a shorter temporary cache TTL for clients on the 820 permanent black/white lists; 2) ignore cached white/black 821 list lookup results after "postfix reload"; 2) adjust the 822 logging, for example "WHITELISTED address (cached)" and 823 "BLACKLISTED address (cached)" to eliminate surprises. 824 Comparing the cache entry time with the white/blacklist 825 file modification time is not foolproof: for example, pcre 826 or CIDR tables are read only once. 827 828 It would be nice if the generic dict_cache(3) cache manager 829 could postpone process suicide until cache cleanup is 830 completed (but that is not possible when postscreen forks 831 into the background to finish already-accepted connections, 832 and it is not desirable when a host is being shut down). 833 834 When postscreen drops a connection, a 521 "greeting" should 835 be of the form "521 servername..." and not have an enhanced 836 status code. The "521 5.7.1" form can be used after EHLO. 837 Of course no spammer is going to complain about Postfix 838 SMTP compliance. 839 840 Find a place to document all the mail routing mechanisms 841 in one place so people can figure out how Postfix works. 842 843 The access map BCC action is marked "not stable", perhaps 844 because people would also expect BCC actions in header/body_checks. 845 How much would it take to make the queue file editing code 846 generally usable? 847 848 Move smtpd_command_filter into smtpd_chat_query() and update 849 the session transcript (see smtp_chat_reply() for an example). 850 851 SMTP connection caching without storing connections, to 852 improve TLS mail delivery performance. 853 854 Should not milter8_mail_event() unset the "hold" default 855 reply? Better, the default reply should not be used for 856 this purpose. 857 858 Don't send MASTER_STAT_TAKEN/MASTER_STAT_AVAIL when a server 859 runs with process limit of 1. But this means the master 860 never learns that the process is successful and will always 861 pause $service_throttle_time before restarting a failed service. 862 863 Don't bother maintaining a per-service lockfile when a 864 server runs with process limit of 1. The purpose of the 865 lockfile is to avoid thundering herd problems when the kernel 866 wakes up multiple processes for each new client connection. 867 868 Implement PREPEND action for milter_header_checks. Save the 869 to-be-prepended text to buffer, then emit it along with the 870 new header. 871 872 Fix the header_body_checks API, so that the name of the map 873 class (e.g. milter_header_checks) is available for logging. 874 875 Fix the mime_state and header_body_checks APIs, so that 876 they use VSTRINGs. This simplifies REPLACE actions. 877 878 Update FILTER_README for multi-instance support, and rename 879 the old document to FILTER_LEGACY_README. 880 881 Need to sign delivery status notifications, to avoid surprises 882 when eventually people start enforcing DKIM etc. signatures. 883 884 Either document or remove the internal_mail_filter_classes 885 feature (it's disabled by default). 886 887 Make the "unknown recipient" test configurable as 888 first|last|never, with "yes"=="last" for backwards 889 compatibility. The "first" setting is good for performance 890 (stress=yes) when all users are defined in local files; but 891 it may perform worse when users are in networked tables. 892 893 Cleanup: make DNSBL query format configurable beyond the 894 client's reversed IP address. 895 896 Maybe change maps_rbl_reject_code default to 521, and 897 update wording in STRESS_README. 898 899 Encapsulate time_t comparisons so that they can be made 900 system dependent (use difftime() where available). 901 902 Encapsulate time_t conversions (e.g. REC_TYPE_TIME) so that 903 they can be made system dependent. 904 905 Plan for time_t larger than long, or wait for LP64 to 906 dominate the world? 907 908 Write delivery rate delay example (which _README?) and auth 909 failure cache example (SASL_README). Then include them in 910 SOHO_README. 911 912 Look for alternatives for the use of non_smtpd_milters. 913 This involves some way to force local submissions to go 914 through a local SMTP client and server, without triggering 915 "mail loops back to myself" false alarms. The advantage is 916 that it makes smtpd_mumble_restrictions available for local 917 and remote mail; the disadvantage is that it makes local 918 submissions more dependent on networking. One possibility 919 is to use "pickup -o content_filter=smtp:127.0.0.1:10025", 920 or a dedicated SMTP client/server on UNIX-domain sockets; 921 we could also decide to always suppress "mail loop" detection 922 for loopback connections. Another option is to have the 923 pickup or cleanup server drive an SMTP client directly; 924 this would require extension of the mail_stream() interface, 925 plus a way to handle bounced/deferred recipients intelligently, 926 but it would be at odds with Postfix design where delivery 927 agents access queue files directly; exposing delivery agents 928 to raw queue files violates another Postfix design principle. 929 930 Consolidate duplicated code in *_server_accept_{pass,inet}(). 931 932 Consolidate duplicated code in {inet,unix,upass}_trigger.c. 933 934 In the SMTP client, handle 421 replies in smtp_loop() by 935 having the input function raise a flag after detecting 421 936 (kill connection caching and be sure to do the right thing 937 with RSET probes), leave the smtp_loop() per-command reply 938 handlers unchanged, and have the smtp_loop() reader loop 939 bail out with smtp_site_fail("server disconnected after 940 %s", where), but only in the case that it isn't already in 941 the final state. But first we need to clean up the handling 942 of do/don't cache, expired, bad and dead sessions. 943 944 Combine smtpd_peer.c and qmqpd_peer.c into a single function 945 that produces a client context object, and provide attribute 946 print/scan routines that pass these client context objects 947 around. With this, we no longer have to update multiple 948 pieces of code when a client attribute is added. Ditto for 949 SASL and TLS context. 950 951 Don't log "warning: XXXXX: undeliverable postmaster 952 notification discarded" for spam from outside. 953 954 Really need a cleanup driver that allows testing against 955 Milter applications instead of synthetic events. This would 956 have to provide stubs for clients that talk to Postfix 957 daemon processes. See if this approach can also be used for 958 other daemons. 959 960 smtpd(8) exempts $address_verify_sender from access controls, 961 but it doesn't know whether cleanup(8) or delivery agents 962 modify the sender. Would it be possible to "calibrate" this 963 exemption, perhaps by having delivery agents pass the probe 964 sender to the verify server, keeping in mind that the probe 965 sender may differ per delivery agent due to output rewriting. 966 967 Update attr_print/scan() so they can send/receive file 968 descriptors. This simplifies kludgy code in many daemons. 969 970 Would there be a problem adding $smtpd_mumble_restrictions 971 and $smtpd_sender_login_maps to the default proxy_read_maps 972 settings? 973 974 Remove defer(8) and trace(8) references and man pages. These 975 are services not program names. On the other hand we have 976 man pages for lmtp(8) and smtp(8), but not for relay(8). 977 Likewise, retry(8) does not have a man page. 978 979 Bind all deliveries to the same local delivery process, 980 making Postfix perform as poorly as monolithic mailers, but 981 giving a possibility to eliminate duplicate deliveries. 982 983 Maybe declare loop when resolve_local(mxhost) is true? 984 985 Update message content length when adding/removing headers. 986 987 Need scache size limit. 988 989 REDIRECT should override original recipient info, and 990 probably override DSN as well. 991 992 Update FILTER_README with mailing list suggestions to tag 993 with a badness indicator and then filter down-stream. 994 995 Make null local-part handling configurable: either expand 996 into mailer-daemon (current behavior) or disallow (strict 997 behavior, currently implemented only in the SMTP server). 998 999 Add M flag (enable multi-recipient delivery) to pipe daemon. 1000 1001 The usage of TLScontext->cache_type is unclear. It specifies 1002 a TLS session cache type (smtpd, smtp, or lmtp), but it is 1003 sometimes used as an indicator that TLS session caching is 1004 unavailable. In reality, that decision is made by not 1005 registering call-back functions for cache maintenance. 1006 1007 Postfix TLS library code should copy any strings that it 1008 receives from the application, instead of passing them 1009 around as pointers. TLScontext->cache_type is a case in 1010 point. 1011 1012 Are transport:nexthop null fields the same as in the case 1013 of default_transport etc. parameters? 1014 1015 Don't lose bits when converting st_dev into maildir file 1016 name. It's 64 bits on Linux. Found with the BEAM source 1017 code analyzer. Is this really a problem, or are they just 1018 using 64 bits for upwards compatibility with LP64 systems? 1019 1020 Do or don't introduce unknown_reverse_client_reject_code. 1021 1022 Check that "UINT32 == unsigned int" choice is ok (i.e. LP64 1023 UNIX). 1024 1025 Tempfail when a Milter application tries to negotiate content 1026 access, while it is configured in an SMTP server that runs 1027 before the smtpd_proxy filter. 1028 1029 Log DSN original recipient when rejecting mail. 1030 1031 Keep whitespace between label and ":"? 1032 1033 Make the map case folding/locking options configurable, if 1034 not at run-time then at least at compile time so we get 1035 consistent behavior across applications. 1036 1037 Investigate what it would take to eliminate oqmgr, and to 1038 make the old behavior configurable in a unified queue 1039 manager. This would shave another 2.7 KLOC from the source 1040 footprint. 1041 1042 Document the case folding strategy for match_list like 1043 features. 1044 1045 Eliminate the (incoming,deferred)->active rename operation. 1046 This requires an in-memory hash of queue file names to avoid 1047 duplicate open() operations. 1048 1049 Softbounce fallback-to-ISP for SOHO users. This heuristic 1050 assumes that when direct-to-MX delivery fails with 5XX, 1051 delivery via the ISP may still succeed. This could be 1052 implemented by enabling soft bounces for destinations other 1053 than the smtp_fallback_relay. So the only benefit of this 1054 over the existing soft_bounce feature is that it has no 1055 effect on smtp_fallback_relay deliveries. 1056 1057 Centralize main.cf parameter input so that defaults work 1058 consistently. What about parameter names that are prefixed 1059 with mail delivery transport names? 1060 1061 Fix default time unit handling so that we can have a default 1062 bounce lifetime of $maximal_queue_lifetime, without causing 1063 panics when a non-default maximal_queue_lifetime setting 1064 includes no time unit. 1065 1066 After the 20051222 ISASCII paranoia, lowercase() lowercases 1067 ASCII text only. 1068 1069 Privacy: remove local command/pathname details from remote 1070 delivery status reports, and log them via local msg_warn(). 1071 1072 Is it safe to cache a connection after it has been used for 1073 more than some number of address verification probes? 1074 1075 Try to recognize that Resent- headers appear in blocks, 1076 newest block first. But don't break on incorrect header 1077 block organization. 1078 1079 Hard limits on cache sizes (anvil, specifically). 1080 1081 Laptop friendliness: make the qmgr remember when the next 1082 deferred queue scan needs to be done, and have the pickup 1083 server stat() the maildrop directory before searching it. 1084 1085 Low: replace_sender/replace_recipient actions in access 1086 maps, so they can be used in policy servers? 1087 1088 Low: configurable order of local(8) delivery methods. 1089 1090 Med: smtp_connect_timeout_budget (default: 3x smtp_connect_timeout) 1091 to limit the total time spent trying to connect. 1092 1093 Med: transform IPv4-in-IPv6 address literals to IPv4 form 1094 when comparing against local IP addresses? 1095 1096 Med: transform IPv4-in-IPv6 address literals to IPv4 form 1097 when eliminating MX mailer loops? 1098 1099 Med: Postfix requires [] around IPv6 address information 1100 in match lists such as mynetworks, debug_peer_list etc., 1101 but the [] must not be specified in access(5) maps. Other 1102 places don't care. For now, this gotcha is documented in 1103 IPV6_README and in postconf(5) with each feature that may 1104 use IPv6 address information. The general recommendation 1105 is not to use [] unless absolutely necessary. 1106 1107 Med: the partial address matching of IPv6 addresses in 1108 access(5) maps is a bit lame: it repeatedly truncates the 1109 last ":octetpair" from the printable address representation 1110 until a match is found or until truncation is no longer 1111 possible. Since one or more ":" are usually omitted from 1112 the printable IPv6 address representation, this does not 1113 really try all the possibilities that one might expect to 1114 be tried. For now, this gotcha is documented in access(5). 1115 1116 Low: reject HELO with any domain name or IP address that 1117 this MTA is the final destination for. 1118 1119 Low: should the Delivered-To: test in local(8) be configurable? 1120 1121 Low: make mail_addr_find() lookup configurable. 1122 1123 Low: update events.c so that 1-second timer requests do not 1124 suffer from rounding errors. This is needed for 1-second 1125 SMTP session caching time limits. A 1-second interval would 1126 become arbitrarily short when an event is scheduled just 1127 before the current second rolls over. 1128 1129 Low: configurable internal/system locking method. 1130 1131 Low: add INSTALL section for pre-existing Postfix systems. 1132 1133 Low: add INSTALL section for pre-existing RPM Postfixes. 1134 1135 Low: disallow smtpd_recipient_limit < 100 (the RFC minimum). 1136 1137 Low: noise filter: allow smtp(8) to retry immediately if 1138 all MXes return a quick ECONNRESET or 4xx reply during the 1139 initial handshake. Retry once? How many times? 1140 1141 Low: make post-install a "postfix-only script" so it can 1142 take data from the environment instead of main.cf. 1143 1144 Low: randomize deferred mail backoff. 1145 1146 Med: separate ulimit for delivery to command? 1147 1148 Med: postsuper -r should do something with recipients in 1149 bounce logfiles, to make sure the sender will be notified. 1150 To be perfectly safe, no process other than the queue manager 1151 should move a queue file away from the active queue. 1152 1153 This could involve tagging a queue file, and use up another 1154 permission bit (postsuper tags a "hot" file, qmgr requeues it). 1155 1156 Low: postsuper re-run after renaming files, but only a 1157 limited number of times. 1158 1159 Low: smtp-source may block when sending large test messages. 1160 1161 Med: find a way to log the sender address when MAIL FROM 1162 is rejected due to lack of disk space. 1163 1164 Low: revise other local delivery agent duplicate filters. 1165 1166 Low: all table lookups should consistently use internalized 1167 (unquoted) or externalized (quoted) forms as lookup keys. 1168 smtpd, qmgr, local, etc. use unquoted address forms as keys. 1169 cleanup uses quoted forms. 1170 1171 Low: have a configurable list of errno values for mailbox 1172 or maildir delivery that result in deferral rather than 1173 bouncing mail. What about "killed by signal" exits? 1174 1175 Low: after reorganizing configuration parameters, add flags 1176 to all parameters whose value can be read from file. 1177 1178 Medium: need in-process caching for map lookups. LDAP servers 1179 seem to need this in particular. Need a way to expire cached 1180 results that are too old. 1181 1182 Low: generic showq protocol, to allow for more intelligent 1183 processing than just mailq. Maybe marry this with postsuper. 1184 1185 Low: default domain for appending to unqualified recipients, 1186 so that unqualified names can be delivered locally. 1187 1188 Low: The $process_id_directory setting is not used anywhere 1189 in Postfix. Problem reported by Michael Smith, texas.net. 1190 This should be documented, or better, the code should warn 1191 about attempts to set read-only parameters. 1192 1193 Low: while converting 8bit text to quoted-printable, perhaps 1194 use =46rom to avoid having to produce >From when delivering 1195 to mailbox. 1196 1197 virtual_mailbox_path expression like forward_path, so that 1198 people can specify prefix and suffix. 1199