Upload
erica
View
46
Download
0
Embed Size (px)
DESCRIPTION
Postfix retrospective. Wietse Venema IBM T. J. Watson Research Center Hawthorne, NY, USA. Postfix expectations before the first release. - PowerPoint PPT Presentation
Citation preview
IBM Research
© 2007 IBM Corporation
Postfix retrospective
Wietse VenemaIBM T. J. Watson Research CenterHawthorne, NY, USA
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Postfix expectations before the first release
[Postfix]: No experience yet, but I’d guess something like a wisened old man sitting on the porch outside the postoffice. Looks at everyone who passes by with deep suspicion, but turns out to be friendly and helpful once he realises you’re not there to rob the place.
Article in alt.sysadmin.recovery, 1997
See http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html for contemporary comments on other mail systems.
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Overview
Good PR can have unanticipated impact.
Why write yet another UNIX mail system?
Postfix architecture and inspiration.
Adding antispam/virus support.
Mission accomplished.
Market share.
Lessons learned.
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Good PR can have unanticipated impact
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
New York Times, December 1998
SHARING SOFTWARE, IBM TO RELEASE MAIL PROGRAM BLUEPRINT
By JOHN MARKOFF
- - -
The program, Secure Mailer, serves as an electronic post office for server computers connected to the Internet. It was developed by Wietse Venema, an IBM researcher and computer security specialist.
- - -
Currently about 70 percent of all e-mail worldwide is handled by Sendmail, a program that has been developed over more. . .
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Postfix (Secure Mailer) project
Business model: if it’s good for the Internet, then it’s good for e-business, and therefore it’s good for IBM.
One year after the first public release, several news articles began to mention Postfix as the project that triggered IBM’s adoption of open source.
Reportedly, this started when IBM’s top management saw the NY Times article.
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
How Postfix (Secure Mailer) helped IBM to embrace Open Source + Linux
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Building up momentum
June 1998 IBM joins the open source Apache project.
Sept 1998 JIKES Java compiler open source release.
Sept 1998 PKIX public key infrastructure software open source release under the name “Jonah”.
Dec 1998 Secure Mailer open source release under the name “Postfix”. IBM’s CEO starts asking questions.
1999 IBM develops an open source license (now CPL), and announces support for Linux on all platforms.
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Why (not) write yet another UNIX mail system
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
New code, new opportunities for bugs(creating bugs faster than they can be found and eliminated)
Source code line counts for contemporary software:
Windows/XP: 40 million; Vista 50+ million
Debian 2.2: 56 million; 3.1: 200+ million
Wietse’s pre-Postfix average: 1 bug / 1000 lines1.
Postfix initial release: 30k lines of opportunity1,2.
1Not included: comment lines, or bugs found in development.
2Today: 97k lines of code (194k with comments + documentation).
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
CERT/CC UNIX mail advisories(it’s not just about Sendmail)
Bulletin Software Impact CA-1988-01 Sendmail 5.58 run any command CA-1990-01 SUN Sendmail unknown CA-1991-01 SUN /bin/mail root shell CA-1991-13 Ultrix /bin/mail root shell CA-1993-15 SUN Sendmail write any file CA-1993-16 Sendmail 8.6.3 run any command CA-1994-12 Sendmail 8.6.7 root shell, r/w any file CA-1995-02 /bin/mail write any file
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
CERT/CC UNIX mail advisories
Bulletin Software Impact CA-1995-05 Sendmail 8.6.9 any command, any file CA-1995-08 Sendmail V5 any command, any file CA-1995-11 SUN Sendmail root shell CA-1996-04 Sendmail 8.7.3 root shell CA-1996-20 Sendmail 8.7.5 root shell, default uid CA-1996-24 Sendmail 8.8.2 root shell CA-1996-25 Sendmail 8.8.3 group id CA-1997-05 Sendmail 8.8.4 root shell
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Postfix primary goals(It’s not just about security)
Compatibility: make transition easy.
Wide deployment by giving it away.
Performance: faster than the competition.
Security: no root shells for random strangers.
Flexibility: C is not an acceptable scripting language.
Reliability: behave rationally under stress.
Easy to configure: simple things should be easy.
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Postfix architecture and inspiration
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Traditional BSD UNIX mail delivery architecture(impersonation requires privileges; monolithic model hinders damage control)
mailbox file
Sendmail*
/bin/mail*
to networkfrom network
local submission
local delivery
* uses root privileges
to |command**
to /file/name**
** in per-user .forward files and in per-system aliases database
owned by recipient
executed as recipient
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Postfix client-server service-oriented architecture(omitted: non-daemon programs for submission and queue management)
smtpd
localpickup
smtpdinternetsmtp
server
otherdaemons
smtpdsmtpd
localdelivery
smtpdsmtpdsmtpclient
mail storeinternetetc
mailbox|command/file/name
mailqueue
privileged
smtpdsmtpdto externaltransports
uucpfaxpager
privileged
unprivileged
unprivileged
unprivileged
unprivileged
smtp/lmtpclient
(local submission)
= root privilege= postfix privilege
input interfaces core output interfaces
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Major influences on Postfix architecture
TIS Firewall smap/smapd: least privilege, chroot jail, “air gap” between receiving and delivering processes.
qmail: parallel deliveries; the maildir format (the MH mail handling system introduced a “one file per message” mailbox store 20 years before qmail).
Apache: reuse processes multiple times.
Sendmail: user interface; lookup table interface.
Traditional routers: multiple interfaces/encapsulations, central core, but alas no queue-skipping fast path :-(
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Adding anti-spam/virus support, part 1: Use standard protocols whenever you can.
“Junk mail is war. RFCs do not apply.”
Wietse on Postfix mailing list, 2001
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
1999 - Melissa ravages the Internet
You can run from Windows but you can’t hide: Postfix becomes deeply involved with malware distribution.
Short term: block “known to be bad” strings in message header text (body strings come later).
Long-term: delegate deep inspection to third-party software.
Emergence of specialized protocols: CVP, Milter, etc.
We already use SMTP for email distribution. Why can’t we also use SMTP to plug in anti-{spam,virus}?
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Postfix content inspection via SMTP (post queue)
Red = dirty, green = clean.
But it can’t be that simple, right?
Using two MTAs must be wasteful!
MTA 1 Filter MTA 2in outsmtp smtp
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Postfix content inspection via SMTP (post queue)
Folding the two MTAs on top each other saves some resources, but also increases complexity.
networksmtp
server
mail queue
smtp client
smtp server
smtp client
content filter
local delivery
local pickup
mailbox command file
network
local submit
MTA 1 = MTA 2
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Postfix post-queue anti-spam/virus support
The advantages of post-queue SMTP-based anti-spam/virus filters outweigh the disadvantages:
– Compatibility: many products are SMTP enabled. SMTP is well understood, as are the workarounds for common implementation errors.
– Performance: decoupling the remote network latencies from local filter concurrencies allows for better resource management than possible with e.g. Milters.
Workarounds for loss of original SMTP client context:
– Xforward, etc.
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Adding anti-spam/virus support part 2: Embrace de-facto standards.
“It's not the spammers who destroy [email], it's those who insist on broken anti-spam measures.”
Wietse on Postfix mailing list, 2003
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
2005 - Proliferation of authentication technologies
SenderID, Domainkeys, DKIM, SPF, BATV, SRS, and the end is not in sight.
Problem: using SMTP-based filters just to “stamp” or “verify” can be clumsy (e.g., missing original SMTP client context). Tighter coupling to MTA is desirable.
Building into the MTA is not practical; besides, many (Linux) distributions are two years behind on Postfix.
Solution: adopt Sendmail Milter protocol and open up access to a large collection of available applications.
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Retrofitting Milter support into a distributed MTA
Red = dirty, green = clean.
The effort was heroic, but the reward was sweet.
network
local pickup
queue inject
milter application(s)
smtp server
local submit
mail queue
Postfix (subset)
smtp events
header body...
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Postfix author receives Sendmail innovation award
MOUNTAIN VIEW, Calif. October 25th, 2006 Today at its 25
Years of Internet Mail celebration event, taking place at the
Computer History Museum in Mountain View, California, Sendmail,
Inc., the leading global provider of trusted messaging, announced
the recipients of its inaugural Innovation Awards.
. . .
Wietse Venema, author, for his contribution of extending Milter
functionality to the Postfix MTA.
http://www.sendmail.com/pdfs/pressreleases/Sendmail%20Innovation%20Awards_10%2025%2006_FINAL.pdf
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Mission accomplished
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Catching up on Sendmail
Initial trigger: the Postfix 2.2 source tar/zip file was larger than the Sendmail 8.13 source tar/zip file.
Analyze eight years of Sendmail, Postfix, and qmail source code:
– Strip comments (shrinking Postfix by 45% :-).
– Format into the “Kernighan and Ritchie C” coding style (expanding qmail by 25% :-).
– Delete repeating (empty) lines.
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
MTA Source lines versus time
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Benefits of Postfix partitioned security architecture
Normally, adding code to an already complex system makes it even more complex.
– New code has unexpected interactions with already existing code, thus reducing over-all system integrity.
The Postfix architecture encourages separation of functions into different, untrusting, processes.
– Each new major Postfix feature is implemented as a separate server with its own simple protocol.
– This separation minimizes interactions with already existing code, thus preserving system integrity.
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Mission accomplished
Postfix 2.3 (now 2.4) is complete enough that I am no longer embarassed to recommend it to other people.
– Built-in: TLS, SASL, MIME, IPv6, LDAP, SQL, DSN.
Further extension via plug-in interfaces:
– DomainKeys, DKIM, SenderID, SPF.
– Non-Cyrus SASL authentication, content inspection.
– Sendmail Milter applications, SMTP server access policy.
Todo: clean up internals, hard-coded behavior, etc.
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Market share (lies, d*mned lies, and ...)
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Interesting result, but what does it mean?Query = sendmail, postfix, exim, qmail
Rel
ativ
e se
arch
vol
ume
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Introducing Google trends
Website: trends.google.com.
Search for RELATIVE popularity of search terms in Google queries (recursive Google?).
Result is a time distribution, with each popularity displayed in its own color.
Peaks are correlated with on-line news articles.
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Pollution by common words and name collisionsQuery = postfix, prefix, infix
Rel
ativ
e se
arch
vol
ume
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Tweaking the query to avoid biasQuery = sendmail server, postfix server
Rel
ativ
e se
arch
vol
ume
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Google trends caveats
As always, the answer you get is only as good as the question you ask. Beware of name collisions, common words, and other forms of pollution.
Regardless, one thing is clear: only a minority of Google search users are interested in mail server technology, and their proportion is steadily declining.
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Fingerprinting 400,000 company domains remotely
After: Ken Simpson and Stas Bekman, O’Reilly SysAdmin, January 2007.
http://www.oreillynet.com/pub/a/sysadmin/2007/01/05/fingerprinting-mail-servers.html
unknown: 15% other: 20%
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Postfix lessons learned
Good PR does make a difference. It’s easy to under-estimate how swiftly a large company can move.
Don’t re-invent mechanisms that already work (e.g., SMTP, Milter, maildir, lookup tables). Invent sparingly.
Build the basic stable protols into the MTA: SMTP, LMTP, TLS, SASL, IPv6, DSN, MIME, LDAP, SQL.
Use plug-ins for future proofing: Anti-Spam, Anti-Virus, DKIM, SenderID, SPF, greylist, etc.
Know when to stop, at least for a while.
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Postfix Pointers
The Postfix website at http://www.postfix.org/
Books by other people:
– Ralf Hildebrandt, Patrick Koetter, The Book of Postfix (2005).
– Kyle Dent, Postfix The Definitive Guide (2003).
– Richard Blum, Postfix (2001).
– Original books and translations in German, Japanese, Chinese, Czech, and other languages.
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Backup
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Where did all that code go?(from Postfix alpha to Postfix 2.3)
4x Growth in size, 8400 lines/year, mostly same author.
Small increase:
– 1.3x Average program size (800 to 1100 lines).
Medium increase:
– 2.5x Program count (from 15 to 36).
Large increase:
– 4x Library code (from 13000 to 52000 lines).
No increase: number of privileged programs.
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Postfix content inspection via SMTP (pre-queue)
SMTP “pass-through” hack built into SMTP server.
No decoupling of remote network latencies from local filter concurrencies.
Less scalable, due to poorer resource management.
But the user wanted pre-queue spam/virus filtering.
smtp server
mail queue
content filter
smtp server
Postfix (subset)
IBM Research
© 2007 IBM Corporation Postfix retrospective - Wietse Venema - CEAS 2007
Postfix RFC lines versus time