[TriLUG] Designing a new SMTP [Was: Re: [TriLUG] Another seal broken... thinking of installing a C/R anti-spam system]

[Brad Jorsch]

If we're designing a new mail protocol... Here are a few thoughts on how
I think such a thing should work.

We should probably start by making the transport 8-bit clean. And
probably use UTF-8 for header field values, so we can get rid of all the
different encoding methods and character set crap (see RFC 2047, for
example). OTOH, then we have to worry about homographs...

Email addresses should be both more and less flexible: allow most
characters in local parts, but don't allow any quoting of disallowed
characters. IDN can handle the domains. Disallow '@', a good pair of
delimiters, stuff from RFC 3454, etc...

Then we'll probably want to change MIME bodies from line-based
divisions to explicitly declaring the octet lengths of the parts. No
more worrying about detecting boundaries. We can still keep a halfway
sensible text representation of the message: make headers line-based,
encode the length as text in a header, and require the body octets to be
immediately followed by \r\n\r\n or \x2029\x2029 or some such before the
start of the next part's MIME headers.

Multiple signing of messages needs to be built into the thing, something
like what DKIM seems to do. We can get rid of the parts for handling
messages modified in transit, since we can just declare that such is not
allowed. We should think about key distribution, DNS based distribution
has a number of advantages but allowing users to issue their own keys
easily isn't one of them.

I'm not sure whether it would be better to build encryption into the
message format (along with the signing) or to use TLS all over the
place. Maybe both.

As for the transport, submission and server-to-server exchange should be
separate protocols. May as well take advantage of SRV to specify the
ports; one daemon could listen on one port to handle both protocols, or
separate daemons could be used on separate ports.

Submission should require auth. No HELO, no MAIL FROM. Adding Date,
Message-Id, and whatever headers can happen here. Besides straight
submission, there should be a mode for "sign this message and give it
back to me".

Server-to-server should build in CSV to give a good hook for reputation
services. DATA should be length-based, and possibly chunked, rather
than looking for a special sequence. It should also allow for
per-recipient responses along the lines of LMTP. There should never be
bounce messages: if a receiver can't make a final disposition decision
immediately it should specify a cookie and retry parameters (at least
min and max frequencies) and the sender should use this to query for the
updated status if it cares. The sender, conversely, should be able to
say "i give up" so the recipient can stop waiting before the max
frequency expires. I'm not sure if MAIL FROM is even necessary, since
there will never be any bounces.

The recipient should be able to specify "send headers only", "send
message hash only", "send list of signers only", "send headers and one
plain-text body part only", and such in response to DATA, and again in
response to the sender retry.  This could even allow a recipient system
to get a minimal version of a suspicious message to present to the user
for the user to make the final disposition decision.

One other interesting feature someone suggested to me in a similar
discussion: allow the recipient MX to tell the sender to contact a
different IP address and port to deliver the message. His idea was that
an ISP's MX could know that the user was online and running a delivery
daemon, and could tell the sender to deliver the message directly to
said user instead of having to store it locally and wait for the user to
do a POP3 poll. We'd have to consider the case where the user is no
longer online, or worse a different user is now on that address.


Ok, that's enough for tonight.