agronholm / smtpproto Goto Github PK

From RFC 6531:

Because there is no guarantee that a next-hop SMTP server will
support the SMTPUTF8 extension, use of the SMTPUTF8 extension always
carries a risk of transmission failure. In fact, during the early
stages of deployment for the SMTPUTF8 extension, the risk will be
quite high. Hence, there is a distinct near-term advantage for
ASCII-only messages to be sent without using this extension. The
long-term advantage of casting ASCII [ASCII] characters (0x7f and
below) as UTF-8 form is that it permits pure-Unicode environments.

So I think in practice, many (most?) servers will support SMTPUTF8, but we shouldn't use it unless the message requires it.

The stdlib smtplib module handles this by testing all of the address to see if any of them are non-ascii: https://github.com/python/cpython/blob/master/Lib/smtplib.py#L953. That's at the whole message level though.

The only way I can really see to resolve this is to add an option for smtputf8=True to the mail method and push handling of the problem upstream, because we need to send the SMTPUTF8 option with the MAIL comamnd if any of the recipient addresses is UTF-8, but when the mail method is called, we don't yet know the recipients.

Originally posted by @cole in #2 (comment)

Hello! I'm testing this library in my pet project, and I'm having quite a number of problems trying to send email.

To start with, the configuration is as follows:

• host "smtp.mail.ru"
• port 465
• the server tells to use "tls/ssl"

Now, some of these problems may come from the non-standard server, but here is my story nonetheless.

First of all, the initial waiting hangs and ends after timeout: https://github.com/agronholm/smtpproto/blob/master/src/smtpproto/client.py#L82

I looked in and discovered, that connect_tcp function accepts ssl_context as argument, but it is not passed (though, the client might be instantiated with it). https://github.com/agronholm/smtpproto/blob/master/src/smtpproto/client.py#L79
So I passed ssl.SSLContext(), and connected. I know, that TLSWrapper is applied later, but for some reason I couldn't even pass this line to get it.

Then I failed trying to authenticate with LoginAuthenticator. The exception was:

ValueError: Unhandled question: VXNlcm5hbWU6

which is "Username:" when decoded. So, essentially the problem is the semicolon. I don't know, whether it's a violation of standard or not, but maybe make an additional check here? https://github.com/agronholm/smtpproto/blob/master/src/smtpproto/auth.py#L72
Like, instead for strict equality, check that question[:8] == "username" or something?

Also, correct me if I'm wrong, but question is never decoded from bytes https://github.com/agronholm/smtpproto/blob/master/src/smtpproto/auth.py#L71 So the latter checks don't make sense.

After I fix all that, I get the final nail in the coffin with

smtpproto.protocol.SMTPProtocolViolation: Tried to send a command before the previous one received a response

To be honest, I don't understand, under which circumstances this error should popup at all? Shouldn't we wait for response, before sending the next command?

P.S. After all that, I tried sending a mail with standard Django smtp EmailBackend for comparison. And you know what? It hanged out completely :-) Maybe its the kind of problem, that Nathaniel Smith wrote here about select.select... https://vorpus.org/blog/control-c-handling-in-python-and-trio/#how-does-the-python-interpreter-pull-it-off

P.P.S. aiosmtplib implementation works without problems.

But anyway, there are definitely some questions to smtpproto as well. Could you check them out, please?

Hi, I created a few test scripts:

1. Using imaplib python & multiprocessing + multi threading
2. Using aioimaplib python & asyncio
3. Using smtpproto python & trio

My imaplib script, connects & sends mail from about 80-90SMTP servers I give it
My aioimaplib script, connects & send mail from about 100-110 SMTP servers I give it
My smtpproto script, connects & send mail from about 20~ of the SMTP servers I give it

These tests were done using identical input smtps. Not sure what the problem is, as smtpproto is supposed to support plaintext auth, over SSL and STARTTLS & plaintext.

I can potentially provide list to devs for trying to solve this issue. My guess would be that it's not handling starttls properly..? I dont know.

It looks like smtpproto correctly handles the message body when the SMTPUTF8 extension is available.

However, per the spec, the MAIL FROM command should include the SMTPUTF8 option in that case. Currently there's no way to pass an option via the mail method. This could be solved by adding an options argument to that method.

It also seems like all output is always ASCII encoded. However, if SMTPUTF8 is supported and being used for the message, the MAIL FROM command (and subsequent RCPT commands) should be sent UTF-8 encoded.