async-email / async-imap Goto Github PK

With #94 we are checking for EOF and similar kind of errors, but there is no check that response is actually OK as long as we get any tagged response. In case of STORE a NO response means we actually failed to store anything, and in case of FETCH it means that no responses not necessarily indicate there no messages, just that IMAP server lost connection to its NFS server or there is a loose SATA cable resulting in a disk read failure.

When forking the original imap library the documentation did not get a proper update, to ensure everything is up to date. Need to go through all doc comments and make sure they match the actual state of things and add missing docs where needed.

Currently Client is parametrized over the Read + Write + fmt::Debug types. Unfortunately, because this is a requirement of three non-marker traits, it does not seem possible to write code which is agnostic to the specific kind of the underlying stream by using a trait object like Box<dyn Read + Write + fmt::Debug>.

My use case is that I need to write some code which works with IMAP servers, but depending on configuration those servers can be TLS or non-TLS. As of now, it does not seem possible to handle such servers in a uniform way.

I'm frankly not sure how, but it would be great if the library supported some kind of a way to write code which does not depend on the connection details of a particular client and would allow working with any of them. I guess the most straightforward way to do it would be to extract the public interface of the Client and Session structs to traits, but given that traits can't have async method now, it might be hard to do...

Without the async-smtp dependency the compilation of the example/basic.rs failed with:

 let messages: Vec<_> = messages_stream.collect::<Result<_>>().await?;
   |                                            ^^^^^^^ method cannot be called on `impl async_std::stream::Stream+std::marker::Send` due to unsatisfied trait bounds

Hi,

this is more a question than an issue itself. The current IDLE implementation is designed in a way that the future resolves for anything that comes from the server. I am wondering then what would be best approach for long IDLE polling, where client listens continuously for any change from the server side. I was thinking of a loop:

        loop {
            let (idle_wait, _interrupt) = idle.wait();
            let idle_result = idle_wait.await.unwrap();
            match idle_result {
               ...
            };
        }

The above approach calls #wait each time and apart from that, there's a tiny possibility to miss server message when the current future is resolved and the execution is in the match statement. What would be best approach to tackle that?

STATUS command currently calls parse_mailbox and attempts to returns a Mailbox structure:

This is the same parser as used for SELECT response. However, STATUS response format is different from the SELECT response:

? SELECT INBOX
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.
* 0 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1697802745] UIDs valid
* OK [UIDNEXT 1] Predicted next UID
? OK [READ-WRITE] Select completed (0.001 + 0.000 secs).
? STATUS INBOX (UIDNEXT)
* STATUS INBOX (UIDNEXT 1)
? OK [CLIENTBUG] Status on selected mailbox completed (0.001 + 0.000 secs).

As a result, when issuing a status("INBOX", "(UIDNEXT)") command, a Mailbox is returned successfully, but without uid_next.

In deltachat-core-rust repl I configure an address [email protected], then run oauth2, set the token generated as mail_pw and then run `configure.

Here is a result of repl with RUST_LOG=async_imap=trace enabled:

 TRACE async_imap::imap_stream > encode: input: Some(RequestId("A0001")), Ok("AUTHENTICATE XOAUTH2")
 TRACE async_imap::imap_stream > decode: input: Ok("+ \r\n")
 TRACE async_imap::imap_stream > encode: input: None, Ok("*CHALLENGE-RESPONSE BASE64 WAS HERE*")
 TRACE async_imap::imap_stream > encode: input: Some(RequestId("A0002")), Ok("CAPABILITY")
 TRACE async_imap::imap_stream > decode: input: Ok("A0001 NO [AUTHENTICATIONFAILED] AUTHENTICATE invalid credentials or IMAP is disabled sc=...\r\n")
 TRACE async_imap::imap_stream > decode: input: Ok("* CAPABILITY IMAP4rev1 CHILDREN UNSELECT LITERAL+ NAMESPACE XLIST BINARY UIDPLUS ENABLE ID AUTH=PLAIN AUTH=XOAUTH2 IDLE MOVE\r\n")
 TRACE async_imap::imap_stream > decode: input: Ok("A0002 OK CAPABILITY Completed.\r\n")

There is a related bug djc/tokio-imap#67 reference in async-imap:

Continue is parsed successfully now, but here, after sending the continuation, result of the command A0001 is never processed:

As a result, the response A0001 NO [AUTHENTICATIONFAILED] ... is simply ignored, CAPABILITY is sent and then Delta Chat continues sending commands, tries to create folders and so on:

 TRACE async_imap::imap_stream > decode: input: Ok("A0003 BAD [CLIENTBUG] LIST Wrong session state for command sc=...\r\n")

It is not obvious that IMAP should be enabled in Yandex even after passing OAuth2 authentication, so users report that Yandex is not working: deltachat/deltachat-android#1526

Currently, this crate is hard-coded to async-std and async-native-tls. There are other options for async runtimes and TLS libraries. Are you open to adding features to support them? I would be proposing sth similiar to async-tungstenite:

• Async Runtimes: async-std-runtime, tokio-runtime, maybe gio-runtime
• TLS libraries: async-tls, async-native-tls, tokio-tls, maybe tokio-rustls

I would be open to implementing this when I have time if you are not opposed.

Same thing for async-smtp.

ImapStream stores inner as a generic Read + Write type, so even when it is a TCP stream, shutdown method can't be called. It should be called after LOGOUT command to close the connection.

Probably a new trait is needed with implementations for TcpStream and TlsStream.

Related PR: deltachat/deltachat-core-rust#1595

Hey guys,

awesome work on this project. I was playing with it a little because I might wanna use it for something, but I encountered what I think is a bug: it crashes on doing a .select() call. Here's the error that I get:

Error: Io(Custom { kind: Other, error: "Error(([42, 32, 70, 76, 65, 71, 83, 32, 40, 92, 65, 110, 115, 119, 101, 114, 101, 100, 32, 92, 70, 108, 97, 103, 103, 101, 100, 32, 92, 68, 101, 108, 101, 116, 101, 100, 32, 92, 83, 101, 101, 110, 32, 92, 68, 114, 97, 102, 116, 32, 92, 42, 41, 13, 10, 42, 32, 79, 75, 32, 91, 80, 69, 82, 77, 65, 78, 69, 78, 84, 70, 76, 65, 71, 83, 32, 40, 92, 42, 32, 92, 65, 110, 115, 119, 101, 114, 101, 100, 32, 92, 70, 108, 97, 103, 103, 101, 100, 32, 92, 68, 101, 108, 101, 116, 101, 100, 32, 92, 83, 101, 101, 110, 32, 92, 68, 114, 97, 102, 116, 41, 93, 32, 80, 101, 114, 109, 97, 110, 101, 110, 116, 32, 102, 108, 97, 103, 115, 13, 10, 65, 48, 48, 48, 51, 32, 79, 75, 32, 91, 82, 69, 65, 68, 45, 87, 82, 73, 84, 69, 93, 32, 83, 69, 76, 69, 67, 84, 32, 99, 111, 109, 112, 108, 101, 116, 101, 100, 13, 10], Alt)) during parsing of [42, 32, 70, 76, 65, 71, 83, 32, 40, 92, 65, 110, 115, 119, 101, 114, 101, 100, 32, 92, 70, 108, 97, 103, 103, 101, 100, 32, 92, 68, 101, 108, 101, 116, 101, 100, 32, 92, 83, 101, 101, 110, 32, 92, 68, 114, 97, 102, 116, 32, 92, 42, 41, 13, 10, 42, 32, 79, 75, 32, 91, 80, 69, 82, 77, 65, 78, 69, 78, 84, 70, 76, 65, 71, 83, 32, 40, 92, 42, 32, 92, 65, 110, 115, 119, 101, 114, 101, 100, 32, 92, 70, 108, 97, 103, 103, 101, 100, 32, 92, 68, 101, 108, 101, 116, 101, 100, 32, 92, 83, 101, 101, 110, 32, 92, 68, 114, 97, 102, 116, 41, 93, 32, 80, 101, 114, 109, 97, 110, 101, 110, 116, 32, 102, 108, 97, 103, 115, 13, 10, 65, 48, 48, 48, 51, 32, 79, 75, 32, 91, 82, 69, 65, 68, 45, 87, 82, 73, 84, 69, 93, 32, 83, 69, 76, 69, 67, 84, 32, 99, 111, 109, 112, 108, 101, 116, 101, 100, 13, 10]" })

I wanted to see what was going on, so what I did actually is I wrote a little proxy around the TlsStream so I can peek into what is happening on the wire. This is the code in order to reproduce this:

use async_imap::error::{Error, Result};
use async_imap::Client;
use async_std::io::{Read, Write};
use async_std::pin::Pin;
use async_std::prelude::*;
use async_std::task;
use async_std::task::{Context, Poll};
use async_std::net::TcpStream;
use futures::io::{AsyncRead, AsyncWrite};
use std::env;
use std::fmt;

#[derive(Debug)]
struct Proxy<T: AsyncRead + AsyncWrite + Unpin + fmt::Debug> {
    transport: T,
}

impl<T: AsyncRead + AsyncWrite + Unpin + fmt::Debug> AsyncRead for Proxy<T> {
    fn poll_read(
        mut self: Pin<&mut Self>,
        cx: &mut Context,
        buf: &mut [u8],
    ) -> Poll<std::io::Result<usize>> {
        let ret = AsyncRead::poll_read(Pin::new(&mut self.transport), cx, buf);
        match ret {
            Poll::Ready(Ok(num)) => {
                println!("> {}", String::from_utf8_lossy(&buf[0..num]));
            }
            _ => {}
        }
        ret
    }
}

impl<T: AsyncRead + AsyncWrite + Unpin + fmt::Debug> AsyncWrite for Proxy<T> {
    fn poll_write(
        mut self: Pin<&mut Self>,
        cx: &mut Context,
        buf: &[u8],
    ) -> Poll<std::io::Result<usize>> {
        let ret = AsyncWrite::poll_write(Pin::new(&mut self.transport), cx, buf);
        match ret {
            Poll::Ready(Ok(num)) => {
                println!("< {}", String::from_utf8_lossy(&buf[0..num]));
            }
            _ => {}
        }
        ret
    }

    fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context) -> Poll<std::io::Result<()>> {
        AsyncWrite::poll_flush(Pin::new(&mut self.transport), cx)
    }

    fn poll_close(mut self: Pin<&mut Self>, cx: &mut Context) -> Poll<std::io::Result<()>> {
        AsyncWrite::poll_close(Pin::new(&mut self.transport), cx)
    }
}

impl<T: AsyncRead + AsyncWrite + Unpin + fmt::Debug> Proxy<T> {
    pub fn new(transport: T) -> Self {
        Proxy {
            transport
        }
    }
}

fn main() -> Result<()> {
    task::block_on(async {
        fetch_inbox_top("imap.example.com", "[email protected]", "password").await?;
        Ok(())
    })
}

async fn fetch_inbox_top(imap_server: &str, login: &str, password: &str) -> Result<Option<String>> {
    let tls = async_native_tls::TlsConnector::new();
    let imap_addr = (imap_server, 993 as u16);
    let stream = TcpStream::connect(imap_addr).await?;
    let ssl_stream = tls.connect(imap_server, stream).await?;
    let proxied = Proxy::new(ssl_stream);
    let mut client = Client::new(proxied);
    let response = client.read_response().await.unwrap()?;
    println!("{:?}", response);

    // the client we have here is unauthenticated.
    // to do anything useful with the e-mails, we need to log in
    let mut imap_session = client.login(login, password).await.map_err(|e| e.0)?;
    println!("-- logged in a {}", login);

    // list all mailbox names
    let folders = imap_session
        .list(None, Some("*"))
        .await?
        .for_each(|folder| println!("{}", folder.unwrap().name()))
        .await;
    let mbox = imap_session.select("NewsLetter").await?;
    println!("{}", mbox);

    // fetch message number 1 in this mailbox, along with its RFC822 field.
    // RFC 822 dictates the format of the body of e-mails
    let messages_stream = imap_session.fetch("1:*", "FLAGS").await?;
    let messages: Vec<_> = messages_stream.collect::<Result<_>>().await?;
    println!("{}", messages.len());

    // be nice to the server and log out
    imap_session.logout().await?;

    Ok(None)
}

When this is run, the output looks like this (with some information redacted, naturally):

> * OK imap.example.com IMAP4 Server (Mail IMAP4rev1 Server version 1.0)

ResponseData { raw: 4096, response: Data { status: Ok, code: None, information: Some("imap.example.com IMAP4 Server (Mail IMAP4rev1 Server version 1.0)") } }
< A0001
<  
< LOGIN "xxx" "xxx"
< 

> * CAPABILITY IMAP4rev1 UNSELECT CHILDREN XLIST NAMESPACE IDLE MOVE ID AUTH=PLAIN SASL-IR UIDPLUS ESEARCH LIST-EXTENDED LIST-STATUS WITHIN COMPRESS=DEFLATE LITERAL- ACL ENABLE CONDSTORE
A0001 OK Success

-- logged in as [email protected]
< A0002
<  
< LIST "" *
< 

> * LIST (\HasNoChildren) "/" "INBOX"
* LIST (\Noinferiors \Drafts) "/" "Drafts"
* LIST (\Noinferiors) "/" "Templates"
* LIST (\Noinferiors \Sent) "/" "Sent"
* LIST (\Noinferiors \Junk) "/" "Spam"
* LIST (\HasNoChildren \Trash) "/" "Trash"
* LIST (\HasNoChildren) "/" "Notification"
* LIST (\HasNoChildren) "/" "NewsLetter"
* LIST (\HasNoChildren) "/" "Archive"
A0002 OK Success

INBOX
Drafts
Templates
Sent
Spam
Trash
Notification
NewsLetter
Archive
< A0003
<  
< SELECT "NewsLetter"
< 

> * 995 EXISTS
* 995 RECENT
* OK [UNSEEN 7]
* OK [UIDVALIDITY 1] UIDs valid 
* OK [UIDNEXT 1386] Predicted next UID 
* OK [HIGHESTMODSEQ 1000000000000002229]
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)
* OK [PERMANENTFLAGS (\* \Answered \Flagged \Deleted \Seen \Draft)] Permanent flags
A0003 OK [READ-WRITE] SELECT completed

flags: [], exists: 995, recent: 995, unseen: Some(7), permanent_flags: [],uid_next: Some(1386), uid_validity: Some(1)
< A0004
<  
< FETCH 1:* FLAGS
< 

0
< A0005
<  
< LOGOUT
< 

I don't know exactly what is going on, what I did is write a little ruby script to decode the bytes in the Io error, which looks like this:

* FLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)
* OK [PERMANENTFLAGS (\* \Answered \Flagged \Deleted \Seen \Draft)] Permanent flags
A0003 OK [READ-WRITE] SELECT completed

I suppose it is choking on the output of the select call or something?

IMAP protocol doesn't guarantee that Responses come in the same order as commands are received. So in theory the following situation is possible:
client: A CMD_A
client: B CMD_B
server: A OK CMD_A Completed
server: B OK CMD_B Completed
It looks like existing code will fail in this situation.

This is not an issue in original rust-imap due to it's synchronous nature.

Unilateral server data responses are responses that can come at any time, so they need to be processed every time response is received. This is also an issue with original rust-imap project.
See https://tools.ietf.org/html/rfc3501 section 7. Server Responses

Hello there!

First of all, thank you for this amazing library, I've been using it for a while and it works like a charm :)

I wanted to extract the from data from the Envelope type, and I see it is not a string but an Address type where all the fields are optional.

I wanted to serialize it into a string of the format Name surname <email>.

I understand that the field name contains the name of the sender, but I'm struggling to understand which field contains the proper email address.

Is it the adl field?

Also, why is it a Vec<Address>? That confuses me a bit

Thank you!

Many servers support UIDplus which would help to know the new UID after an imap move operation.

Thanks for the library. I'm using it with others for a load injector. I'm depending already on tokio.

I have this compilation error:

error[E0277]: the trait bound `tokio::net::TcpStream: futures_io::if_std::AsyncRead` is not satisfied
  --> src/imap_client.rs:6:13
   |
6  |     client: Client<TlsStream<TcpStream>>,
   |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `futures_io::if_std::AsyncRead` is not implemented for `tokio::net::TcpStream`

I have reproduced a similar compilation issue with the example: I removed optional features and set tokio as configured in my project (that includes same features from tokio i.e. rt-multi-thtread and macros). So I came with the following Cargo.toml:

[package]
name = "async-imap-examples"
version = "0.1.0"
publish = false
authors = ["dignifiedquire <[email protected]>"]
license = "Apache-2.0/MIT"
edition = "2018"


[dependencies]
anyhow = "1"
async-imap = { path = "../", features = [ "tokio" ]  }
async-native-tls = { version = "0.5", features = [ "tokio" ] }
futures = "0.3.28"
tokio = { version = "1.32", features = ["net", "io-util", "time", "rt", "rt-multi-thread", "macros"] }

And the headers from the idle file:

use std::env;
use std::time::Duration;

use anyhow::{bail, Result};
use async_imap::extensions::idle::IdleResponse::*;
use futures::StreamExt;

use tokio::{net::TcpStream, task, time::sleep};

#[tokio::main]
async fn main() -> Result<()> {
    // ...
}
// ...

What is missing or is misconfigured in this file?
(sorry if it's a dumb question, I'm not a rust advanced user yet)

IMAP METADATA extension is supported in imap-proto, but there is no interface in async-imap.

Support for GETMETADATA and SETMETADATA command is needed.

This can be added similarly to how QUOTA extension was added in #47.

The documentation of Client::new states:

For an example of how to use this method to provide a pure-Rust TLS integration, see the
rustls.rs in the examples/ directory.

There is no such file.

The types::Fetch struct allows to iterate over flags no matter whether the field is actually returned by the server. It does not distinguish between the case of the fetch response not containing any flag updates and the message not having any flags. This is an issue for unsolicited fetch responses, as they might or might not contain any flag updates, leading to ambiguity.

Example: FETCH response with no flags set:
12 FETCH (FLAGS () UID 20)

Example: FETCH response with other data:
12 FETCH (UID 20)

There currently is no way to distinguish these two types of fetch responses via public API, right?

I can consistently observe that requests like

        let messages = session.fetch("7", "BODY[HEADER]").await.unwrap();

always hang when processing the messages stream, i.e. something like

        futures::pin_mut!(messages);
        let message = messages.next().await.unwrap();

never finishes.

The issue goes away if BODY[HEADER] is not requested, e.g. with BODY[TEXT] or UID or ENVELOPE or BODYSTRUCTURE. BODY[] does not work as well, so it seems that there is some issue with headers processing?

I see this consistently with different mailboxes and different messages on a Fastmail account (imap.fastmail.com).

All of these requests can be done just fine with a raw telnet connection to the server.

So I need to fetch both the send date and the received date of an email.

I am not so familiar with the IMAP protocol so this is more a question about IMAP rather than a problem with this library.

Until now I have been assuming that the send date is the one on the Envelope date field.

Is that correct? I've seen that there is an INTERNALDATE argument that can be passed on the Fetch query, is that going to return the received date?

Hello there:) This is more a question than an issue.

Is it possible to read the attachments of an email? Since IMAP protocol supports it I was wondering if this library can also handle that, otherwise I could try to contribute :)

Thank you 🚀

I'm testing things, so I prefer not to use TLS to keep my domain's certificate logs clean. A self-signed certificate requires additional configuration and has risk to introduce extra issues (because they can't be trusted by third party servers).

The examples just use send, whereas at least for STARTTLS with authentification, a prior call to connect is required.

Currently imap_proto has a git dependency, which is different, although with the same version, from the one on crates.io. This makes it inconvenient to use the library, because in-project manifest must have a matching the git dependency, because async-imap has types from imap_proto in its public API and can't sensibly be used without it.

I believe that the most idiomatic way would be to re-export the entire imap_proto crate, so there will be no need for the downstream users to import a correct version of it.

ouroboros is no longer maintained and has a RUSTSEC-2023-0042 advisory suggesting switch to self_cell.

Upstream issue: someguynamedjosh/ouroboros#88

Dear @async-email team,

Can you add the support of RFC 9266: Channel Bindings for TLS 1.3?

• https://datatracker.ietf.org/doc/html/rfc9266

Channel Bindings for TLS: https://datatracker.ietf.org/doc/html/rfc5929

• XEP-0388: Extensible SASL Profile: https://xmpp.org/extensions/xep-0388.html
• XEP-0440: SASL Channel-Binding Type Capability: https://xmpp.org/extensions/xep-0440.html
• XEP-0474: SASL SCRAM Downgrade Protection: https://xmpp.org/extensions/xep-0474.html
• XEP-0480: SASL Upgrade Tasks: https://xmpp.org/extensions/xep-0480.html

Little details, to know easily:

• tls-unique for TLS =< 1.2
• tls-server-end-point
• tls-exporter for TLS = 1.3

I think that you have seen the jabber.ru MITM and Channel Binding is the solution:

• https://notes.valdikss.org.ru/jabber.ru-mitm/
• https://snikket.org/blog/on-the-jabber-ru-mitm/
• https://www.devever.net/~hl/xmpp-incident
• https://blog.jmp.chat/b/certwatch

Thanks in advance.

Linked to:

• scram-sasl/info#1
• #99

The current unsafe code is actually not safe, needs to be fixed

I'm not entirely sure what the issue is, however I have tried downloading a mailbox with thousands of mails and this occured: on a very big message, the result of session.uid_fetch(...).await?.try_next().await? returned None and the next request broke the connection with the error: io: inner stream closed. I did some further inspection with RUST_LOG=trace and found out that on every message I try to fetch which causes this error, the following happens:

[2022-12-17T16:05:25Z TRACE async_imap::imap_stream] decode: input: Ok("* 343 FETCH (INTERNALDATE ... some very long data and an HTML email ... </body>\r\n</html>\r\n UID ")
[2022-12-17T16:05:25Z TRACE async_imap::imap_stream] decode: incomplete data, need minimum 1 bytes

After that, it just return None and the next request returns an error. There seems to be an issue whenever the current input buffer ends with " UID ", because the only messages that ended with " UID " were the ones that caused this issue.

Anyone tried to connect to gmail imap using email and access-token?
It simply go dead on me. Tried a couple days, nothing worked.
I simply use the example code.
It dies on client.Authenticate("XOAUTH2", @Gmailauth)

Who had any success? by the way, I use runtime-tokio feature.

An email client will most of the time use an IDLE loop to get the new messages/deleted messages events in real time.
What would be the best way to do this with async-imap as the session is borrowed by idle?

I have stored a session in a struct:

pub struct ImapClient {
    session: async_imap::Session<async_native_tls::TlsStream<TcpStream>>
}

That session is initialized with a try_new method.

Then I have a method that makes an idle wait. It returns the number of fetched messages.

   pub async fn wait_for_new_messages(&mut self) -> Result<usize> {
        self.session.select("INBOX").await.unwrap();
        let mut idle = self.session.idle(); // boom: self.session moved due to this method call
        idle.init().await?
       // code for idle/fetch EXISTS
    }

cf this file

The caller is responsible to loop like:

loop {
    match imap_client.wait_for_new_messages().await {
        Ok(nb) => debug!("received {} messages", nb),
        Err(err) => error!("{:?}", err)
    }
}

I understand that the session moves to the idle Handler, to return it when idle is done.

How could I use the session instance with idle?
(@hpk42 I will push a doc PR with my findings, I just want to reach a working example)
Thank you for your answers.

What I tried:

• encapsulate idle handler in the struct ImapClient but when creating the session, same issue is happening with self
• encapsulate the Client and recreating a session each time but that means re-authenticate at each idle loop step thus having an unnecessary network overhead
• encapsulate the Client and making the idle loop inside the ImapClient. But same issue happens with the borrow in the loop

Placeholder for tracking things that need to happen to bring async-imap back to rust-imap.

• Make generic over the executor
• Add examples for async-std, [email protected] and futures executors
• Implement append (currently unimplemented)

cc @hpk42 @jonhoo

Running any of these:
(with any valid uid)

let mails: Vec<_> = session.uid_fetch("22213", "BODY.PEEK[1]").await?.try_collect().await?;
let mails: Vec<_> = session.uid_fetch("22213", "BODY[1]").await?.try_collect().await?;

causes no body, text or any other data to be parsed, and the trace shows it hitting this:

The need minimum X bytes is always a little lower than the size of the data returned. In this case 220 bytes returned, but it says minimum 130.

Run with gmail imap servers, with this crate setup:

async-imap = { version = "0.6.0", default-features = false, features = ["runtime-tokio"] }

calling BODY[] or BODY.PEEK[] or any other call works fine.

Connecting to the imap server directly like so:

openssl s_client -connect imap.gmail.com:993 -crlf

and running the same commands (with same UIDs) seems to work fine.

just got a crash in delta chat async-imap branch ---

operating_system = 'unix:Ubuntu'
crate_version = '1.0.0-beta.7'
explanation = '''
Cause: internal error: entered unreachable code. Panic occurred in file '/home/hpk/.cargo/registry/src/github.com-1ecc6299db9ec823/async-imap-0.1.1/src/parse.rs' at line 187
'''
method = 'Panic'
backtrace = '''
stack backtrace:
   0:     0x7fae7320f427 - backtrace::backtrace::trace::h7ad15d3ba1db492a
   1:     0x7fae7320d943 - backtrace::capture::Backtrace::new::h7932c2e753adbbce
   2:     0x7fae733abbb7 - human_panic::report::Report::new::he2ac877c8b711f90
   3:     0x7fae731539b2 - human_panic::handle_dump::hbeafbbc6aef4e8e5
   4:     0x7fae729189f9 - deltachat::dc_context_new::{{closure}}::h7c5d0f9d6826014b
   5:     0x7fae73237890 - std::panicking::rust_panic_with_hook::hafa4d144cdeac0c6
                               at src/libstd/panicking.rs:468
   6:     0x7fae72b08a05 - std::panicking::begin_panic::h4c92c43e9be7146a
   7:     0x7fae72a6aec2 - std::future::poll_with_tls_context::h2717626796b099e9
   8:     0x7fae72a640cb - std::future::get_task_context::h8dae28bd70e0773f
   9:     0x7fae72a99505 - <std::future::GenFuture<T> as core::future::future::Future>::poll::h4f2d9c4dfff0731b
  10:     0x7fae7327f69e - std::thread::local::LocalKey<T>::with::hcd29b87972716505
  11:     0x7fae7327e833 - std::thread::local::LocalKey<T>::with::h9abfbb0e139be77c
  12:     0x7fae72b9637a - async_std::task::block_on::block_on::h71a86ca31485710b
  13:     0x7fae72b1c31f - deltachat::job_thread::JobThread::connect_and_fetch::h7ac4e219fcb7a546
  14:     0x7fae72b1bd9e - deltachat::job_thread::JobThread::fetch::hbbc4a3202d408418
  15:     0x7fae72b9e753 - deltachat::job::perform_inbox_fetch::hbf2e0e14c673b4f2
  16:     0x7fae7291ad3f - dc_perform_imap_fetch
  17:     0x7fae7290d4e0 - imap_thread_func
  18:     0x7fae8af516ba - start_thread
  19:     0x7fae850f941d - clone
  20:                0x0 - <unknown>
'''

warning: the following packages contain code that will be rejected by a future version of Rust: async-imap v0.5.0, rental v0.5.6
note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 274`

Looking at the incompatibility report, it seems that rental is the cause of this, which was fixed in #52, but not cut as a release.

> warning: using `procedural-masquerade` crate
>   --> /home/nick/.cargo/registry/src/github.com-1ecc6299db9ec823/async-imap-0.5.0/src/types/name.rs:7:1
>    |
> 7  | / rental! {
> 8  | |     pub mod rents {
> 9  | |         use super::*;
> 10 | |
> ...  |
> 17 | |     }
> 18 | | }
>    | |_^
>    |
>    = note: `#[allow(proc_macro_back_compat)]` on by default
>    = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
>    = note: for more information, see issue #83125 <https://github.com/rust-lang/rust/issues/83125>
>    = note: The `procedural-masquerade` crate has been unnecessary since Rust 1.30.0. Versions of this crate below 0.1.7 will eventually stop compiling.
>    = note: this warning originates in the macro `rental` (in Nightly builds, run with -Z macro-backtrace for more info)
> 

Would it be possible to cut a new release of this crate?

Hi, I'm trying the example code, and I stumbled upon this specific line.

When reproducing it, I had to add a .as_ref() to make the compiler stop complaining. Also an extra unwrap(). As so:

    let e = fetch.as_ref().unwrap().envelope().unwrap();

I don't see the CI failing. So I'm wondering if the CI never ends up running building this line of code or whether I'm doing something wrong.

I am 0.4.1.

Thanks for all the work.

cargo check --examples is not sufficient, because examples are currently in a separate directory with their own Cargo.toml. Switching to examples/ and running cargo check throws various errors.

The CAPABILITY command is available before login (specified under "Client Commands - Any State" in RFC3501).
This library only seems to implement it on Session, which is constructed after authentication.

It would be useful to check the capabilities before authentication, for example to see which authentication methods are available.

Dear @async-email team,

Can you add supports of :

• SCRAM-SHA-1
• SCRAM-SHA-1-PLUS
• SCRAM-SHA-256
• SCRAM-SHA-256-PLUS
• SCRAM-SHA-512
• SCRAM-SHA-512-PLUS
• SCRAM-SHA3-512
• SCRAM-SHA3-512-PLUS

You can add too:

• SCRAM-SHA-224
• SCRAM-SHA-224-PLUS
• SCRAM-SHA-384
• SCRAM-SHA-384-PLUS

"When using the SASL SCRAM mechanism, the SCRAM-SHA-256-PLUS variant SHOULD be preferred over the SCRAM-SHA-256 variant, and SHA-256 variants [RFC7677] SHOULD be preferred over SHA-1 variants [RFC5802]".

• SCRAM-SHA-1(-PLUS):
  -- https://tools.ietf.org/html/rfc5802
  -- https://tools.ietf.org/html/rfc6120

• SCRAM-SHA-256(-PLUS):
  -- https://tools.ietf.org/html/rfc7677 since 2015-11-02
  -- https://tools.ietf.org/html/rfc8600 since 2019-06-21: https://mailarchive.ietf.org/arch/msg/ietf-announce/suJMmeMhuAOmGn_PJYgX5Vm8lNA

• SCRAM-SHA-512(-PLUS):
  -- https://tools.ietf.org/html/draft-melnikov-scram-sha-512

• SCRAM-SHA3-512(-PLUS):
  -- https://tools.ietf.org/html/draft-melnikov-scram-sha3-512

• SCRAM BIS: Salted Challenge Response Authentication Mechanism (SCRAM) SASL and GSS-API Mechanisms:
  -- https://tools.ietf.org/html/draft-melnikov-scram-bis

https://xmpp.org/extensions/inbox/hash-recommendations.html

-PLUS variants:

• RFC5056: On the Use of Channel Bindings to Secure Channels: https://tools.ietf.org/html/rfc5056
• RFC5929: Channel Bindings for TLS: https://tools.ietf.org/html/rfc5929
• Channel-Binding Types: https://www.iana.org/assignments/channel-binding-types/channel-binding-types.xhtml
• RFC 9266: Channel Bindings for TLS 1.3: https://tools.ietf.org/html/rfc9266

IMAP:

• RFC9051: Internet Message Access Protocol (IMAP) - Version 4rev2: https://tools.ietf.org/html/rfc9051

LDAP:

• RFC5803: Lightweight Directory Access Protocol (LDAP) Schema for Storing Salted: Challenge Response Authentication Mechanism (SCRAM) Secrets: https://tools.ietf.org/html/rfc5803

HTTP:

• RFC7804: Salted Challenge Response HTTP Authentication Mechanism: https://tools.ietf.org/html/rfc7804

2FA:

• Extensions to Salted Challenge Response (SCRAM) for 2 factor authentication: https://datatracker.ietf.org/doc/html/draft-ietf-kitten-scram-2fa

IANA:

• Simple Authentication and Security Layer (SASL) Mechanisms: https://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml

Linked to:

• scram-sasl/info#1
• #100

This is a follow-up to #66 bug. While setting the number of bytes requested by the parser in #74 made the issue harder to reproduce and fixed particular testcase, the original issue of closed state being erroneously set was not investigated.

I looked into it and found that async-imap sometimes tries to read into a zero-sized buffer, then treats zero read bytes as the indication of closed stream:

ensure_capacity() at the beginning of this snippet is supposed to ensure at least one byte is free in the buffer, but due to a bug it sometimes does not allocate even a single byte. I reproduced the issue at #77.

async-smtp has a per-connection timeout setting that limits how long we wait until we receive a command response:
https://github.com/async-email/async-smtp/blob/da7e8d407cb4e37fdf2aa7e0bd5d402a263de2df/src/smtp/smtp_client.rs#L218

But async-imap currently uses timeout only for the IDLE command. This may cause IMAP client getting stuck forever trying to read the response if connection is lost (we suspended our machine and NAT has forgot about the connection, for example). This is very likely a cause for this problem of IMAP task getting stuck forever in Delta Chat:
deltachat/deltachat-android#2400 (comment)

The simplest way is to do like async-smtp and have a 60-second timeout for each read and for each write call.

Hello,

I was using DeltaChat a while ago (which uses your crate) and I could not have their client authenthicate with SASL, encrypted connection and password.

My understanding is that this is not yet supported by async-imap - is this something you would evaluate for implementation at some point? I'm speaking as the final user with very vague idea of the effort needed, so I'll be happy to receive some more context.

Thanks for working on this crate by the way :)

Need to run the docker container with greenmail to fix it