minosiants / pencil Goto Github PK

First of all, thank you for this excellent library! I've put it to good use in my current project. However, I've run into the following issue: Some intermediate SMTP servers reject the messages that were sent. The error message is 501 Syntax error - line too long

I'm using smtp.gmail.com to send HTML messages, and the server always accepts the send request: Code(250, Requested mail action okay, completed). Whether the email actually arrives in the recipient's inbox depends on the email service provider the recipient is using. The emails are successfully delivered if the recipients use gmail.com or outlook.com. One example where the messages are not delivered is when the recipient has an account with email service gmx.de. In such a case, the Google SMTP server sends a message delivery failure email to the senders inbox. This message contains the error code I have described above.

I've briefly scanned the pencil codebase yesterday, and I assume that it generates message bodies that do not comply with the RFC2045 specification. Here is why:

• The message body is created in Smtp.mimeBody [1]

• This calls the toBase64 method on the Body which essentially forwards the Base64 encoding to scodecs BitVector [2]

• Depending on the body content, this will create a very long Base64 string without linebreaks

• This contradicts RFC2045 Section 6.8 - Base64 Content-Transfer-Encoding:

  "The encoded output stream must be represented in lines of no more than 76 characters each." [3]

Please note that this is only an assumption and I might be totally wrong here. This is the StackOverflow post that pointed me in this direction: [4]

Hi there,
Trying to use Pencil with AWS SES I found the following error com.minosiants.pencil.data.Error$SmtpError. After enabling DEBUG level in the logs configuration I found a Getting Error: replies/code: 530 code does not exist related to auth issues. Then I implemented another client and i found that the the following is being thrown 500,Syntax error, command unrecognized by the following commands:

• Smtp.data[F]()
• Smtp.mail[F]()
• Smtp.rcpt[F]()
• Smtp.endEmail[F]()
• Smtp.quit[F]()
  Removing those, emails are being sent. I don't know too much about SMTP and I'll do a deeper research.

Empty line before Content-Type header causes an incorrect message rendering.

That uses platform encoding, which may not be an ascii|utf8 compliant encoding.

String.getBytes(Standardcharsets.UTF_8) or String.getBytes(Standardcharsets.US_ASCII) should be preferred

An email generated like this

val email = Email.text(
  From(...),
  To(...),
  Subject(...),
  Body.Ascii("foo: bar")
)
client.send(email)

results in an empty email message body because RFC822 interprets the body as a header. The workaround is to insert a newline before any body text that has a : on the first line:

val safeEmail = if (email.body.exists(_.value.takeWhile(_ != '\n').contains(':'))) {
  email.copy(body = email.body.map(b => Body.Ascii("\n" + b.value)))
} else {
  email
}
client.send(safeEmail)

I think the best would be if this library only depends on log4cats-core and ask for users for an instance of a Logger[F].

This way, the library wouldn't need to depend on log4cats-slf4j & logback-classic and wouldn't need to have a logback.xml file in the main resources.

Hi, this will be a weird issue since I do not really know if this is a pencil problem, but right now I do not know w that else to do and thought that I will not lose anything by asking here.

So first let me explain what the problem is.
I have a couple of Scala applications running in AWS ECS containers, those applications send some emails from time to time, those emails contain some Excel (xlsx) files as attachments. And sometimes, the email got corrupted.
The received email doesn't contain the excel file and the body of the email is filled with strange characters; my assumption is that those characters are the Excel file that for some reason, instead of being the MIMEPart for the attachments, was moved to the body.

I have done a couple of experiments and the results are:

• The problem is sporadic. - Sometimes happens, sometimes no (this is the most frustrating part of this).
• The problem is not related to the account sending the email or the account receiving it. - I have tried with different accounts, both professional and personal (however, all accounts used to send the email where office accounts).
• The problem never happens if running in local, only in AWS ECS. - Of course, since the problem is sporadic, all my local attempts may have just been lucky, but I have done enough to believe it is not related (note that I have tried both running directly from sbt as well as running the same Docker image that runs in AWS).
• The problem never happens with emails without attachments. - Same disclaimer as above.
• The problem never happens with plain (txt) files. - Same disclaimer as above.

My reason to believe this could be related to pencil is that previously I was using javax-mail and it never happened. But, I haven't tested with that again.

Relevant information:

• Scala version 2.13.3
• Docker image adoptopenjdk/openjdk11:alpine-jre
• I am using the Graal JIT through the use of -XX:+UseJVMCICompiler.

If I can help with any other experiment let me know, I will report results ASAP.

Hello, thanks for this amazing library!

I found this little issue which is somewhat annoying.
Small script to easily reproduce the error:

// import $ivy.`com.minosiants::pencil:0.3.0`

import cats.effect.{Blocker, Concurrent, ContextShift, IO, Resource, Sync}
import cats.syntax.all._
import com.minosiants.pencil.Client
import com.minosiants.pencil.data.{Error => PencilError, _}
import fs2.io.tcp.SocketGroup
import fs2.io.tls.TLSContext

import java.nio.file.{Path, Paths}

object Mail {
  final case class MailConfig(
      emailUser: String,
      emailPassword: String,
      emailTo: String
  )

  def sendEmail[F[_] : Concurrent : ContextShift](
    blocker: Blocker,
    mailConfig: MailConfig,
    attachment: Path
  ): F[Unit] = {
    val clientR = for {
      sg <- SocketGroup[F](blocker)
      tlsContext <- Resource.liftF(TLSContext.system[F](blocker))
    } yield Client[F](
      host = "outlook.office365.com",
      port = 587,
      credentials = Credentials(
        username = Username(mailConfig.emailUser),
        password = Password(mailConfig.emailPassword)
      ).some
    )(
      blocker,
      sg,
      tlsContext
    )

    clientR.use { client =>
      for {
        from <- Sync[F].fromEither(Mailbox.fromString(mailConfig.emailUser))
        to <- Sync[F].fromEither(Mailbox.fromString(mailConfig.emailTo))
        email = Email.mime(
          from = From(from),
          to = To(to),
          subject = Subject(s"Test UTF-8 attachment"),
          body = Body.Utf8("")
        ).addAttachment(Attachment(attachment))
        _ <- client.send(email)
      } yield ()
    }
  }
}

@main
def main(): Unit = {
  implicit val cs = IO.contextShift(scala.concurrent.ExecutionContext.global)

  Blocker[IO].use { blocker =>
    Mail.sendEmail[IO](
      blocker,
      mailConfig = Mail.MailConfig(
        emailUser = "", // CHANGE THIS!
        emailPassword = "", // CHANGE THIS!
        emailTo = "" // CHANGE THIS!
      ),
      attachment = Paths.get("test-ñ.txt") // Or any other non-ascii character.
    ).recoverWith {
      case pencilError: PencilError => IO(println(pencilError.show))
    }
  }.unsafeRunSync()
}

Which produces the following error:

Smtp error: US-ASCII cannot encode character 'ñ'

However, I can send such file using the Office 365 web client, or an Outlook desktop client.

I will try to find some time to send a PR fixing this, but I can't promise right now.

Hi,

I'm using this library to send some html email with attachments, and there is issue when emails are sent to iCloud accounts.

The email content is pretty simple, html + pdf file.
However, emails are rejected by iCloud mail server.

The error iCloud mail server returns is (not much more explanation):

554 5.7.1 [CS01] Message rejected due to local policy. Please visit https://support.apple.com/en-us/HT204137

The culprit seems to be Message-ID header. Once it is present those emails are rejected.

For the test I sent a simple email via curl and then another one with Message-ID generated by the library, - the second email got blocked.

curl --ssl-reqd \
--url 'smtp://smtp.gmail.com:587' \
--user 'user:pass' \
--mail-from [email protected] \
--mail-rcpt <some>@icloud.com \
--upload-file - <<EOF
From: <[email protected]>
To: <[email protected]>
Message-ID: <GENERATED>
Subject: You are awesome!
Content-Type: multipart/alternative; boundary="boundary-string"

--boundary-string
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

This is the body 

--boundary-string
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

<!doctype html>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body style="font-family: sans-serif;">
  This is html body 
  </body>
</html>

--boundary-string--
EOF

Сan it be made optional/removed?

Thanks in advance!