smileychris / django-mailer-2 Goto Github PK

I have a crash log for situation with auth error on SMTP server. After this event I saw infinity hang. Looks like that lockfile didn't free.I had to kill process and to delete lockfile by hands.
Traceback (most recent call last):
File "manage.py", line 11, in
execute_manager(settings)
File "/usr/lib/python2.6/site-packages/django/core/management/init.py", line 438, in execute_manager
utility.execute()
File "/usr/lib/python2.6/site-packages/django/core/management/init.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/lib/python2.6/site-packages/django/core/management/base.py", line 191, in run_from_argv
self.execute(_args, *_options.dict)
File "/usr/lib/python2.6/site-packages/django/core/management/base.py", line 220, in execute
output = self.handle(_args, *_options)
File "/usr/lib/python2.6/site-packages/django/core/management/base.py", line 351, in handle
return self.handle_noargs(**options)
File "/usr/lib/python2.6/site-packages/django_mailer/management/commands/send_mail.py", line 47, in handle_noargs
send_all(block_size, backend=settings.USE_BACKEND)
File "/usr/lib/python2.6/site-packages/django_mailer/engine.py", line 87, in send_all
connection.open()
File "/usr/lib/python2.6/site-packages/django/core/mail/backends/smtp.py", line 48, in open
self.connection.login(self.username, self.password)
File "/usr/lib/python2.6/smtplib.py", line 589, in login
raise SMTPAuthenticationError(code, resp)
smtplib.SMTPAuthenticationError: (454, '4.7.0 Cannot authenticate due to temporary system problem. Try again later. q53sm52029eeh.4')

Hi Chris!

This is a great library. I found the problem. I moved from linode.com (ubuntu / linux) on FreeBSD. And the library is no longer working. I commented out the code to work with LockFile and it solved my problem. But it's bad ... Prompt me what to do with FreeBSD?

>>> from lockfile import FileLock, AlreadyLocked, LockTimeout
>>> lock = FileLock('/tmp/123.lock')
>>> lock.acquire(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "lockfile.py", line 261, in acquire
    raise LockTimeout
lockfile.LockTimeout
>>> 

Help me please.

Thanks!

The latest commit changed the version number from (1, 1, 0, "alpha") to (1, 2, 0).
This breaks the setup because get_version expects 4 elements in the version tuple.

This issue only affects people using postgresql. Basically send_all does not explicitly close the db connection when finished, so postgresql logs the above error. The problem arises when one regularly runs the command (in my case via cron every minute). Every minute an error is logged to the postgresql logs about the connection being dropped on the client side.

Note that django-mailer works completely as expected - the patch created by carl.j.meyer and available at the link below just stops cluttering up the pg log files.

Patch and more info on the problem available here:
http://code.google.com/p/django-mailer/issues/detail?id=15

I have applied this simple patch and can confirm that it resolves this issue for me.

Am raising it here as it might be a good idea to apply this patch for the folks using postgresql.

i just got UnicodeEncodeError in django_mailer/engine.py", line 91, in send_all

fix:

try:
    result = send_queued_message(message, smtp_connection=connection, blacklist=blacklist)
except Exception, e:
    result = constants.RESULT_FAILED
    logger.error(e)

None of django's other backends than the smtp backend can be used, as for example the console backend. This because django-mailer-2 uses the connection attribute of the backend directly, and those backends have no such attribute.

I tryed fixing this by checking for this attribute like-so:

if hasattr(smtp_connection, 'connection'):
                smtp_connection.connection.sendmail(message.from_address,
                                                    [message.to_address],
                                                    message.encoded_message)
else:
    smtp_connection.send_messages([message])

But that doesn't work because the send_messages method expects a django.core.mail.messages.EmailMessage instance, instead of the django_mailer.models.EmailMessage that is passed. And calling the backend 'smtp_connection' what is not is strange as well.

How to solve this? As it is quite nice to be able to test mail sending in development...

Hi Chris,

I have a need for sending HTML emails with bcc and I was just wondering if you're open to accepting patches for either or both features.

My current plan is to make the send_mail command accept extra arguments (html_message and bcc).

For HTML content, this is relatively easy, I just need to tweak the send_mail command slightly to use EmailMultiAlternatives rather than the EmailMessage[1].

The bcc support would require an additional field to the Message model.

Regards,
Selwin

[1]https://docs.djangoproject.com/en/dev/topics/email/

settings.py:

EMAIL_BACKEND = 'django_mailer.smtp_queue.EmailBackend'

in code somewere:

except Exception, e:
    logger.error('discount_cards.views.register -> new_user.save -> %s' % e.message)

raise BadHeaderError: Header values can't contain newlines

truncated traceback:

File "/u01/funvit/nginx/env/lib/python2.6/site-packages/django/utils/log.py", line 101, in emit
  html_message=html_message)

File "/u01/funvit/nginx/env/lib/python2.6/site-packages/django/core/mail/__init__.py", line 96, in mail_admins
  mail.send(fail_silently=fail_silently)

File "/u01/funvit/nginx/env/lib/python2.6/site-packages/django/core/mail/message.py", line 251, in send
  return self.get_connection(fail_silently).send_messages([self])

File "/u01/funvit/nginx/env/src/django-mailer/django_mailer/smtp_queue.py", line 31, in send_messages
  queue_email_message(email_message)

File "/u01/funvit/nginx/env/src/django-mailer/django_mailer/__init__.py", line 122, in queue_email_message
  encoded_message=email_message.message().as_string())

File "/u01/funvit/nginx/env/lib/python2.6/site-packages/django/core/mail/message.py", line 219, in message
  msg['Subject'] = self.subject

File "/u01/funvit/nginx/env/lib/python2.6/site-packages/django/core/mail/message.py", line 127, in __setitem__
  name, val = forbid_multi_line_headers(name, val, self.encoding)

File "/u01/funvit/nginx/env/lib/python2.6/site-packages/django/core/mail/message.py", line 87, in forbid_multi_line_headers
  raise BadHeaderError("Header values can't contain newlines (got %r for header %r)" % (val, name))

BadHeaderError: Header values can't contain newlines (got u'[Django] ERROR: discount_cards.views.register -> new_user.save -> value too long for type character varying(30)\n' for header 'Subject')

im currently testing dirty patch via adding at start of def queue_email_message

# file: __init__.py

email_message.subject = email_message.subject.replace(u'\n', u' ')

Running the retry_deferred command causes the following traceback:

Traceback (most recent call last):
  File "manage.py", line 11, in 
    execute_manager(settings)
  File "/home/zerobug/webapps/fbs/lib/python2.6/django/core/management/__init__.py", line 362, in execute_manager
    utility.execute()
  File "/home/zerobug/webapps/fbs/lib/python2.6/django/core/management/__init__.py", line 303, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/zerobug/webapps/fbs/lib/python2.6/django/core/management/base.py", line 195, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/zerobug/webapps/fbs/lib/python2.6/django/core/management/base.py", line 222, in execute
    output = self.handle(*args, **options)
  File "/home/zerobug/webapps/fbs/lib/python2.6/django/core/management/base.py", line 351, in handle
    return self.handle_noargs(**options)
  File "/home/zerobug/webapps/fbs/fbs/../fbs/django_mailer/management/commands/retry_deferred.py", line 11, in handle_noargs
    count = models.QueuedMessage.objects.retry_deferred()
  File "/home/zerobug/webapps/fbs/fbs/django_mailer/managers.py", line 53, in retry_deferred
    self.deferred().update(**update_kwargs)
  File "/home/zerobug/webapps/fbs/lib/python2.6/django/db/models/query.py", line 413, in update
    query.add_update_values(kwargs)
  File "/home/zerobug/webapps/fbs/lib/python2.6/django/db/models/sql/subqueries.py", line 232, in add_update_values
    return self.add_update_fields(values_seq)
  File "/home/zerobug/webapps/fbs/lib/python2.6/django/db/models/sql/subqueries.py", line 245, in add_update_fields
    val = field.get_db_prep_save(val)
  File "/home/zerobug/webapps/fbs/lib/python2.6/django/db/models/fields/__init__.py", line 192, in get_db_prep_save
    return self.get_db_prep_value(value)
  File "/home/zerobug/webapps/fbs/lib/python2.6/django/db/models/fields/__init__.py", line 569, in get_db_prep_value
    return connection.ops.value_to_db_datetime(self.to_python(value))
  File "/home/zerobug/webapps/fbs/lib/python2.6/django/db/models/fields/__init__.py", line 565, in to_python
    _('Enter a valid date/time in YYYY-MM-DD HH:MM[:ss[.uuuuuu]] format.'))
django.core.exceptions.ValidationError: Enter a valid date/time in YYYY-MM-DD HH:MM[:ss[.uuuuuu]] format.

Environment 1:
Win XP
Django 1.1.1
Python 2.6.4
PostgreSQL 8.2.7

Environment 2:
RHEL
Django 1.1.1
Python 2.6
PostgreSQL 8.2.3
Just found this - could be related: http://code.djangoproject.com/ticket/10443

Will keep digging and update with any new info.

Joe

it seems to me, there is a bug in django_mailer/engine.py at line 21:

LOCK_PATH = getattr(settings.LOCK_PATH,
                    os.path.join(tempfile.gettempdir(), 'send_mail'))

IMHO, this code must be rewritten as follow:

LOCK_PATH = getattr(settings, 'LOCK_PATH',
                    os.path.join(tempfile.gettempdir(), 'send_mail'))

When queue_email_message gets an EmailMessage with a long subject line (more than 255 chars) it tries to put all of it into Message.subject but since it has a maximum length of 255, PostgreSQL refuses to write this record.
When creating the Message you should take email_message.subject[:255] to prevent this.

Application operate with not aware times.
with USE_TZ = True django_mailer generate many "DateTimeField received a naive datetime" runtime warnings.

The Django 1.8.3 security update introduces a backport (django/django@76c526f) which casts the verbosity argument to an int.

This results in the error:

  File ".../django-mailer/django_mailer/management/commands/__init__.py", line 13, in create_handler
    handler.setLevel(LOGGING_LEVEL[verbosity])
KeyError: 1

In my fork I just needed to change LOGGING_LEVEL in https://github.com/SmileyChris/django-mailer-2/blob/master/django_mailer/management/commands/__init__.py#L4 from:

LOGGING_LEVEL = {'0': logging.ERROR, '1': logging.WARNING, '2': logging.DEBUG}

to:
LOGGING_LEVEL = {0: logging.ERROR, 1: logging.WARNING, 2: logging.DEBUG}

because I don't need to maintain backwards compatibility with earlier Django versions.

A backwards compatible fix in this source repo would need either a version check to set LOGGING_LEVEL appropriately or a try-catch in create_handler to cast if there's a KeyError.

Actually is just a question:
Why is django_mailer so slow the first time a mail is sent, then responses instantly the next times ? At least on the dev server