Sync does not continue where it left off about gmvault HOT 4 CLOSED

Quick update, it always seems to crash out at email num 6228. Here is the trace:

Process email num 6228 (imap_id:12157) from 2006-11.
error message = System Error. traceback:Traceback (most recent call last):
  File "/Users/me/bin/gmvault-v1.0-beta/lib/gmv/imap_utils.py", line 109, in wrapper
    return the_func(*args, **kwargs)
  File "/Users/me/bin/gmvault-v1.0-beta/lib/gmv/imap_utils.py", line 316, in fetch
    return self.server.fetch(a_ids, a_attributes)
  File "/Users/me/bin/gmvault-v1.0-beta/lib/python-lib/lib/python2.7/site-packages/imapclient/imapclient.py", line 673, in fetch
    typ, data = self._imap._command_complete('FETCH', tag)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/imaplib.py", line 903, in _command_complete
    self._check_bye()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/imaplib.py", line 815, in _check_bye
    raise self.abort(bye[-1])
abort: System Error

from gmvault.

Thanks for reporting this potential problem.

First be also sure to not have the 1000 emails limit by folders activated in Gmail Setting -> IMAP (see http://flic.kr/p/bUwNK9).

I would like to understand why it fails always on the same email.
Could you please run it in debug mode (--debug) as I would like to see the error.
Gmail in the past has corrupted some of the emails. I had this issue where some of my emails cannot be read anymore.
Gmvault flags these emails and they are ignored. You might be in a case that I did not encountered.

Now Gmvault does not re-download the email if it has already been stored locally. It only gets the metadata information (gm id, imap id, dates, ...) so it should be much shorter the second time.

To even shorter the backup process I have added a restart mode in sync like there was one in restore. It will be available in the next version of Gmvault (available in one to two weeks).

from gmvault.

The trace above (ending with abort: System Error) is what I see repeated 5 or 6 times in a row. It's the same thing reported here #4.

I'm rerunning now and will look more closely at that last email that fails.

from gmvault.

Unfortunately I'm unable to find the exact email that is failing. gmvault throws the exception and does not write the message / metadata to the db.

I took a quick look in the code but it was non-obvious where I could inject a variable dump to capture the email after it was fetched from google but before the exception occurred.

If you have an idea of where to inject an if nb_emails_processed == 6228: ... please let me know. Happy to test it.

from gmvault.