Long documents fail about docspell HOT 9 CLOSED

from docspell.

Great, now all is fine again.

from docspell.

Oh that seems to break at the db level. Which db are you using? Postgres, for example, states to support "any length" for the datatype text. But mariadb may not….

It's also interesting that it was somehow blocking. If it finally fails (after last retry), you should be able to at least see it and add metadata manually. Since the failure happened when storing metadata, there are no suggestions or anything. I need to reproduce it to see what's going on.

from docspell.

Which db are you using?

I'm using mariadb which is Arch Linux's default implementation of MySQL.

Since the failure happened when storing metadata, there are no suggestions or anything.

I guess text search is also unavailable?

If it finally fails (after last retry), you should be able to at least see it and add metadata manually.

I could see this behavior on my second try.

from docspell.

Here is a quick workaround: To fix the data model, run this against your database (this is safe to not loose data, but no guarantees here ;)):

ALTER TABLE `attachmentmeta`
MODIFY COLUMN `content` longtext;

ALTER TABLE `attachmentmeta`
MODIFY COLUMN `nerlabels` longtext;

ALTER TABLE `attachmentmeta`
MODIFY COLUMN `itemproposals` longtext;

Then you can start reprocessing the item with (using bash):

USER=…
PASS=…
ITEMID="9sW3Qz…cQQi"

LOGIN_URL="http://localhost:7880/api/v1/open/auth/login"
REPROCESS_URL="http://localhost:7880/api/v1/sec/item/$ITEMID/reprocess"

auth_token=$(curl --fail -XPOST --silent --data-binary "{\"account\":\"$USER\", \"password\":\"$PASS\"}" "$LOGIN_URL" | jq -r .token)
curl --fail -XPOST -H "X-Docspell-Auth: $auth_token" \
    -H 'Content-Type: application/json' \
    -d '{"ids":[]}' "$REPROCESS_URL"

Where you need to specify account, password and the item id that contains the files to reprocess.

from docspell.

Longtext worked :)

I've did a simple reupload to scan again, but there's still the same document somewhere invisible blocking so I had to disable the duplicate check.

from docspell.

Yes, this is another bug that is being fixed with this one …

The problem is that the failing item is not updated correctly and so the processing task thinks, it already exists. It is an item in state premature or processing and is so not visible in the gui. You could set all to created with an sql update statement (from off my head: update item set state = 'created'). This should make the item visible in the ui (if it is this problem :)).

from docspell.

Yes, now I was able to delete the failed one and the February one. Sadly now all documents are displayed as new…

Interestingly it also showed me a third document which I haven't seen before, also no labels are suggested so I guess it has also failed before.
A reupload shows me execution failed. Retrying later.: Invalid date 'February 29' as '2005' is not a leap year, so yes, it's like the other one.

from docspell.

Oh no, I'm so sorry - this update statement makes all items new :-/ I wrote too quickly without thinking a lot…. You could make all "confirmed" again by using the same update statement with state 'confirmed'. The date bug is being fixed, too. I guess I'll make a bugfix release soon. Thank you so much for reporting all this!

from docspell.