Java heap out of memory error about szurubooru-auto-tagger HOT 5 CLOSED

What's the size of your tagMapFile (by default tagMap.yaml)?. It's been years since I worked on this but the tag map is inefficient since it requires to be fully loaded into memory. Still, I can't imagine it being so huge to cause OOM, maybe the issue is with the yaml parser.

from szurubooru-auto-tagger.

My tagfile size is 109K and only has 4170 lines.

from szurubooru-auto-tagger.

Can you build/run from source code? I've swapped the yaml parser with a more common one but I don't really have the booru setup to test it properly.
It's possible too your tag map has some weird tag that messed up the parser.

from szurubooru-auto-tagger.

Sure. I've run it from the source code and it works great so far. I've not yet run it to completion as there are many posts that need to be tagged but I think it's safe to say the changes work as the program no longer breaks after "Reading tag map...". Thank you. :)

Szurubooru auto tagger Using config from config-szu-tag.yaml Booru connectivity looks ok Reading tag map... Obtaining tags... URL http://192.168.9.244:8090/api/posts/?offset=0&limit=100&query=auto_tagme timed out. Retrying in 3 seconds... URL http://192.168.9.244:8090/api/posts/?offset=100&limit=100&query=auto_tagme timed out. Retrying in 3 seconds... URL http://192.168.9.244:8090/api/posts/?offset=200&limit=100&query=auto_tagme timed out. Retrying in 3 seconds... URL http://192.168.9.244:8090/api/posts/?offset=300&limit=100&query=auto_tagme timed out. Retrying in 3 seconds... URL http://192.168.9.244:8090/api/posts/?offset=400&limit=100&query=auto_tagme timed out. Retrying in 3 seconds... URL http://192.168.9.244:8090/api/posts/?offset=500&limit=100&query=auto_tagme timed out. Retrying in 3 seconds... URL http://192.168.9.244:8090/api/posts/?offset=600&limit=100&query=auto_tagme timed out. Retrying in 3 seconds... URL http://192.168.9.244:8090/api/posts/?offset=700&limit=100&query=auto_tagme timed out. Retrying in 3 seconds... URL http://192.168.9.244:8090/api/posts/?offset=800&limit=100&query=auto_tagme timed out. Retrying in 3 seconds... URL http://192.168.9.244:8090/api/posts/?offset=900&limit=100&query=auto_tagme timed out. Retrying in 3 seconds... URL http://192.168.9.244:8090/api/posts/?offset=1000&limit=100&query=auto_tagme timed out. Retrying in 3 seconds... There are 1071 posts that needs to be tagged Searching IQDB match for post 1079... Found post 1079 match: https://danbooru.donmai.us/posts/2873430 Updated post 1079 source Updated post 1079 safety to Safe Found bigger version of post 1079, created comment. Updated post 1079 tags. Completed 1/1071. (rest omitted)

from szurubooru-auto-tagger.

Great, I'll upload the jar to releases. Just a heads up, I don't think you can stop it midway since that won't update notes and tag metadata. This project is in a need of a rewrite to fix all the bad design decisions. I planned to do so but ultimately decided that szurubooru doesn't fit my needs. I think I should update the README to say that this project isn't really maintained.

from szurubooru-auto-tagger.