Code Monkey home page Code Monkey logo

e621-batch-reverse-search's Introduction

e621 Batch Reverse Search

A script that iterates over a directory and performs a reverse search for every image using remote services.

Used services:

Requirements:

Windows

Package comes with compiled PHP 7.2 library (x86 Non Thread Safe) and all required extensions.

You will need Visual C++ 2015 Redistributable (x86) for it to run - https://www.microsoft.com/en-us/download/details.aspx?id=52685

Linux

Install PHP library (>=7.2), cURL, GD and zip extensions - sudo apt-get install php-cli php-curl php-gd php-zip

Warning about PHP version

While the script will still work with PHP >=5.6 it won't be able to work with some special characters that can be in the file names, PHP 7.2+ is recommended.

Usage:

  • Put images into 'images' folder
  • Run it with 'run.bat' ('run.sh' on linux)
  • Wait, this can take a very long time, depending on how many images you got there...
  • Matched images will be moved to 'found' folder, not matched images will be moved to 'not found' folder
  • List file 'links.html' (in 'found' folder) will be created containing all the links, open it with a web browser

Logging in (for e621 IQDB search):

  • Rename config.cfg.example to config.cfg
  • Fill your login details inside it:
    • E621_LOGIN - your e621 username
    • E621_API_KEY - obtained from e621 -> Account -> Manage API Access

Advanced

  • You can pass any directory as an argument to the run script (on Windows you can move a directory over run.bat)
  • Rename config.cfg.example to config.cfg to make the script use it, configure it how you want

Contributing

See CONTRIBUTING for more information.

License

See LICENSE.

e621-batch-reverse-search's People

Contributors

jacklul avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

millcore

e621-batch-reverse-search's Issues

fatal libpng error: Read Error: truncated data

PHP Fatal error: imagecreatefrompng(): fatal libpng error: Read Error: truncated data in phar:///media/pictures/gayfurries/e6brs/e6BRS.phar/app.php on line 743

This error appeared after picture 541/1671 of a Tumblr rip I did on my blog. The picture that it threw this error on is in fact truncated, hence the error message, but isn't there a fail-safe that can be implicated to work around this? Had I not been at my computer while this was running, I would've come back hours later to realize that the tool had thrown an error message hours before.

Burst limit for SauceNAO, but not for e621 IQDB

There seems to be a function in this script that automatically waits out the 30 second burst limit. However, there doesn't seem to be one for the throttle that e621 implemented for their self-hosted IQDB recently. Would it be possible to add a "sleep" command of sorts between e621 IQDB requests? I keep getting "Throttled: Too many requests".

What to do with the links and post deleted by author.

This is mostly a question, what can I do know with the links? And is there a way to filter those that were deleted by the author?

I think this app has so much potential, yet I'm practically stuck with a bunch of links that I can't find the way to bulk download them on their original sizes.

Maybe is there something I'm missing?

error: Authentication required, check configuration!

I am encountering an issue when running the program and obtaining the following log:

[1/15] Searching '004.jpeg':
Trying md5 sum... no matching posts found!
Trying reverse search #1 (e621.net/iqdb_queries)... error: Authentication required, check configuration!

I have confirmed that my configuration is set up correctly:

DEBUG=true
LOGGING=true
E621_LOGIN='MyUsername'
E621_API_KEY='MyE621ApiKey'
SAUCENAO_API_KEY='MySauceNaoApiKey'

I was wondering if you could assist me in understanding why I am receiving this "Authentication required" error.

"Daily Search Limit Exceeded" on SauceNAO

Daily Search Limit Exceeded.
***.***.***.***, your IP has exceeded the unregistered user's daily limit of 150 searches.

If the SauceNAO option is enabled and the count of images that is displayed after "Scanning for images..." is over 150, it should display some kind of message that users can disable, warning them about the 150 picture daily limit.

Windows: Looking for a file 'e621BRS.phar'

On windows 10, with php5.6 x86 NTS, running run.bat whether from src\ or parent directory, results in this error:
Could not open input file: C:\Users\hamstap85\Desktop\e621-Batch-Reverse-Search-master\src\/e621BRS.phar

Edit: on Linux as well. Is there some build that I'm missing?

PHP Parse error: syntax error

I'm not exactly a PHP programmer, but I do come across this error every time I try to use the run.sh script:
PHP Parse error: syntax error, unexpected '.', expecting ',' or ';' in phar:///mnt/c/Users/Faux-Pa/Downloads/0/e621BRS.phar/app.php on line 60

Crash with OOM error on large image

I have one 2.1 MB image that causes this tool to consistently crash with:

[12/168] Searching '1575180225943.jpg':
 Trying reverse search #1 (e621.net/iqdb_queries)...PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 24576 bytes) in phar:///***/e621-Batch-Reverse-Search/build/e621BRS.phar/app.php on line 1375

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 24576 bytes) in phar:///***/e621-Batch-Reverse-Search/build/e621BRS.phar/app.php on line 1375
Press ENTER key to continue...

It'd be nice if the tool would at the least gracefully handled this case and ignored the image with a message that it was too large. It took me three runs before I figured out what the cause was.

App doesn't control search rate on SauceNAO

Got to try the app after last fix and I noticed that the app throws an unhandled error on each iteration when it reaches SauceNAO's limits (Running with api key, basic account, 6 per 30 sec, 200 per 24h)

I checked your code and seems that JSON returned by SauceNAO changed a bit thus causing your app to completely ignore intended delays.

if and elseif ~ line 1009 return false cuz current JSON returned by the site does not include "short_remaining". or "long_remaining".

Reaching 6 per 30s limit
1

Reaching 200 per 24h limit
2

JSON returned after repeatedly reaching 6 per 30s limit
3

SauceNao

I have inserted the SauceNAO Api key

image

Use of more databases

Is there any plans to implement another reverse search engine, like iqdb for example?

Since SauceNAO has a relatively small limit and the other engine usually isn't that useful.

Error trying SauceNao

While testing some images, I found two interesting things.

First is that you dind't consider when the extension is on capitals: eg: ".JPG".

And the second I couldn't know what was it about. I'm no good with web things, sorry. But there is the debug log.

debug_error.txt

The interesting part is that only happens with some images and when going through saucenao.

Duplicate printing of found match links

1.1.9

When running the batch process, the output prints all the found links so far, including previous images in the batch process. The found/links.html is fine, it's only showing links associated with each image.

It also prints empty links for what I presume are unmatched files, they're not showing up in found/

SauceNao - 307 Temporary Redirect

App throws an unhandled error on each iteration due to SauceNao returning 307. Happens with default settings and with personal SauceNao api key.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.