kelciour / batch-download-pictures-from-google-images Goto Github PK

While adding 2000 cloze cards, I get stuck near the middle. My anki windows are unclickable and need to force quit afterwards. I added 2 newlines to append option if that matters. Also oddly it says "sleeping for 30,60,90..... seconds" after starting. My settings:

Anki 23.12.1 (1a1d4d54) (ao)
Python 3.9.15 Qt 6.5.3 PyQt 6.5.3
Platform: macOS-13.3.1-arm64-arm-64bit

Traceback (most recent call last):
File "/Users/DaniDevito/Library/Application Support/Anki2/addons21/561924305/init.py", line 434, in
a.triggered.connect(lambda _, b=browser: onAddImages(b))
File "/Users/DaniDevito/Library/Application Support/Anki2/addons21/561924305/init.py", line 427, in onAddImages
updateNotes(browser, nids)
File "/Users/DaniDevito/Library/Application Support/Anki2/addons21/561924305/init.py", line 369, in updateNotes
r.raise_for_status()
File "requests.models", line 1021, in raise_for_status
requests.exceptions.HTTPError: 429 Client Error: Too Many Requests for url: https://www.google.com/sorry/index?continue=https://www.google.com/search%3Fq%3DParts%2Bof%2Ba%2Bhabit:%26safe%3Dactive%26ie%3Dutf8%26oe%3Dutf8%26ucbcb%3D1%26udm%3D2&q=EhAmA4ABE_BRgKn4tl7-vdYAGI3qsLAGIjAUqQVLxmxDiTTSL_1LFTXRkjgc6-yIwXXSP2AhZgZo7SWXubAU2xR0KOWAY2GrD0syBWpjbmRyWgFD

===Add-ons (active)===
(add-on provided name [Add-on folder, installed at, version, is config changed])
AwesomeTTS - Add speech to your flashcards ['1436550454', 2024-02-08T22:42, 'None', '']
Batch Download Pictures From Google Images ['561924305', 2024-02-21T09:38, 'None', mod]
Batch Editing ['291119185', 2023-10-25T17:38, 'None', '']

===IDs of active AnkiWeb add-ons===
1436550454 291119185 561924305

===Add-ons (inactive)===
(add-on provided name [Add-on folder, installed at, version, is config changed])
HyperTTS - Add speech to your flashcards ['111623432', 2024-02-10T04:02, 'None', mod]

Anki Version 2.1.15 (442df9d6), Qt 5.15.2 PyQt 5.15.6

I use this version of Anki because some addons don't work in higher versions. But addon batch-download-pictures-from-google-images worked fine early.

When I run now I don't get error messages, it seems ok, but field don't filled. See screenshots, maybe I did mistake somewhere in settings.

I go to steps:

The add-on was updated recently and it seems to work for me. If it doesn't work, maybe run this minimal example in the debug console - https://gist.github.com/kelciour/86bfa1bbcc256fbaa780ab725da14714 - copy the output and create a new issue at https://github.com/kelciour/batch-download-pictures-from-google-images/issues

https://docs.ankiweb.net/misc.html?highlight=debug#debug-console

debug-console.txt

Hello since the latest Anki update, the addon no longer works. Could you possibly update it?

Dear author,

I'm very keen on knowing how I can get your presumably awesome add-on, so I can easily add pictures to my Anki decks.
However, I get an error code when I use the add-on code or when I load the downloaded file.
I hope to hear back from you soon.

Best regards,

Ernst

Hi. I selected "Source Field" and selected "Target Field" instead of "". Then I hit "Start". A message appeared: "Processed 1 note", but I have no images at all. I have Anki 2.1.35 Mac OS

import re
... import json
... import requests
...
... from bs4 import BeautifulSoup
...
... headers = {
... "User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.67 Safari/537.36"
... }
...
... query = "cat"
...
... r = requests.get("https://www.google.com/search?tbm=isch&q={}&safe=active".format(query), headers=headers, timeout=15)
...
... print('-----------------------')
... print(r.status_code)
... print('-----------------------')
...
... html = r.text
...
... print('-----------------------')
... print(html)
... print('-----------------------')
...
... soup = BeautifulSoup(html, "html.parser")
... rg_meta = soup.find_all("div", {"class": "rg_meta"})
... metadata = [json.loads(e.text) for e in rg_meta]
... results = [d["ou"] for d in metadata]
...
... if not results:
... regex = re.escape("AF_initDataCallback({")
... regex += r'[^<]?data:[^<]?' + r'([[^<]+])'
...
... for txt in re.findall(regex, html):
... data = json.loads(txt)
...
... try:
... for d in data[31][0][12][2]:
... try:
... results.append(d[1][3][0])
... except Exception as e:
... pass
... except Exception as e:
... pass
...
... if not results:
... try:
... for d in data[56][1][0][0][1][0]:
... try:
... d = d[0][0]["444383007"]
... results.append(d[1][3][0])
... except:
... pass
... except:
... pass
...
... print('-----------------------')
... print(' IMAGES ')
... print('-----------------------')
... print('\n\n'.join(results))
... print('-----------------------')
... print('Found Images:', len(results))

200

ALL

IMAGES

VIDEOS

BOOKS

Domestic cat www.nationalgeographic.com	Tabby cat - Wikipedia en.wikipedia.org	Cat | Breeds & Facts |... www.britannica.com	Cat - Wikipedia en.wikipedia.org
Cat | Breeds & Facts |... www.britannica.com	Thinking of getting a cat?... icatcare.org	Tabby Cat: Breed Profile,... www.thesprucepets.com	Cat Clip Art Images - Free... www.freepik.com
Kitten - Wikipedia en.wikipedia.org	cat - Wiktionary en.wiktionary.org	How the Cat Gets Its... www.nytimes.com	9 Signs Your Cat Loves You... www.oprahdaily.com
Baby Cats - Cute and Funny... www.youtube.com	International Cat Day 2021:... www.ndtv.com	25 Best Cat Instagram... www.goodhousekeeping.com	Cat Images - Free Download... www.freepik.com
International Cat Day... www.daysoftheyear.com	Home - Kit Cat... kitcat.com.sg	Are Cats Really Aloof? :... www.npr.org	What does your cat mean by... www.theguardian.com

   IMAGES          

Found Images: 0

Error
An error occurred. Please start Anki while holding down the shift key, which will temporarily disable the add-ons you have installed.
If the issue only occurs when add-ons are enabled, please use the Tools > Add-ons menu item to disable some add-ons and restart Anki, repeating until you discover the add-on that is causing the problem.
When you've discovered the add-on that is causing the problem, please report the issue on the add-on support site.
Debug info:
Anki 2.1.22 (0ecc189a) Python 3.8.0 Qt 5.14.1 PyQt 5.14.1
Platform: Windows 10
Flags: frz=True ao=True sv=1
Add-ons, last update check: 2020-04-06 10:05:45

C:\Users\xyz\AppData\Roaming\Anki2\addons21\batch_download_pictures_from_google_images\vendor\PIL\TiffImagePlugin.py:788: UserWarning: Corrupt EXIF data. Expecting to read 4 bytes but only got 0.
warnings.warn(str(msg))

Hello, the add-on crashes on my anki desktop version 2.1.54. I attached a result of the debug consode below.

Thanks!

debug output.txt

when I run the add-on, my computer will freeze and then anki will report it encounters into a problem. I have no idea what to do. I had just updated my anki and the add-on to the latest version, but the problem still occurs.

I copy a dubug info if it helps.
Anki 23.12.1 (1a1d4d54) (ao)
Python 3.9.15 Qt 6.6.1 PyQt 6.6.1
Platform: Windows-10-10.0.22621

Traceback (most recent call last):
File "urllib3.connection", line 203, in _new_conn
File "urllib3.util.connection", line 85, in create_connection
File "urllib3.util.connection", line 73, in create_connection
socket.timeout: timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "urllib3.connectionpool", line 790, in urlopen
File "urllib3.connectionpool", line 491, in _make_request
File "urllib3.connectionpool", line 467, in _make_request
File "urllib3.connectionpool", line 1096, in _validate_conn
File "urllib3.connection", line 611, in connect
File "urllib3.connection", line 212, in _new_conn
urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x00000224D14AE160>, 'Connection to www.google.com timed out. (connect timeout=15)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "requests.adapters", line 486, in send
File "urllib3.connectionpool", line 844, in urlopen
File "urllib3.util.retry", line 515, in increment
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.google.com', port=443): Max retries exceeded with url: /search?tbm=isch&q=cat&safe=active&ie=utf8&oe=utf8&ucbcb=1 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x00000224D14AE160>, 'Connection to www.google.com timed out. (connect timeout=15)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\Lucy\AppData\Roaming\Anki2\addons21\561924305_init_.py", line 434, in
a.triggered.connect(lambda , b=browser: onAddImages(b))
File "C:\Users\Lucy\AppData\Roaming\Anki2\addons21\561924305_init.py", line 427, in onAddImages
updateNotes(browser, nids)
File "C:\Users\Lucy\AppData\Roaming\Anki2\addons21\561924305_init_.py", line 368, in updateNotes
r = requests.get("https://www.google.com/search?tbm=isch&q={}&safe=active&ie=utf8&oe=utf8&ucbcb=1".format(query), headers=headers, cookies={"CONSENT":"YES+"}, timeout=15)
File "requests.api", line 73, in get
File "requests.api", line 59, in request
File "requests.sessions", line 589, in request
File "requests.sessions", line 703, in send
File "requests.adapters", line 507, in send
requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='www.google.com', port=443): Max retries exceeded with url: /search?tbm=isch&q=cat&safe=active&ie=utf8&oe=utf8&ucbcb=1 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x00000224D14AE160>, 'Connection to www.google.com timed out. (connect timeout=15)'))

===Add-ons (active)===
(add-on provided name [Add-on folder, installed at, version, is config changed])
Advanced Review Bottom Bar ['1136455830', 2024-02-05T02:38, 'None', mod]
AnkiConnect ['2055492159', 2024-02-27T12:37, 'None', '']
AwesomeTTS - Add speech to your flashcards ['1436550454', 2024-02-09T14:42, 'None', '']
Batch Download Pictures From Google Images ['561924305', 2024-02-22T01:38, 'None', mod]
Enhance main window ['877182321', 2023-10-23T04:28, 'None', '']
Remove Duplicate Cards ['95590040', 2021-05-06T02:12, 'None', '']
Review Heatmap ['1771074083', 2022-06-30T09:43, 'None', '']

===IDs of active AnkiWeb add-ons===
1136455830 1436550454 1771074083 2055492159 561924305 877182321 95590040

===Add-ons (inactive)===
(add-on provided name [Add-on folder, installed at, version, is config changed])
Fast Word Query Multi-threaded queries for words from local or web dictionaries ['1807206748', 2019-06-23T11:10, 'None', '']
Word Query ['775418273', 2018-01-01T01:07, 'None', '']

I wish you would spare some time to look at my question, if your add-on could work, it would be a great help to me. thank you very much!
(p.s: I'm in China, if my location is of any important.)

Hi there,

I've tried asking GPT (my guide to explaining system errors like i'm a four year old)... but I can't get another further with this error:

Here's a breakdown of the error message:

The add-on attempts to import the PIL module in its code.
Within the PIL module, there's an attempt to import the _imaging module, which is a part of PIL's internal implementation.
However, when trying to load the _imaging module, there's an ImportError because it cannot find the required library libXdmcp.6.dylib.
This error suggests that the PIL module is missing a necessary library (libXdmcp.6.dylib), which is needed for its functionality. The add-on appears to be referencing this library within its vendor directory, but there's an issue with loading it.

I thought perhaps it wasn't running because my macbook runs Python 2.7 so I downloaded 3.9.2rc1 (I'm running OSX 10.13)
But I'm up against more my own ignorance again... once I have 3.9 installed, don't I need to "configure your system to use Python 3.9.2 as the default version" ...?

My Anki version is
Version ⁨2.1.62 (77dd1dc1)⁩
Python 3.9.15 Qt 5.14.2 PyQt 5.14.1 (which is why I thought maybe the Add-on isn't working because it needs newer Python...)

Please advise, I'd love to get your Add-on working, and would definitely donate coffee funds for the assistance!

regards
Dan

The addon doesn't work and I attached a file where one can see the result of the debug-console.
a.txt

The Qt5 compatibility code is now off by default, which may break some add-ons that haven't been updated to support Qt6 in the last 2 years. If you find any have broken, please report the issues to the add-on author, and start Anki after setting the env var ENABLE_QT5_COMPAT to 1 to temporarily restore the compatibility code.

Error
Un complemento que has instalado ha fallado al cargarse. Si los problemas persisten, por favor ve a Herramientas> Menú de complementos o deshabilita este complemento.

Mientras cargando '⁨Batch Download Pictures From Google Images⁩':
⁨Traceback (most recent call last):
  File "aqt.addons", line 239, in loadAddons
  File "C:\Users\Khonkhortisan\AppData\Roaming\Anki2\addons21\561924305\__init__.py", line 29, in <module>
    from .designer.main import Ui_Dialog
  File "C:\Users\Khonkhortisan\AppData\Roaming\Anki2\addons21\561924305\designer\main.py", line 11, in <module>
    from PyQt5 import QtCore, QtGui, QtWidgets
ModuleNotFoundError: No module named 'PyQt5'

fix:
main.py

#from PyQt5 import QtCore, QtGui, QtWidgets
try:
    from PyQt6 import QtCore, QtGui, QtWidgets
except ImportError:
    from PyQt5 import QtCore, QtGui, QtWidgets

replace every QtWidgets.QSizePolicy. with QtWidgets.QSizePolicy.Policy.
replace every setFrameShape(QtWidgets.QFrame.HLine) with setFrameShape(QtWidgets.QFrame.Shape.HLine)
replace every setFrameShadow(QtWidgets.QFrame.Sunken) with setFrameShadow(QtWidgets.QFrame.Shadow.Sunken)

561924305\__init__.py

replace every Qt.Align with Qt.AlignmentFlag.Align

    #if not d.exec_():
    if not d.exec():
        return

Version ⁨2.1.65 (aa9a734f)⁩
Python 3.9.15 Qt 6.5.0 PyQt 6.5.0
Batch Download Pictures From Google Images
8.29MB. Updated 2023-08-06.

'/Users/unickq/Library/Application Support/Anki2/addons21/561924305/vendor/PIL/_imaging.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))

Before, there was an error saying PIP image didn't match the architecture. I installed it manually through pip install Pillow==9.4.0 and copied it to /Users/unickq/Library/Application Support/Anki2/addons21/561924305/vendor/PIL

Still see cannot import name '_imaging' from 'PIL' , any ideas what can be done?

Error
An error occurred. Please start Anki while holding down the shift key, which will temporarily disable the add-ons you have installed.
If the issue only occurs when add-ons are enabled, please use the Tools > Add-ons menu item to disable some add-ons and restart Anki, repeating until you discover the add-on that is causing the problem.
When you've discovered the add-on that is causing the problem, please report the issue to the add-on author.
Debug info:
Anki 2.1.65 (aa9a734f) Python 3.9.15 Qt 6.5.0 PyQt 6.5.0
Platform: macOS-13.5-arm64-arm-64bit
Flags: frz=True ao=True sv=2
Add-ons, last update check: 2023-08-16 11:36:19
Add-ons possibly involved: ⁨Batch Download Pictures From Google Images⁩

Caught exception:
Traceback (most recent call last):
  File "/Users/unickq/Library/Application Support/Anki2/addons21/561924305/__init__.py", line 401, in <lambda>
    a.triggered.connect(lambda _, b=browser: onAddImages(b))
  File "/Users/unickq/Library/Application Support/Anki2/addons21/561924305/__init__.py", line 394, in onAddImages
    updateNotes(browser, nids)
  File "/Users/unickq/Library/Application Support/Anki2/addons21/561924305/__init__.py", line 47, in updateNotes
    from PIL import Image, ImageSequence, UnidentifiedImageError
  File "/Users/unickq/Library/Application Support/Anki2/addons21/561924305/vendor/PIL/Image.py", line 103, in <module>
    from . import _imaging as core
ImportError: cannot import name '_imaging' from 'PIL' (/Users/unickq/Library/Application Support/Anki2/addons21/561924305/vendor/PIL/__init__.py)

Skip to content
kelciour/debug_console.py Secret
Last active last month • Report abuse
Code
Revisions
2
Google Images Debug
debug_console.py
import re
import json
import requests

from bs4 import BeautifulSoup

headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.67 Safari/537.36"
}

query = "cat"

r = requests.get("https://www.google.com/search?tbm=isch&q={}&safe=active".format(query), headers=headers, timeout=15)

print('-----------------------')
print(r.status_code)
print('-----------------------')

html = r.text

print('-----------------------')
print(html)
print('-----------------------')

soup = BeautifulSoup(html, "html.parser")
rg_meta = soup.find_all("div", {"class": "rg_meta"})
metadata = [json.loads(e.text) for e in rg_meta]
results = [d["ou"] for d in metadata]

if not results:
regex = re.escape("AF_initDataCallback({")
regex += r'[^<]?data:[^<]?' + r'([[^<]+])'

for txt in re.findall(regex, html):
    data = json.loads(txt)

    try:
        for d in data[31][0][12][2]:
            try:
                results.append(d[1][3][0])
            except Exception as e:
                pass
    except Exception as e:
        pass

if not results:
    try:
        for d in data[56][1][0][0][1][0]:
            try:
                d = d[0][0]["444383007"]
                results.append(d[1][3][0])
            except:
                pass
    except:
        pass

print('-----------------------')
print(' IMAGES ')
print('-----------------------')
print('\n\n'.join(results))
print('-----------------------')
print('Found Images:', len(results))
Add a quote, <Cmd+Shift+.>
Add code, <Cmd+e>
Add a link, <Cmd+k>
Directly mention a user or team
Reference an issue or pull request
Leave a comment
Footer
© 2022 GitHub, Inc.
Footer navigation
Terms
Privacy
Security
Status
Docs
Contact GitHub
Pricing
API
Training
Blog
About

Hi, I just tried your addon and I am not getting any images. I am in the EU, if that matters.

Hope this can be fixed. Looks like an amazing tool!

    Anki 2.1.65 (aa9a734f) Python 3.9.15 Qt 6.4.3 PyQt 6.4.0
    Platform: Windows-10-10.0.22621
    Flags: frz=True ao=True sv=3

Hello, I'm using Anki 2.1.23 on macOS Catalina.

The source field has one word in it, and the destination field is empty. When I hit Start, it processes, and then no images appear in the destination field.

Do you know why this might be? I installed the add-on directly from the Ankiweb Add-ons page.

Thank you

I think google may have released a DOM change and as a result, this add on seems to be completely broken. I've debugged it a bit and it comes down to this:

Traceback (most recent call last):
  File "/Users/selliott/Library/Application Support/Anki2/addons21/561924305/__init__.py", line 242, in getImages
    for d in data[31][0][12][2]:
IndexError: list index out of range

I've also dumped the json to a file, but I'm really not sure why the [31][0][12][2] path is being used. Like I said, I'm guessing there was a google DOM update on their search results, but I'm not sure. I will link the json in a comment.

Any way to add chosen file as a link, file, etc. instead of query?

Hello,
I am using anki 2.1.66 (Win 10)
addon is the latest
User Agent configuration is set correctly
No error is coming, but also no images

Would appreciate your help.
Thank you!

Hi. I found that some of the downloaded images have names starting with an underscore _. I think this might interfere with Anki's static images. This makes Anki not delete these images when deleting unused media.

Maybe the addon could remove the starting underscores when importing the images?

Thanks!

Anki is not stopping the process

Why?

It is the only add on i have installed:
batch-download-pictures-from-google-images

Hi, addon "batch-download-pictures-from-google-images" in ANKI 2.1.54 not work.

Installation procedure missing.

Just wanted to suggest using Pixabay as another source for images. They are free and offer an API. :)

Hello, my question about the addon; Batch Download Pictures From Google Images

Is there any way of to do a query where i can avoid the pictures with letters?
I mean, the images, which are only phrases.

Thanks in vorehand.

Dazy.

I've loved this add-on and found it very useful, but within the last couple of days it's stopped being able to pull images. I'm not sure if the issue is on my end or not. I've tried disabling all other add-ons, playing around with different settings and reinstalling it, but nothing seems to have helped.
There's no errors or anything. It says processed, but no images are added to the cards. I'm lost, any ideas?

Anki 24.04 (429bc9e1) (ao)
Python 3.9.18 Qt 6.6.2 PyQt 6.6.1
Platform: Windows-10-10.0.22000

===Add-ons (active)===
(add-on provided name [Add-on folder, installed at, version, is config changed])
Batch Download Pictures From Google Images ['561924305', 2024-02-22T04:38, 'None', mod]

===IDs of active AnkiWeb add-ons===
561924305

===Add-ons (inactive)===
(add-on provided name [Add-on folder, installed at, version, is config changed])

Image of my set-up which used to work.

Hi, I can't add images to Anki today (it still worked normally a few days ago). I'm using Anki 2.1.49.

I have 2 errors:

1. There is no image in the picture field after I add the image
2. Endless "Processing..."

I attached the output and image below. Please help.
Thank you!

output.txt