Batch download does not recognize .mp4 as having a size about flickr-download HOT 19 CLOSED

Feh. There's some other logic in there for videos:
https://github.com/beaufour/flickr-download/blob/master/flickr_download/flick_download.py#L132
maybe we just need to make sure that detection works for this type too? Or maybe it does detect it, but it's not HD MP4 that is the magic size for this type of video. Neither API or my code is particularly pretty wrt this handling...

from flickr-download.

@lazzarello do you have a (public) example of this, so I test and fix this?

from flickr-download.

Seems like Site MP4 works for these old videos – or at least for a couple of ones I've tested.

from flickr-download.

This should be fixed on master

from flickr-download.

Would it be helpful if I found that .mp4? I don't have it off hand since I was doing a batch. I can run it again if needed.

from flickr-download.

I searched for videos uploaded between 2008 and 2009 and found a few that I could test on :) I also added a --download_photo feature, so you can easily test just one photo (or video) which makes things a lot easier.

If you do happen to have that .mp4, please do test if I fixed it though

from flickr-download.

Hi, I'm finding this tool to be very useful, so first off, thanks! I hit an issue with downloading videos.

I ran this command (Nov 2022):
flickr_download -k -s --download_user --user_auth --cach api_cache --metadata_store

And it downloads photos just fine (although it does timeout a lot. When the tool comes across a .mp4, this is the error that pops up:

ERROR:root:Flickr error saving photo: The requested size is not available

The comments above say it's fixed but I wanted to see if this was still an issue, since I'm facing it.

Thanks in advance!
~Mike

from flickr-download.

from flickr-download.

Hi Beaufour, can you try this video? https://www.flickr.com/photos/selenamichael/35144138580/
I haven't tested downloading a single video yet as the tool is running and it's on a roll so I don't want to stop it. :)

from flickr-download.

:) Perfect. That gives me something to debug. Thanks!

from flickr-download.

Ok. It seems like Flickr has changed some things since 2016, and I've now also fixed the underlying API, so things should be good again on master. I just released a new version (v0.3.2) of the tool which should fix this.

from flickr-download.

Thanks beaufour. I'll try it out.
Can I use the same same command as before, and the tool will only download the .mp4 files? (I used the --metadata_store option before as well).

flickr_download -k -s --download_user --user_auth --cach api_cache --metadata_store

from flickr-download.

from flickr-download.

Hi Beaufour,

Thanks, it worked (I ran a pip install flickr_download==0.3.2).

For one of the albums that had .mp4, the video files were downloaded successfully.

On another album, I hit an issue. I tried several times and it seems to stop at the same point. This is the printout in the terminal (in Pycharm) when there's an error. Does this give enough information (FYI it's an album with many iphone .mp4s which I believe are "live" photos. I never knew they were actually just .mp4s.

Thanks in advance,
Mike

Traceback (most recent call last):
File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_api\objects.py", line 118, in getattr
return self.dict[name]
~~~~~~~~~~~~~^^^^^^
KeyError: 'index'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in run_code
File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Scripts\flickr_download.exe_main.py", line 7, in
File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_download\flick_download.py", line 592, in main
download_user(
File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_download\flick_download.py", line 337, in download_user
download_set(photoset.id, get_filename, size_label, skip_download, save_json)
File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_download\flick_download.py", line 125, in download_set
download_list(
File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_download\flick_download.py", line 174, in download_list
do_download_photo(
File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_download\flick_download.py", line 221, in do_download_photo
fname = get_full_path(dirname, get_filename(pset, photo, suffix))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_download\filename_handlers.py", line 84, in title_increment
extra = f"({photo.index})"
^^^^^^^^^^^
File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_api\objects.py", line 120, in getattr
raise AttributeError(
AttributeError: 'Photo' object has no attribute 'index'

from flickr-download.

That's because I managed to introduce a bug in the last release 🤦 . Try 0.3.3

from flickr-download.

Thanks @beaufour, 0.3.3 fixed most issues. For one album I am still seeing issues with downloading some videos. It's specific to this album but I believe it's because the videos are created/generated from either the LINE app or the WhatsApp app (these are videos that were taken on a phone and then I had transferred them to my computer and then uploaded to Flickr). I've made one such video public: https://www.flickr.com/photos/selenamichael/24170412187/ so you can try out any debugging.

The command I use to try and download this was the batch download where I would get this error:
"ERROR:root:IO error saving photo: HTTP Error 404: Not Found"

and when I try an individual download of the video I get:

flickr_download -k de061931c871cfe7ffcab56eb6475f91 -s -t -i https://www.flickr.com/photos/selen
amichael/24170412187/
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in run_code
File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Scripts\flickr_download.exe_main.py", line 7, in
File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_download\flick_download.py", line 544, in main
download_photo(
File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_download\flick_download.py", line 304, in download_photo
do_download_photo(".", None, photo, size_label, suffix, get_filename, skip_download, save_json)
File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_download\flick_download.py", line 224, in do_download_photo
fname = get_full_path(dirname, get_filename(pset, photo, suffix))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_download\filename_handlers.py", line 91, in title_increment
if not photo.title:
^^^^^^^^^^^
File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_api\objects.py", line 116, in getattr
self.load()
File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_api\objects.py", line 176, in load
props = self.getInfo()
^^^^^^^^^^^^^^
File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_api\reflection.py", line 256, in call
r = method_call.call_api(method=flickr_method, **method_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_api\method_call.py", line 150, in call_api
raise FlickrAPIError(resp["code"], resp["message"])
flickr_api.flickrerrors.FlickrAPIError: 1 : Photo not found

from flickr-download.

There are some videos the Flickr API simply doesn't return any video URLs for. It only returns web pages with a player. You can see for yourself in the API response here (put your own API key in):

https://www.flickr.com/services/rest/?method=flickr.photos.getSizes&photo_id=24170412187&format=rest&api_key=INSERT_API_KEY

If you look for the entries with media=url you can see that they all have a source that is a player, not a video file...

from flickr-download.

Thanks @beaufour, I was able to individually download those videos with this issue. I really appreciate your time and support.

from flickr-download.

No problem @emikelai ! Happy to hear.

from flickr-download.