beaufour / flickr-download Goto Github PK

Hello,

I tried the option -l on a user in order to retake a download of all datasets broken (due to errornetwork) and made a list of all a user's set.
I extracted the list but opening it I see the max entries is 500 when this user has clearly more

In this case it's this user: https://www.flickr.com/photos/biodivlibrary/

Is it a normal limitation due to API or something could be done retaining let's say the last ID of set and launching again the query?

I did it as my downloading of sets usually stops around 450, I don't know if it can be related but in this case the result of the command is the one expected.

Thanks for everything : )))

Remember when Flickr launched short videos? Before Vine and Snapchat? No, me either but they did and it seems I uploaded some! This causes an exception due to this file type not having a size, or so it seems from the error output

Saving: lazzarello/5094118879.mp4
Traceback (most recent call last):
  File "/usr/local/bin/flickr_download", line 9, in <module>
    load_entry_point('flickr-download==0.2.15', 'console_scripts', 'flickr_download')()
  File "/usr/local/lib/python2.7/site-packages/flickr_download-0.2.15-py2.7.egg/flickr_download/flick_download.py", line 267, in main
    download_user_photos(args.download_user_photos, get_filename, args.quality)
  File "/usr/local/lib/python2.7/site-packages/flickr_download-0.2.15-py2.7.egg/flickr_download/flick_download.py", line 178, in download_user_photos
    download_list(user, username, get_filename, size_label)
  File "/usr/local/lib/python2.7/site-packages/flickr_download-0.2.15-py2.7.egg/flickr_download/flick_download.py", line 146, in download_list
    photo.save(fname, photo_size_label)
  File "/usr/local/lib/python2.7/site-packages/flickr_api/objects.py", line 1312, in save
    r = urllib2.urlopen(self.getPhotoFile(size_label))
  File "/usr/local/lib/python2.7/site-packages/flickr_api/objects.py", line 1290, in getPhotoFile
    raise FlickrError("The requested size is not available")
flickr_api.flickrerrors.FlickrError: The requested size is not available

iter() returned non-iterator of type 'java.util.ArrayList$Itr'

The current way of running the tests are deprecated, and we should switch to using tox

Hi,

Is a wiki exist for script installation?

Flickr allows you to not put a photo in a set. I think this is a recent change within the last year, which may explain why this script is not designed for this use-case. It only seems able to download all photos within a set, but not all photos not in a set.

I think the -u USERNAME option should be changed to download all photos for a given user, including photos not in a set.

Add exif data to json?
Some photos after download lose the exif data, but it's present on Flickr. Example: https://www.flickr.com/photos/jfroberts/45787037825/in/dateposted/

What I do is rename the files after download, because I'm interested on camera or lenses samples, so intead of 29226498103.jpg
I preffer LEICA SL Typ 601_Summilux-M 1.4 28 ASPH._29226498103.jpg

I use a dirty one liner bash script to do so:

for FILE in /mnt/f/Pictures/FlickrDownloader/John\ F.\ Roberts/*.jpg; do \
	FILENAME=`basename "${FILE}" | grep -oP '.*.(jpeg|jpg|png)'`;
	MODEL=$(exif "${FILE}" | grep '^Model' | grep -oP '\|.*' | sed 's/|//;s// /' | sed 's/[\/()<>:.\\|?*]\+//g');
	LENS=$(exif "${FILE}" | grep '^Lens Model' | grep -oP '\|.*' | sed 's/|//;s// /' | sed 's/\// /' | sed 's/1://' | sed 's/ f / f/' | sed 's/[\/()<>:\\|?*]\+//g');
	FNAME=$MODEL"_"$LENS"_"$FILENAME;
	NEWFILENAME=$(echo $FNAME | sed 's/^[ \t_]*//g' | sed '/^\s*$/d' | sed 's/__/_/g' );
	mv "${FILE}" "/mnt/f/Pictures/FlickrDownloader/John F. Roberts/${NEWFILENAME}";
done

Hi again
I have this error because I don't know how to find user name
Please help

Traceback (most recent call last):
  File "/usr/local/bin/flickr_download", line 11, in <module>
    load_entry_point('flickr-download==0.2.15', 'console_scripts', 'flickr_download')()
  File "/usr/local/lib/python2.7/dist-packages/flickr_download/flick_download.py", line 299, in main
    print_sets(args.list)
  File "/usr/local/lib/python2.7/dist-packages/flickr_download/flick_download.py", line 223, in print_sets
    user = Flickr.Person.findByUserName(username)
  File "/usr/local/lib/python2.7/dist-packages/flickr_api/reflection.py", line 288, in static_call
    r = method_call.call_api(method=flickr_method, **method_args)
  File "/usr/local/lib/python2.7/dist-packages/flickr_api/method_call.py", line 134, in call_api
    raise FlickrAPIError(resp["code"], resp["message"])
flickr_api.flickrerrors.FlickrAPIError: 1 : User not found```

Hi Allan,

I have your flickr_download successfully running on a Raspberry Pi (os: latest Raspbian).
Running commands from the command line does the job.

I put the command into a flickr_download.sh, made it executable and it is doing it's job like that, too.

Unfortunately my attempt of having cron daemon running the script putting my flickr_download.sh into my user's (or root's) crontab does not execute and outputs following error:

File "/usr/local/bin/flickr_download", line 9, in <module>
    load_entry_point('flickr-download==0.2.14', 'console_scripts', 'flickr_download')()
  File "/usr/local/lib/python2.7/dist-packages/flickr_download/flick_download.py", line 252, in main
    if sys.stdout.encoding.lower() != 'utf-8':
AttributeError: 'NoneType' object has no attribute 'lower'

I assumed crontab was not using the correct locale settings. Therefore I made sure that crontab is using the same locale like the system. Checking both gives a result as follows:

LANG=en_US.UTF-8
LANGUAGE=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-8

So for now I don't know where or what for to keep on searching.
I'd be super happy if you had a hint for me where I could continue to finally find a solution.

Thanks a lot in advance and best regards from France
Johannes

Hi beaufour,

First off, thanks for all the support you've provided. I'm the same user that reported the video downloading issues.

I have two Flickr accounts and am now trying to download all the sets for my 2nd Flickr account. I ran what I believe are the same steps as the first one but get stuck at a very initial stage. I have generated a different api token for each account.

1. When I run this command:
   flickr_download -k 5286d917cc468371452813a1140a036e -s <my_secret> --download_user alteramike --user_auth --cach api_cache --metadata_store
   I receive this printout:
   INFO:root:Caching is enabled
   Traceback (most recent call last):
   File "", line 198, in _run_module_as_main
   File "", line 88, in run_code
   File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Scripts\flickr_download.exe_main.py", line 7, in
   File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_download\flick_download.py", line 536, in main
   download_user(
   File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_download\flick_download.py", line 337, in download_user
   user = find_user(username)
   ^^^^^^^^^^^^^^^^^^^
   File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_download\flick_download.py", line 318, in find_user
   user = Flickr.Person.findByUserName(userid)
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_api\reflection.py", line 294, in static_call
   r = method_call.call_api(method=flickr_method, **method_args)
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "C:\Users\Mike\AppData\Local\Programs\Python\Python311\Lib\site-packages\flickr_api\method_call.py", line 150, in call_api
   raise FlickrAPIError(resp["code"], resp["message"])
   flickr_api.flickrerrors.FlickrAPIError: 98 : Invalid auth token

+++++++++++++
2) the prior command I ran was this:
flickr_download -k 5286d917cc468371452813a1140a036e -s <my_secret> -t
ERROR: Nothing to do?

...and then a printout of the help....
+++++++++++++

I figured I needed to run item (2) before item (1) as per the README but decided to take a shot at running item (1) anyway and the error message is as expected. Do you know why item (2) shows an "ERROR: Nothing to do?" message?

I created an api key via the link you provided:
https://www.flickr.com/services/api/misc.api_keys.html
and that's what I have in the -k and -s fields.

Thanks,
Mike

When downloading all my photos, flickr-download saves Title.jpg.jpg and then proceeds to complain it can't find Title.jpg. I did a terrible thing and worked around this in the code, but I thought I'd mention it in case it was an issue going forward.

Traceback (most recent call last): File "/home/pmow/.local/bin/flickr_download", line 11, in <module> sys.exit(main()) File "/home/pmow/.local/lib/python2.7/site-packages/flickr_download/flick_download.py", line 357, in main args.skip_download) File "/home/pmow/.local/lib/python2.7/site-packages/flickr_download/flick_download.py", line 237, in download_user download_set(photoset.id, get_filename, size_label, skip_download) File "/home/pmow/.local/lib/python2.7/site-packages/flickr_download/flick_download.py", line 100, in download_set download_list(pset, pset.title, get_filename, size_label, skip_download) File "/home/pmow/.local/lib/python2.7/site-packages/flickr_download/flick_download.py", line 137, in download_list do_download_photo(dirname, pset, photo, size_label, suffix, get_filename, skip_download) File "/home/pmow/.local/lib/python2.7/site-packages/flickr_download/flick_download.py", line 207, in do_download_photo os.utime(fname, (taken_unix, taken_unix))

The final line is:
OSError: [Errno 2] No such file or directory: '2018-01-01 Album Name Here/DSCF2545.jpg'

Tonight it looks like Yahoo is having server issues so I'm getting some HTML back after 10-15 photos. My ghetto "fix" is at least downloading the photos. I commented out:

fname = fname + suffix

I also concatenated the os.utime call to include the suffix to balance. I don't know python, or the flickrapi library.

I noticed that when not specifying --quality SIZE_LABEL, the code tries to get the larger one.
However, I don't think this is the best way because you'll probably end up with a file that is larger than the original (!) and without much or its metadata (EXIF).
A better approach would be to download the Original.

What do you think?

Example:
https://secure.flickr.com/photos/boston_public_library/4325490130

Downloading the following album ID: 72157661696317310
(using flickr_download.exe -t -k -s -d %id%)
gives the following output:

Traceback (most recent call last):
  File "C:\Python27\Scripts\flickr_download-script.py", line 9, in <module>
    load_entry_point('flickr-download==0.2.12', 'console_scripts', 'flickr_download')()
  File "C:\Python27\lib\site-packages\flickr_download\flick_download.py", line 220, in main
    download_set(args.download, get_filename, args.quality)
  File "C:\Python27\lib\site-packages\flickr_download\flick_download.py", line 121, in download_set
    print('Saving: {0}'.format(fname))
  File "C:\Python27\lib\encodings\cp850.py", line 12, in encode
    return codecs.charmap_encode(input,errors,encoding_map)
UnicodeEncodeError: 'charmap' codec can't encode characters in position 50-51: character maps to <undefined>

It is probably due to this text: 複製 (it's in the description of https://www.flickr.com/photos/138222944@N04/23047089214/in/album-72157661696317310/)

I wonder if that is fixable on Windows as there are no real utf8 locales.

I constantly get this error:

Traceback (most recent call last):
  File "/usr/bin/flickr_download", line 11, in <module>
    load_entry_point('flickr-download==0.2.19', 'console_scripts', 'flickr_download')()
  File "/usr/lib64/python3.4/site-packages/pkg_resources/__init__.py", line 565, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib64/python3.4/site-packages/pkg_resources/__init__.py", line 2598, in load_entry_point
    return ep.load()
  File "/usr/lib64/python3.4/site-packages/pkg_resources/__init__.py", line 2258, in load
    return self.resolve()
  File "/usr/lib64/python3.4/site-packages/pkg_resources/__init__.py", line 2264, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib64/python3.4/site-packages/flickr_download/flick_download.py", line 195
    except IOError, ex:
                  ^
SyntaxError: invalid syntax

If there're two photos in the same set which have the same description "Foo", the script will try to save them both as "Foo.jpg", and only one of them will be saved to disk.

The script should probably save them as "Foo (1).jpg" and "Foo (2).jpg".

I have some photos that I cross-posted from Instagram. It used the description from the source end as the title on the destination end. This produced some humorous results when using the -p USERNAME mode with default naming.

I'll take a look at the code tonight. I imagine this utility will have some growing popularity due to people leaving Flickr after the Verizon Yahoo! purchase. That's how I found it.

Saving: lazzarello/new dangly earring. copper and crystal for the Lost Boys look circa 1984. courtesy Lavender Buttons @lavenderbuttons 💋👏.jpg
Saving: lazzarello/Today marks the day I realized I can fit my whole office into a canvas saddle bag attached to the back of my bike. This includes a 15" MacBook Pro, a Happy Hacker Keyboard pro, a Wacom Bamboo tablet and a plastic stand for the computer. I look ridiculous..jpg
Traceback (most recent call last):
  File "/usr/local/bin/flickr_download", line 9, in <module>
    load_entry_point('flickr-download==0.2.15', 'console_scripts', 'flickr_download')()
  File "/usr/local/lib/python2.7/site-packages/flickr_download-0.2.15-py2.7.egg/flickr_download/flick_download.py", line 267, in main
    download_user_photos(args.download_user_photos, get_filename, args.quality)
  File "/usr/local/lib/python2.7/site-packages/flickr_download-0.2.15-py2.7.egg/flickr_download/flick_download.py", line 178, in download_user_photos
    download_list(user, username, get_filename, size_label)
  File "/usr/local/lib/python2.7/site-packages/flickr_download-0.2.15-py2.7.egg/flickr_download/flick_download.py", line 146, in download_list
    photo.save(fname, photo_size_label)
  File "/usr/local/lib/python2.7/site-packages/flickr_api/objects.py", line 1313, in save
    with open(filename, 'wb') as f:
IOError: [Errno 63] File name too long: u'lazzarello/Today marks the day I realized I can fit my whole office into a canvas saddle bag attached to the back of my bike. This includes a 15" MacBook Pro, a Happy Hacker Keyboard pro, a Wacom Bamboo tablet and a plastic stand for the computer. I look ridiculous..jpg'

Hi,
I run the command to start the download, but it only returned "72157662608843826 - Auto Upload"
Nothing happened.

what's wrong here?

Thank you!

iter() returned non-iterator of type 'java.util.ArrayList$Itr'

오류가 뜨는데 어떻게 해결할 수 있나요?ㅠㅠ

A better introduction to all the different command line arguments and some more examples would go a long way.

First image saves successfully.

$ python flick_download.py -k MYAPIKEY -s MYSECRET -d 72157629549177588
Saving: 5716838906.jpg
Traceback (most recent call last):
File "flick_download.py", line 94, in
sys.exit(main())
File "flick_download.py", line 87, in main
download_set(args.download)
File "flick_download.py", line 54, in download_set
taken = parser.parse(info['taken'])
File "/Library/Python/2.7/site-packages/python_dateutil-2.0-py2.7.egg/dateutil/parser.py", line 698, in parse
return DEFAULTPARSER.parse(timestr, *_kwargs)
File "/Library/Python/2.7/site-packages/python_dateutil-2.0-py2.7.egg/dateutil/parser.py", line 302, in parse
res = self._parse(timestr, *_kwargs)
File "/Library/Python/2.7/site-packages/python_dateutil-2.0-py2.7.egg/dateutil/parser.py", line 350, in _parse
l = _timelex.split(timestr)
File "/Library/Python/2.7/site-packages/python_dateutil-2.0-py2.7.egg/dateutil/parser.py", line 144, in split
return list(cls(s))
TypeError: iter() returned non-iterator of type '_timelex'
SEGmac:Woodbury seg$ python flick_download.py -k 8b25db725b263858661904abc4997ce4 -s 40433c69026d2c13 -d 72157629549177588
Skipping 5716838906.jpg, as it exists already
Saving: 5716278531.jpg
Traceback (most recent call last):
File "flick_download.py", line 94, in
sys.exit(main())
File "flick_download.py", line 87, in main
download_set(args.download)
File "flick_download.py", line 54, in download_set
taken = parser.parse(info['taken'])
File "/Library/Python/2.7/site-packages/python_dateutil-2.0-py2.7.egg/dateutil/parser.py", line 698, in parse
return DEFAULTPARSER.parse(timestr, *_kwargs)
File "/Library/Python/2.7/site-packages/python_dateutil-2.0-py2.7.egg/dateutil/parser.py", line 302, in parse
res = self._parse(timestr, *_kwargs)
File "/Library/Python/2.7/site-packages/python_dateutil-2.0-py2.7.egg/dateutil/parser.py", line 350, in _parse
l = _timelex.split(timestr)
File "/Library/Python/2.7/site-packages/python_dateutil-2.0-py2.7.egg/dateutil/parser.py", line 144, in split
return list(cls(s))
TypeError: iter() returned non-iterator of type '_timelex'
SEGmac:Woodbury seg$

When downloading a set, I occaisionally see the error WARNING:root:IO error saving photo: None

It happened 6 times when downloading the set with ID 72157662915003917

I notice that there are several photos which have partially downloaded. They truncate about halfway down the image.

I have plenty of space on my SSD and a stable Internet connection.

Is this something I'm doing wrong?

A little behind here :)

Branch at: https://github.com/beaufour/flickr-download/tree/python3 but we need a new flickr_api release: alexis-mignon/python-flickr-api#109

python-dateutil had an additional updated release in Apr2021 with 2.8.2.
A few of my other packages in pip require the newer one.
Can you bump the dependency to allow for greater versions for python-dateutil in pyproject.toml?
from
python-dateutil = "2.8.1"
to
python-dateutil = "^2.8.1"
?

I will submit a PR when I make sure it works.

Hi,

So I am no coder, but I managed to figure out how to use the Python 3 build. I had to overcome the "No such file or directory" error by downloading alexis-mignon's edited objects.py in python-flickr-api, then having to correct the photo_file undefined error it produced by re-adding r = urllib.request.urlopen(photo_file) after line 1349 as recommended, with spaces not tabs. Phew.

So this started downloading the files no problem

Saving: Dealer Ads 0571\Wall St 2 sm 2.jpg (https://www.flickr.com/photos/38552878@N02/51145104308/)
Saving: Camera Ads 0571\Beseler 1 sm 2.jpg (https://www.flickr.com/photos/38552878@N02/51145112038/)
Saving: Camera Ads 0571\Da-Lite 1 sm 2.jpg (https://www.flickr.com/photos/38552878@N02/51145992720/)
Saving: Camera Ads 0571\Exakta 1 sm 2.jpg (https://www.flickr.com/photos/38552878@N02/51144886661/)
Saving: Camera Ads 0571\GAF 1 sm 2.jpg (https://www.flickr.com/photos/38552878@N02/51145992155/)
Saving: Camera Ads 0571\Gossen 1 sm 2.jpg (https://www.flickr.com/photos/38552878@N02/51144208917/)

Upon encountering a folder with a : colon as it's title, it failed to save and gave me NotADirectoryError: [WinError 267] The directory name is invalid: 'RR: Minolta SRT-101'.

Am I missing an option/setting, or is this a genuine error? If so, how could this (and all the other errors caused by 'invalid' file name characters in Windows) be dealt with?

I've seen albums where each photo is named the same. When downloading It says file exists and skips it.
Can you add something like (1) if file name exists already?
Edit: Oh, looks like #26 is similar.

For exmaple, this album has 25 photos, but it only downloads 4 photos:

https://www.flickr.com/photos/devorobin/albums/72157675497780042
flickr_download -k *** -s *** --download 72157675497780042 --naming title_and_id

Same issue when trying to download albums of a given user:

https://www.flickr.com/photos/devorobin/albums
flickr_download -k *** -s *** --cache api_cache --metadata_store --download_user "Dwayne Martin"

The utility could have a feature where it would delete photos locally that has been deleted on Flickr to keep Flickr and the local download in sync.

This is a follow up for issue #23

Hi there I am very new to computer languages and just got into learning Java. But this project looks interesting and definitely want to try this out. I have installed Python and ran the command pip install flickr_download from cmd and it says successful. Then Where do I set api_key to my api key and secret. I opened flickr_download.py, there in the def init where shall I put my key and secret?. Please help.

Thank you very very much for this repo! Saved my day. 😃

Just a quick suggestion. In the --help output and as well on the README be sure to add these lines:

-n NAMING_MODE, --naming NAMING_MODE
                        Photo naming mode
                        Tip: Use "flickr_download --list_naming" to get a list or possible NAMING_MODEs

and

-q SIZE_LABEL, --quality SIZE_LABEL
                        Quality of the picture
                        Tip: Refer to Flickr documentation https://www.flickr.com/services/api/flickr.photos.getSizes.html.
                        Examples are: "Original", "Large", "Small", ...

That would have saved me time, so I think users would greatly appreciate.

Thanks again!

Hi, I have working well on 1ft account and now I try to connect on 2nd account and I have this error
flickr_download -t -k <api_key> -s <secret_key>

  File "/usr/local/bin/flickr_download", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/flickr_download/flick_download.py", line 288, in main
    ret = _init(args.api_key, args.api_secret, args.user_auth)
  File "/usr/local/lib/python2.7/dist-packages/flickr_download/flick_download.py", line 50, in _init
    auth = Flickr.auth.AuthHandler()  # creates the AuthHandler object
  File "/usr/local/lib/python2.7/dist-packages/flickr_api/auth.py", line 89, in __init__
    resp = urllib2.urlopen(req.to_url())
  File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 475, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 558, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 401: Unauthorized```

On version 2.17 all our files are saved as .jpg. Is there some flag that fixes this, and appends the correct filetype?

Github Actions can automatically publish on PyPi: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python#publishing-to-package-registries

@beaufour to your knowledge does the Flickr API offer any sort of MD5 hash of the image data? Would be handy to not have to download it and instead just make a comparison against the local hash to the hash signature available in the API.

For example, in order to download all user's photos we have to specify its literal name like this: --download_user_photos "★☆洪爺の銀鹽專賣★☆"
Wouldn't it be better if we could specify the username like this? Example: --download_user_photos "titsanic002"
This could avoid write issues on disk when the name has special characters.

The username is in the link: https://www.flickr.com/photos/titsanic002/

Hello,

First of all, thank you very much for developing this tool. It is a saviour when making a backup of my Flickr database.

When running the tool, I found out that the metadata option does not seem to work when downloading all sets of a user. I am using this option now to make a copy of all of my Flickr database. However, because of the errors every now and then, I am automatically restarting the script each time.

Even though the cache option already helps a great deal when checking for already downloaded files, I imagine that it would be even smoother with the metadata storage option.

Therefore, I was wondering if it is possible to implement this option for the user sets as well. Hopefully, this is a possibility!

Thank you in advance for your response!

I get a User not found error for user 151745458@N02 when running via flickr_download -k <api key> -s <api secret> -l 151745458@N02 (of course I inserted my key and secret). I also authorized a token because the photos are not public but accessible with my account.

It would be very handy to have an option to delete local images that are no longer in the stream. Even if you could just list the images in an album would be very handy.

They are mostly non-existent...

I've been getting the following error - OverflowError: mktime argument out of range.

I think it's for old photos where the album owner has set it to a date/time too far in the past and unix can't change the date/time to it. Commenting out the date/time change worked for me.

How do i use this library to download speccific images?

The default naming setting will give .jpg filename to all photos without a title. It really should detect that the title is empty and pick the id instead so that at least we download something.

That or switch to a different default option.

When trying to download the JSON info/metadata sibling file with a photo it fails to add anything to the JSON and errors with the following:

WARNING:root:Trouble saving photo info: <class 'flickr_api.flickrerrors.FlickrAPIError'>

Full command and photo used:

$ python3 flickr_download -k <key> -s <secret> -i 52693275262 -n id -j
INFO:root:Will save photo info in .json file with same basename as photo
INFO:root:Saving photo info: ./52693275262.jpg.json
WARNING:root:Trouble saving photo info: <class 'flickr_api.flickrerrors.FlickrAPIError'>
INFO:root:Saving: ./52693275262.jpg (https://www.flickr.com/photos/110898173@N04/52693275262/)

Then I tried the OAuth authentication method instead using the -t command I was able to successfully authenticate according to the terminal messages but it still didn't output a JSON with anything in it.

What I entered for the script's OAuth verification prompt was the last value in the flickr URL from the plaintext page that appears after allowing the permission on the flickr site (the value from the part named oauth_verifier=<value>). Perhaps this wasn't what was meant to be entered? Could use clarification on this.

OAuth token was saved, re-run script to use it.
$ python3 flickr_download -k <key> -s <secret> -i 52693275262 -n id -j -t
INFO:root:Will save photo info in .json file with same basename as photo
INFO:root:Saving photo info: ./52693275262.jpg.json
WARNING:root:Trouble saving photo info: <class 'flickr_api.flickrerrors.FlickrAPIError'>
INFO:root:Saving: ./52693275262.jpg (https://www.flickr.com/photos/110898173@N04/52693275262/)

Am I missing something?

Environment:

• Kubuntu
• Latest script version installed via pip (had no prior version of the script installed)

Hi

Thanks for this amazing script. It does exactly what I was looking for!

I struggled a bit with the -t option, but figured it out eventually. The readme doesn't entirely make it clear that after initialising authorisation you need to add the -t option to all subsequent calls to use it.

Currently when downloading photosets, we do a PhotoSet.getInfo() and a PhotoSet.getPhotos(), and then for each photo a Photo.getInfo() and a Photo.getSizes(). If you call PhotoSet.GetPhotos() with a bunch of extras parameters we'd have just about all the metadata needed to download all the photos, and we could save all the Photo.* calls. It's a bit of a major incision into the flickr API library though as it expects the Photo class to do calls to get its metadata, and the size and save operations needs calls too. But maybe a custom Photo class could work here. It would speed up things quite a bit. Especially as the Flickr API is a bit slow.

First of all, thanks for the great tool.

One thing that may be worth looking into is adding parallelism or some other kind of batching. Right now on a 100 mbit link, it takes a number of hours to download a ~10G set, and the link only becomes saturated in bursts. This seems to indicate that the work around downloading a photo using the API is a non-trivial amount of work that may benefit from being done in parallel.

I'm not sure if I'll have time to submit a pull request soon, so I'm submitting this ticket in lieu for now to see your thoughts on adding parallelism support to flickr-download. When I have more time I will get some more profiles and see if there are any low hanging fruit, as well.

Worth opening a separate post for.

This is not about the flickr-api being slow or even that it has errors- just the way the program handles this in my attempt to run it unattended.

The program will occasionally hit HTTP Server Error 500 which forces it to close. Thankfully, upon restart it will check to see if a file has already been downloaded. The issue is how this is handled.

It seems a working connection to the Flickr website is required to check files. I'm guessing it starts the download/connection process with the latest photoset, and then only check if it exists locally.

This leaves the program liable to fail at the checking stage due to API errors, making it very hard to reach the resume download stage with large photosets, as well as being slow.

For example - I am on 17,402 images. On average I'm going to guess it can check 1.6 photos a second, which means 3 hours of no API issues before it can resume downloading.

Using https://github.com/chebum/Supervisor to automatically restart the program, I have left it running all day just to see it stuck checking files.

I wish to download an album from flickr but...

OSError: [Errno 36] File name too long: "Monographie des melastomac\xc3\xa9es, comprenant toutes les plantes de cet ordre r\xc3\xa9cueillies jusqu'\xc3\xa0 ce jour, et notamment au M\xc3\xa8xique, dans l'Ile de Cuba, dans les provinces de Caracas, de Cumana, et de Barc\xc3\xa8lone, aux Andes de la Nouvelle-Grenade, de Quito et du"

Original album link :

https://www.flickr.com/photos/biodivlibrary/albums/72157711550179608

Thanks