Add option to cache `Etag`/`LastModified` about podcast-dl HOT 4 CLOSED

One stopper is that in order to support name templating, we need to have access to the feed data in order to generate the paths/names. We could require the path be specified without templating (and default to the current working directory).

We'd open/save to a JSON file with keys that correspond to the provided URL, and save the ETag and Last-Modified headers there for now.

from podcast-dl.

It does seem like a chicken and egg problem, doesn't it? The podcast name needs to be known to resolve the default output folder, but there's little point in caching the feed in that folder if the RSS has to be downloaded to know the podcast name.

I think it would make the most sense to go along with user intent and hint at how to accomplish what they want. If the user requests to cache the feed and has not specified an explicit out-dir, cache the feed anyway, but also print a message along the lines of:

NOTICE: The feed has been cached as you requested, but the cache can only be
utilized if you specify the output directory without a template. For this
feed, the actual path is: /path/to/resolved/out-dir

Also, don't forget the cache-control headers. If present, those set a TTL for the cached copy that should be followed regardless of the other headers.

from podcast-dl.

@calebj - Good thoughts. Appreciate it!

from podcast-dl.

It's been a while! Taking a look at this again, I think it would be easier and a better separation of concerns to use something like curl before running podcast-dl to check if the resource has changed. You can use --etag-save and --etag-compare: https://man7.org/linux/man-pages/man1/curl.1.html

from podcast-dl.