itaybb / pysmartdl Goto Github PK

View Code? Open in Web Editor NEW

188.0 188.0 56.0 692 KB

A Smart Download Manager for Python

License: The Unlicense

Python 100.00%

downloader pysmartdl python

pysmartdl's People

Stargazers

Watchers

Forkers

benroeder corydolphin pombreda priestd09 amin1024 tienpp hanipcode neymello mrswiss sumityadav6541 drfenerbache xon91 ug-code savefromgoogle allyunion carp3 jjisnow funtw bhasindhruv117 solismaa ogli324 minerllabs dashlt isouzasoares opoplawski t-rekttt tuanbinh93 faisalalwazir blatzar xyxdaily alfonsodg incognito009 shadow443 cryguy jinzer0-fork epic-r-r abirhasan2005 haithamofficial nitrah24 cadu-leite jackabu ayu-dash jpsamarino alexsilversword chitranjan6g amirulandalib amkrajewski kmkhant msinamsina saeedtaghavi heartlog vaibhavsijaria longlinh slycordinator mathewgeola

pysmartdl's Issues

Customizing number of threads to use

It seems PySmartDL doesn't support custom threads number, as it always uses 5.
I think this would be an useful option.

Completes download, restarts, runtime error

[*] 236.0 MB / 236.0 MB @ 0 bytes/s [##################] [100%, 0s left]
[*] 236.0 MB / 236.0 MB @ 267 kB/s [##################] [100%, 0s left]
<class 'RuntimeError'> cannot join thread before it is started

URL [https://streaming.lemonstream.me/*****] from https://gostream.is/film/rick-and-morty-season-1-9011/watching.html?ep=127301.

always download with only one thread

Hello,

I am using pySmartDL 1.3.4 on ubuntu 16.04. Even if I set thread = 5 explicitly, the tmp file only has .000.

Does this mean pySmartDL only use one thread to download file?

Whereas when I use it on windows, every file has multiple tmp files: .000, .001, .002, .003, .004.

Can somebody provide instruction on how to use multi-threading download on ubuntu?

Thanks very much!

Is there pause and resume support planned?

As stated in the heading, is there any pause and resume support down the line in future versions?
Also is there any option to set the 'pySmartDL' to try 'n' time for downloading a file?

RuntimeError: cannot join thread before it is started

======================================================================
ERROR: test_hash (main.TestSmartDL)

Traceback (most recent call last):
File "test_pySmartDL.py", line 65, in test_hash
obj.wait()
File "..\pySmartDL\pySmartDL.py", line 455, in wait
self.post_threadpool_thread.join()
File "C:\Python37\lib\threading.py", line 1027, in join
raise RuntimeError("cannot join thread before it is started")
RuntimeError: cannot join thread before it is started

Ran 9 tests in 54.065s

FAILED (errors=1)

is there any way for it to just return the current progress

i want to add the progress bar to an gui so is there any way i can get the current value while its downloading

Free Palestine

How to increase the timeout threshold for a download?

Hi Itay,

Background

I'm working on a project that is using pySmartDL to download media files from a server.
The server had been using the HTTP protocol but just switched to HTTPS.
Now downloads take a long time to get started (even without using pySmartDL; it seems to be some issue with the server).

The problem

Unfortunately these more-slow download-starts seem to cause pySmartDL to hit a URLError exception on line 129 of pySmartDL.py, where it makes a call to utils.is_HTTPRange_supported(self.url).

A potential solution

I think I might be able to avoid the exception if I could pass in a larger timeout threshold to that function, for example by passing in a timeout argument to the SmartDL() call.

The stack trace:

Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/usr/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/usr/lib/python3.6/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/home/vagrant/.local/lib/python3.6/site-packages/eventlet/green/ssl.py", line 392, in wrap_socket
    return GreenSSLSocket(sock, *a, _context=self, **kw)
  File "/home/vagrant/.local/lib/python3.6/site-packages/eventlet/green/ssl.py", line 87, in __init__
    self.do_handshake()
  File "/home/vagrant/.local/lib/python3.6/site-packages/eventlet/green/ssl.py", line 260, in do_handshake
    super(GreenSSLSocket, self).do_handshake)
  File "/home/vagrant/.local/lib/python3.6/site-packages/eventlet/green/ssl.py", line 115, in _call_trampolining
    timeout_exc=timeout_exc('timed out'))
  File "/home/vagrant/.local/lib/python3.6/site-packages/eventlet/hubs/__init__.py", line 164, in trampoline
    return hub.switch()
  File "/home/vagrant/.local/lib/python3.6/site-packages/eventlet/hubs/hub.py", line 297, in switch
    return self.greenlet.switch()
ssl.SSLError: ('timed out',)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/vagrant/src/__init__.py", line 131, in wrapper_function
    thread_function(*args, **kwargs)
  File "/vagrant/src/__init__.py", line 199, in media_downloader_service
    download_queued_media(retry_failed_downloads=True)
  File "/vagrant/src/DKServerAPI/download_queued_media.py", line 52, in download_queued_media
    download_was_successful = _download_media(session, queued_download, tick_step_size, retry_failed_downloads)
  File "/vagrant/src/DKServerAPI/download_queued_media.py", line 104, in _download_media
    smart_downloader = SmartDL(full_url_to_download, media_item_specific_download_folder, progress_bar=False)
  File "/home/vagrant/.local/lib/python3.6/site-packages/pySmartDL/pySmartDL.py", line 129, in __init__
    if not utils.is_HTTPRange_supported(self.url):
  File "/home/vagrant/.local/lib/python3.6/site-packages/pySmartDL/utils.py", line 107, in is_HTTPRange_supported
    urlObj = urllib.request.urlopen(req, timeout=timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/usr/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/usr/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error ('timed out',)>

Downloading error in python 3.8

[INFO||7032@{C:\Users\<username>\AppData\Roaming\Python\Python38\site-packages\pySmartDL\pySmartDL.py:100}] Using url "https://rarlab.com//rar/winrar-x64-590b3nl.exe" [WARNING||7032@{C:\Users\<username>\AppData\Roaming\Python\Python38\site-packages\pySmartDL\pySmartDL.py:137}] Destination "C:\Users\<username>\AppData\Local\Temp\pySmartDL\" already exists. Existing file will be removed. [INFO||7032@{C:\Users\<username>\AppData\Roaming\Python\Python38\site-packages\pySmartDL\pySmartDL.py:142}] Creating a ThreadPool of 5 thread(s). [INFO||7032@{C:\Users\<username>\AppData\Roaming\Python\Python38\site-packages\pySmartDL\pySmartDL.py:236}] Starting a new SmartDL operation. [INFO||7032@{C:\Users\<username>\AppData\Roaming\Python\Python38\site-packages\pySmartDL\pySmartDL.py:246}] One URL is loaded. [INFO||7032@{C:\Users\<username>\AppData\Roaming\Python\Python38\site-packages\pySmartDL\pySmartDL.py:254}] Downloading 'https://rarlab.com//rar/winrar-x64-590b3nl.exe' to 'C:\Users\<username>\AppData\Local\Temp\pySmartDL\'... [INFO||7032@{C:\Users\<username>\AppData\Roaming\Python\Python38\site-packages\pySmartDL\pySmartDL.py:275}] Content-Length is 3554136 (3.4 MB). [INFO||7032@{C:\Users\<username>\AppData\Roaming\Python\Python38\site-packages\pySmartDL\pySmartDL.py:285}] Launching 1 thread (downloads 3.4 MB). [INFO||9072@{C:\Users\<username>\AppData\Roaming\Python\Python38\site-packages\pySmartDL\download.py:13}] Downloading 'https://rarlab.com//rar/winrar-x64-590b3nl.exe' to 'C:\Users\<username>\AppData\Local\Temp\pySmartDL\.000'... [INFO||3880@{C:\Users\<username>\AppData\Roaming\Python\Python38\site-packages\pySmartDL\control_thread.py:29}] Control thread has been started. [*] 3.4 MB / 3.4 MB @ 80 kB/s [##################] [100%, 0s left] Exception in thread Thread-1: Traceback (most recent call last): File "C:\Program Files\Python38\lib\threading.py", line 932, in _bootstrap_inner self.run() File "C:\Program Files\Python38\lib\threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "C:\Users\<username>\AppData\Roaming\Python\Python38\site-packages\pySmartDL\pySmartDL.py", line 654, in post_threadpool_actions utils.combine_files(*args) File "C:\Users\<username>\AppData\Roaming\Python\Python38\site-packages\pySmartDL\utils.py", line 32, in combine_files shutil.move(parts[0], dest) File "C:\Program Files\Python38\lib\shutil.py", line 786, in move raise Error("Destination path '%s' already exists" % real_dst) shutil.Error: Destination path 'C:\Users\<username>\AppData\Local\Temp\pySmartDL\.000' already exists [INFO||3880@{C:\Users\<username>\AppData\Roaming\Python\Python38\site-packages\pySmartDL\control_thread.py:64}] File downloaded within 16.62 seconds.

It gives above error. It downloads the part in the said directory then try to move it in the same directory which results in the above Exception.

Support custom list of SSL CAs

Beyond corporate firewall it is sometimes needed to provide a list of additional certification authorities:

https://docs.python.org/3/library/urllib.request.html#urllib.request.urlopen

urllib.request.urlopen(url, data=None, [timeout, ]*, cafile=None, capath=None, cadefault=False, context=None)
...
The optional cafile and capath parameters specify a set of trusted CA certificates for HTTPS requests. cafile should point to a single file containing a bundle of CA certificates, whereas capath should point to a directory of hashed certificate files. More information can be found in ssl.SSLContext.load_verify_locations().

Add support for cafile and capath parameters.

1973

HTTP NTLM Authentication

Cannot download from websites that requires NTLM http auth like
requests.get('https://api.github.com/user', auth=HttpNtlmAuth('user', 'pass'))

Error while downloading 🤒

Error thrown even if caught on losing of internet connection

Module throws an error even if everything is caught when I loose my Internet connection in-between file download

TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

Hey, thx for this module.
When I download the very same zip file from github sometimes it crashes but most of the time it works.

    download(URL, master_zip, Config.DOWNLOAD_CACHE_DIR)
  File "/usr/lib/python3.5/contextlib.py", line 30, in inner
    return func(*args, **kwds)
  File "/usr/local/lib/python3.5/dist-packages/arm_now/download.py", line 30, in download
    obj.start()
  File "/usr/local/lib/python3.5/dist-packages/pySmartDL/pySmartDL.py", line 266, in start
    self.filesize = int(urlObj.headers["Content-Length"])
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

http://github.com/nongiach/arm_now

reading the filename with content-disposition header

hi,
it should be possible to first perform a head request, then from that returned response, read the headers.
this way, it will be possible to obtain the filenames from dynamically generated URLS
the filename will be stored in content-disposition header

Can't stop download

Stopping a download in non blocking mode restarts the download rather than stopping it. The reason is, the script retries the download. Up to maximum number of times (4 by default).

Last byte is truncated for file with specific size and specific number of threads

Hello,

I use pySmartDL 1.3.2 on Ubuntu 18.04. I noticed that the last byte of a downloaded file is truncated, when a file size is 1906023034 and a number of threads is 20.

Reproduction steps:

Generate a file with size 1906023034. I used 'fallocate' command:
$ fallocate -l 1906023034 test_file
Put a test file on HTTP server. I used nginx, but any server which supports multiple connections and range-based requests shall be fine.
Download a file. Here is my program:

import logging
import tempfile

from pySmartDL import SmartDL

if __name__ == '__main__':
    logging.basicConfig(level=logging.DEBUG)
    urls = [
        'http://localhost:8000/test_file']
    tempdir = tempfile.mkdtemp()
    obj = SmartDL(urls=urls,
                  dest=tempdir,
                  threads=20,
                  logger=logging.getLogger(__name__))
    obj.timeout = 30
    obj.start(blocking=True)

Verify file size after download:
$ du -b test_file
1906023033 test_file

I also noticed that if you use 100 threads, then the file size is correct.

pause

hello i want to know how can i pause and resume download of file
so thanks

Not saving in any Directory

Hello, i tried to work with you package but it doesnt seem to be saving the data to any location. It however claims to finish but no specific directory works.

How to set retries if initial download fails

So in some parts of the world we have dodgy internet.
Connections die, connections time out on trying to negotiate ssl etc. etc.

If there an easy way to tell pySmartDL to try and download a url X-number of times, regardless any possible error?

Wrong error message, if total file size doesn't match expected file size.

In pySmartDL.py in function 'post_threadpool_actions' there is the following line:
errMsg = 'Diff between downloaded files and expected filesizes is {}B (filesize: {}, expected_filesize: {}, {} threads).'.format(total_filesize, expected_filesize, diff, threads)

This line is wrong and should be replaced by:
errMsg = 'Diff between downloaded files and expected filesizes is {}B (filesize: {}, expected_filesize: {}, {} threads).'.format(diff, total_filesize, expected_filesize, threads)

The current error is very misleading.

Download of multiple files

The constructor takes list of URLs but download only the first index in destination folder.

Reduce download speed at last

hi
thanks for your useful module
i use this lib to download files with multi threading ( 5 thread)
but problem is :
download start with fast speed (10 ~ 20 MB/sec)
but at last (after 80%) download speed reduce under 1 MB/sec
any way to fix this ?

functionality to use different url if one fails and continue downloading

will it be possible to add functionality like using a different url only if one fails and continuing downloading #14 from where it left off, this will be usefull for downloading files which have exipre time so even after the link expires we can just use a new link and continue from where we left off. the current mirror function checks before hand if the 2nd url is valid even when the first one is vaild it would be great if it only checked the 2nd url in case the first one fails or returns an error

edit: found a way to do this, pypdl

403 error

The url from this website doesnt work in pySmarDL, but will downlaod in the browser and in other download managers.
https://9xbud.com/https://openload.co/f/B1avsUR3ZQE/%5BHorribleSubs%5D_Ao_no_Exorcist_-_Kyoto_Fujouou-hen_-_01_%5B720p%5D-r964-rh-173.mp4 (the Download Now button)

[WARNING] Server does not support HTTPRange. threads_count is set to 1. [INFO] Creating a ThreadPool of 1 thread(s). [INFO] One URL is loaded. [INFO] Downloading 'http://s01.9xbuddy.com/down/aHR0cHM6Ly9vcGVubG9hZC5jby9zdHJlYW0vQjFhdnNVUjNaUUV-MTQ5MTk3NDM5M34xNjMuMTcyLjAuMH5sd1ctVFJ3UD9taW1lPXRydWU%2C/96947a5224fed79b4e7de91d9aab980e-1491902395' to 'C:\Users\edwar\AppData\Local\Temp\pySmartDL\96947a5224fed79b4e7de91d9aab980e-1491902395'... [WARNING] HTTP Error 403: Forbidden Traceback (most recent call last): File "C:/Users/edwar/Downloads/python_test/test pydownload 9xbuddy.py", line 15, in <module> dl.start() File "C:\Users\edwar\AppData\Local\Programs\Python\Python35-32\lib\site-packages\pySmartDL\pySmartDL.py", line 250, in start urlObj = urllib.request.urlopen(req, timeout=self.timeout) File "C:\Users\edwar\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 163, in urlopen return opener.open(url, data, timeout) File "C:\Users\edwar\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 472, in open response = meth(req, response) File "C:\Users\edwar\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "C:\Users\edwar\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 504, in error result = self._call_chain(*args) File "C:\Users\edwar\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 444, in _call_chain result = func(*args) File "C:\Users\edwar\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 696, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "C:\Users\edwar\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 472, in open response = meth(req, response) File "C:\Users\edwar\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "C:\Users\edwar\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 510, in error return self._call_chain(*args) File "C:\Users\edwar\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 444, in _call_chain result = func(*args) File "C:\Users\edwar\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden

How to resume partially downloaded file in pysmartDL

I would like to resume downloads from the previously downloaded part. Now each time the download starts from the beginning. How to check for the downloaded data...

Question: Will the mirror function pick the fastest one?

When I add several mirrors to the list, let's say in random order, will the mirror functionality choose the fastest one?

I use archive.org as a fallback. That is reliable but very slow. I don't want people to end up with the slow mirror if the others are still available.

Add supprt for request.session cookies or passing referer

I have a url. curl -H "Referer: Url" Link
How to implement it in pysmartdl
tried
request_args = {"headers": {"Referer": "url", "User-Agent": "Mozilla/5.0"}}

but not working says 410 gone. same error when curl is run without referer

Downloading in parts

Would be nice if this could download in 1 file, so combining isn't needed as this is very slow if for example you download to a network location.

I think queue can do this ? like this one:
https://github.com/DiSiqueira/DSDownload

RuntimeError: cannot schedule new futures after shutdown

Got an error while using your pySmartDL:

2022-02-09T15:19:28.538821+00:00 app[web.1]: Exception in thread Thread-5:
2022-02-09T15:19:28.538840+00:00 app[web.1]: Traceback (most recent call last):
2022-02-09T15:19:28.538858+00:00 app[web.1]: File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
2022-02-09T15:19:28.539230+00:00 app[web.1]: self.run()
2022-02-09T15:19:28.539247+00:00 app[web.1]: File "/usr/lib/python3.8/threading.py", line 870, in run
2022-02-09T15:19:28.539522+00:00 app[web.1]: self._target(*self._args, **self._kwargs)
2022-02-09T15:19:28.539540+00:00 app[web.1]: File "/usr/local/lib/python3.8/dist-packages/pySmartDL/pySmartDL.py", line 646, in post_threadpool_actions
2022-02-09T15:19:28.539776+00:00 app[web.1]: SmartDLObj.retry(str(pool.get_exception()))
2022-02-09T15:19:28.539793+00:00 app[web.1]: File "/usr/local/lib/python3.8/dist-packages/pySmartDL/pySmartDL.py", line 341, in retry
2022-02-09T15:19:28.539941+00:00 app[web.1]: self.start()
2022-02-09T15:19:28.539957+00:00 app[web.1]: File "/usr/local/lib/python3.8/dist-packages/pySmartDL/pySmartDL.py", line 300, in start
2022-02-09T15:19:28.540098+00:00 app[web.1]: req = self.pool.submit(
2022-02-09T15:19:28.540114+00:00 app[web.1]: File "/usr/local/lib/python3.8/dist-packages/pySmartDL/utils.py", line 343, in submit
2022-02-09T15:19:28.540293+00:00 app[web.1]: future = super().submit(fn, *args, **kwargs)
2022-02-09T15:19:28.540308+00:00 app[web.1]: File "/usr/lib/python3.8/concurrent/futures/thread.py", line 179, in submit
2022-02-09T15:19:28.540443+00:00 app[web.1]: raise RuntimeError('cannot schedule new futures after shutdown')
2022-02-09T15:19:28.540498+00:00 app[web.1]: RuntimeError: cannot schedule new futures after shutdown

Progress?

It doesn't show download progress am i doing something wrong? code:

from pySmartDL import SmartDL

url = "https://downloads.niresh.co/temp/2021-02-03/1637901250/macOS%20Catalina%20Bootable%20Image.dmg"
dest = "/content/"

obj = SmartDL(url, dest)
obj.start()
path = obj.get_dest()

New Feature: currently it does not support cookie

One may want to download from a site which require Authentication or cookie to download.
changes in pySmartDL.py:
So, i tried to introduce a cookie_jar in the constructor on pySmartDL
def __init__(self, urls, dest=None, progress_bar=True, fix_urls=True, threads=5, timeout=5, logger=None, connect_default_logger=False, request_args=None, cookie_jar=cookie()):
self.cookie_jar = cookie_jar

and did following change in start() method:

        for i, arg in enumerate(args):
            req = self.pool.submit(
                download,
                self.url,
                self.dest+".%.3d" % i,
                self.requestArgs,
                arg[0],
                arg[1],
                self.timeout,
                self.shared_var,
                self.thread_shared_cmds,
                self.logger,
                3,
                self.cookie_jar
            )

changes in download.py:

    opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cookie_jar))
    cj = cookie_jar._cookies[domain]['/'][cookie_name]  #i provided domain and cookie_name explicitly but we also have to fetch it or take it from user
    ctuple = (cj.name, cj.value)
    opener.addheaders = [ctuple, ('User-Agent', 'Mozilla/5.0')]
    urllib.request.install_opener(opener)

but after introducing it, it still not working on site with cookies it is giving socket time out error and i don't know much about networking so i hope you can help.

utils.url_fix breaks url

When using url_fix it corrupts a valid url:

self.mirrors[0]
'<url_removed>'

utils.url_fix(self.mirrors[0])
'<url_removed>'

Given that True is the default it took me a long time to figure this out...

python 3.7.4 supporting

is this library supporting python 3.7.4 ? I think doesnt support because doesnt work properly in this python version.

Progress of download is not shown in PyCharm Run Console

How to reproduce

Write ordinary code for downloading in some .py file:

obj = SmartDL(url, dest)
obj.start()

and run it using PyCharm.

What happens

Progress is not displayed.

Proposed solution

I propose to use tqdm for displaying progress if it is installed. It's lightweight, have no dependencies, and works well with PyCharm. If you agree with my proposal, I can implement it if you want and submit PR.

ConnectionResetError(104, 'Connection reset by peer') causes re-download from scratch

First of all: great lib @iTaybb -- Thank you for putting it together.

I'm dealing with large files (~17Gb) being transferred from Azure CDN to Amazon EC2. The internet connection is stable I think but I'm noticing an issue that's causing a lot of pain for me:

When the host from which the files are getting downloaded resets connection, the SmartDL client just gives up and restarts the download from scratch. This of course is a big issue when I'm trying to get such large files. Below is sample log & code.

This is somewhat related to issue #19 but not entirely since Connection reset by peer is technically not lack of internet connection but rather a standard way HTTP endpoints communicate.

Seems to me SmartDL should be able to deal with connection reset and just ask to re-download the http chunk / range it was working on at that time. Thoughts?

CODE:

def download_url(self, url, location):
    downloader = SmartDL(url, location, progress_bar=False, fix_urls=False, threads=10, logger=logger)
    downloader.timeout = 20
    downloader.attemps_limit = 10
    downloader.minChunkFile = 64 * (1024 ** 2)  # 64 MB

    try:
        downloader.start(blocking=False)

        while not downloader.isFinished():
            logger.info('Upload progress: SIZE: {} {} SPEED: {} ETA: {}'.format(
                "{0:.1f}%".format(downloader.get_progress() * 100),
                downloader.get_dl_size(human=True),
                downloader.get_speed(human=True),
                downloader.get_eta(human=True),
            ), url=url, location=location, status=downloader.get_status())
            time.sleep(5)

        if downloader.isSuccessful():
            logger.info('Upload completed',
                        url=url,
                        location=downloader.get_dest(),
                        duration=downloader.get_dl_time(human=True))
        else:
            logger.error('Upload error',
                         url=url,
                         location=downloader.get_dest(),
                         errors=downloader.get_errors())

            errors_text = ''
            for e in downloader.get_errors():
                errors_text = errors_text + "\n" + str(e)

            raise Exception(errors_text)
    except Exception as error:
        file_list = glob.glob('{}*'.format(location))
        for file_path in file_list:
            try:
                os.remove(file_path)
            except:
                logger.error('Error while deleting file', file_path=file_path)
        raise error

LOG:

"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 88.0% 15.06 GB SPEED: 7.1 MB/s ETA: 4 minutes, 50 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:08:20.608056Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 88.2% 15.09 GB SPEED: 7.1 MB/s ETA: 4 minutes, 57 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:08:25.626230Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 88.4% 15.12 GB SPEED: 7.8 MB/s ETA: 4 minutes, 58 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:08:30.666378Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 88.6% 15.15 GB SPEED: 7.5 MB/s ETA: 4 minutes, 39 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:08:35.681668Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 88.8% 15.19 GB SPEED: 7.4 MB/s ETA: 4 minutes, 21 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:08:40.713161Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 89.0% 15.22 GB SPEED: 8.2 MB/s ETA: 4 minutes, 23 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:08:45.718700Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 89.2% 15.25 GB SPEED: 4.8 MB/s ETA: 5 minutes, 1 second", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:08:50.724313Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 89.3% 15.27 GB SPEED: 4.9 MB/s ETA: 6 minutes, 23 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:08:55.729961Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 89.5% 15.30 GB SPEED: 5.5 MB/s ETA: 5 minutes, 54 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:00.735431Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 89.6% 15.32 GB SPEED: 3.6 MB/s ETA: 7 minutes, 59 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:05.740615Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 89.7% 15.34 GB SPEED: 3.9 MB/s ETA: 7 minutes, 56 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:10.774994Z"}
{"event": "ConnectionResetError(104, 'Connection reset by peer')", "level": "exception", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.287198Z"}
NoneType
{"event": "Starting a new SmartDL operation.", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.287604Z"}
{"event": "One URL is loaded.", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.287741Z"}
{"event": "Downloading 'http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014' to '/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b'...", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.287870Z"}
{"event": "Content-Length is 18360254880 (17.10 GB).", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.615813Z"}
{"event": "Launching 10 threads (downloads 1.71 GB/thread).", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.616141Z"}
{"event": "Downloading 'http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014' to '/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b.000'...", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.617045Z"}
{"event": "Downloading 'http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014' to '/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b.001'...", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.617507Z"}
{"event": "Downloading 'http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014' to '/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b.002'...", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.617891Z"}
{"event": "Downloading 'http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014' to '/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b.003'...", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.618336Z"}
{"event": "Downloading 'http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014' to '/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b.004'...", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.618734Z"}
{"event": "Downloading 'http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014' to '/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b.005'...", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.619121Z"}
{"event": "Downloading 'http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014' to '/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b.006'...", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.657019Z"}
{"event": "Downloading 'http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014' to '/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b.007'...", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.657721Z"}
{"event": "Downloading 'http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014' to '/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b.008'...", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.658263Z"}
{"event": "Downloading 'http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014' to '/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b.009'...", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.659037Z"}
{"event": "Control thread has been started.", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.676908Z"}
{"event": "Diff between downloaded files and expected filesizes is 1054262929.0kB.", "level": "warning", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.677279Z"}
{"event": "Starting a new SmartDL operation.", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.677813Z"}
{"event": "One URL is loaded.", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.677947Z"}
{"event": "Downloading 'http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014' to '/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b'...", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.678064Z"}
{"event": "Content-Length is 18360254880 (17.10 GB).", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.760694Z"}
{"event": "Launching 10 threads (downloads 1.71 GB/thread).", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:12.761018Z"}
{"event": "Control thread has been started.", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:13.558547Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 0.2% 27.2 MB SPEED: 17.0 MB/s ETA: 0 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:15.931493Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 0.6% 108.5 MB SPEED: 21.1 MB/s ETA: 14 minutes, 36 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:20.949592Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 1.1% 197.7 MB SPEED: 20.5 MB/s ETA: 14 minutes, 11 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:25.958432Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 1.6% 286.0 MB SPEED: 19.7 MB/s ETA: 14 minutes, 22 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:30.994674Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 2.1% 373.6 MB SPEED: 21.1 MB/s ETA: 14 minutes, 44 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:36.071302Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 2.7% 464.9 MB SPEED: 20.4 MB/s ETA: 14 minutes, 2 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:41.314663Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 3.1% 548.0 MB SPEED: 19.3 MB/s ETA: 15 minutes, 29 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:46.325928Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 3.6% 631.5 MB SPEED: 18.4 MB/s ETA: 15 minutes, 5 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:51.337558Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 4.1% 718.0 MB SPEED: 20.0 MB/s ETA: 14 minutes, 28 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:09:56.537931Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 4.6% 808.3 MB SPEED: 21.2 MB/s ETA: 13 minutes, 51 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:10:01.544831Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 5.1% 892.5 MB SPEED: 19.2 MB/s ETA: 14 minutes, 2 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014", "level": "info", "logger": "content.the_client", "timestamp": "2019-06-22T19:10:06.578321Z"}
{"location": "/tmp/3099cbe6-3496-4e64-9d9e-4aea8a90fe4b", "event": "Upload progress: SIZE: 6.0% 1.03 GB SPEED: 36.9 MB/s ETA: 8 minutes, 57 seconds", "status": "downloading", "url": "http://HOST-OBFUSCATED/2018/ACC/2e9046c1564bb85f46fd97d3ab2c07f6adcbb508.mp4?sv=2014-02-14&sr=b&sig=6iaaT3ZKnuLUbfgDyBi57i0p8AR6el3e%2BKhEM0nw1uI%3D&se=2019-06

Programatically set overwrite to true

Hi, is there any way to disable asking for "file already exists, would you like to overwrite" and have it overwrite whenever a duplicate file was being downloaded?

Support resuming downloads after halt

Couldn't you check for an existing file with the same name < content-lenght and then set the range header with the size of the existing part as start?

ssl verification failed

File "D:\Python37\lib\urllib\request.py", line 1317, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File "D:\Python37\lib\http\client.py", line 1229, in request
self._send_request(method, url, body, headers, encode_chunked)
File "D:\Python37\lib\http\client.py", line 1275, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "D:\Python37\lib\http\client.py", line 1224, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "D:\Python37\lib\http\client.py", line 1016, in _send_output
self.send(msg)
File "D:\Python37\lib\http\client.py", line 956, in send
self.connect()
File "D:\Python37\lib\http\client.py", line 1392, in connect
server_hostname=server_hostname)
File "D:\Python37\lib\ssl.py", line 412, in wrap_socket
session=session
File "D:\Python37\lib\ssl.py", line 850, in _create
self.do_handshake()
File "D:\Python37\lib\ssl.py", line 1108, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)

Add support for passing custom urllib.request.Request object

This would allow for far more granular control of headers, proxies, etc. while not having to implement this functionality in pySmartDL. ~~Would close #30.~~ Sorry, misread documentation.

Gives error even with a stable bandwidth and CPU resources

Gives this error when I use pySmartDL on aws server to download some files from a very stable site.

Exception in thread Thread-37:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.6/dist-packages/pySmartDL/pySmartDL.py", line 680, in post_threadpool_actions
SmartDL_obj.retry(str(pool.get_exceptions()[0]))
File "/usr/local/lib/python3.6/dist-packages/pySmartDL/pySmartDL.py", line 312, in retry
self.start()
File "/usr/local/lib/python3.6/dist-packages/pySmartDL/pySmartDL.py", line 300, in start
self.wait(raise_exceptions=True)
File "/usr/local/lib/python3.6/dist-packages/pySmartDL/pySmartDL.py", line 452, in wait
raise self.errors[-1]
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.6/dist-packages/pySmartDL/pySmartDL.py", line 680, in post_threadpool_actions
SmartDL_obj.retry(str(pool.get_exceptions()[0]))
File "/usr/local/lib/python3.6/dist-packages/pySmartDL/pySmartDL.py", line 312, in retry
self.start()
File "/usr/local/lib/python3.6/dist-packages/pySmartDL/pySmartDL.py", line 300, in start
self.wait(raise_exceptions=True)
File "/usr/local/lib/python3.6/dist-packages/pySmartDL/pySmartDL.py", line 452, in wait
raise self.errors[-1]
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.6/dist-packages/pySmartDL/pySmartDL.py", line 680, in post_threadpool_actions
SmartDL_obj.retry(str(pool.get_exceptions()[0]))
File "/usr/local/lib/python3.6/dist-packages/pySmartDL/pySmartDL.py", line 312, in retry
self.start()
File "/usr/local/lib/python3.6/dist-packages/pySmartDL/pySmartDL.py", line 300, in start
self.wait(raise_exceptions=True)
File "/usr/local/lib/python3.6/dist-packages/pySmartDL/pySmartDL.py", line 452, in wait
raise self.errors[-1]
urllib.error.HTTPError: HTTP Error 0: The maximum retry attempts reached (The read operation timed out)

ssl certificate

I have the download link that has the ssl certificate expired. Can you add an option to bypass ssl certificate verification?

And I would also like PySmartDL to have an option to be able to customize the name of the file.