Comments (4)
Hello again,
I have looked some more at using 'response.raw'. As mentioned above you may end up with zip'ed files when setting 'stream=True' and using 'response.raw'. However, it turns out that the 'read()' method has an optional parameter 'decode_content' that can be set to True to have the response decoded (unzipped in this case):
response.raw.read(decode_content=True)
So the following code are using 'stream=True', and response.raw (with response.raw.read(decode_content=True) called in a loop to copy all data).
The documentation on .read() states that you can not specify an amount to read and at the same time set 'response.raw.read(decode_content=True)'. So this still leaves the question about the ability to read (very) large files. I have tried reading files up to about 600MB and I can see that the file is transferred in a single call to .read() (the while loop is only passed one). So, how to transfer really large files?
Jørgen
--- /home/jojo/Downloads/easywebdav-1.0.7/easywebdav/client.py 2012-11-13 23:31:47.000000000 +0100 +++ /usr/local/lib/python2.7/dist-packages/easywebdav-1.0.7-py2.7.egg/easywebdav/client.py 2013-02-24 09:51:50.638723255 +0100 @@ -74,7 +74,7 @@ self.session.auth = (username, password) def _send(self, method, path, expected_code, **kwargs): url = self._get_url(path) - response = self.session.request(method, url, allow_redirects=False, **kwargs) + response = self.session.request(method, url, allow_redirects=False, stream=True, **kwargs) if isinstance(expected_code, Number) and response.status_code != expected_code \ or not isinstance(expected_code, Number) and response.status_code not in expected_code: raise OperationFailed(method, path, expected_code, response.status_code) @@ -124,7 +124,12 @@ response = self._send('GET', remote_path, 200) with open(local_path, 'wb') as f: #f.write(response.content) - shutil.copyfileobj(response.raw, f) + #shutil.copyfileobj(response.raw, f) + line = response.raw.read(decode_content=True) + while line: + f.write(line) + line = response.raw.read(decode_content=True) + def ls(self, remote_path='.'): headers = {'Depth': '1'} response = self._send('PROPFIND', remote_path, (207, 301), headers=headers)
read(): http://urllib3.readthedocs.org/en/latest/helpers.html#module-urllib3.response
http://docs.python-requests.org/en/latest/user/advanced/
http://docs.python-requests.org/en/latest/api/
from easywebdav.
I'm going to release v1.0.8 after a really long while - please let me know if the problem persists with the new version.
from easywebdav.
Works for me with
requests==2.7.0
easywebdav==1.2.0
python==2.7.10
from easywebdav.
This worked for me:
response.raw.decode_content = True
shutil.copyfileobj(response.raw, out_file)
because
the response.raw file-like object will not, by default, decode compressed responses (with GZIP or deflate). You can force it to decompress for you anyway by setting the decode_content attribute to True
ref:
https://www.codementor.io/tips/3443978201/how-to-download-image-using-requests-in-python
from easywebdav.
Related Issues (20)
- 1.0.8 HOT 1
- NameError: global name 'basestring' is not defined (python 3) HOT 4
- null valued contentlength prop on zope
- OneDrive returns 302 Found on PROPFIND HOT 2
- please tag releases
- Implement move and copy methods HOT 4
- Error on performing file operations HOT 4
- What is "safe=False"?
- Requesting ls() with colon ':' in path
- How to get/put a whole dir
- Running upload() on python 3 fails HOT 3
- mkdirs() fails on some webdav servers
- ls() fails on Nextcloud server
- Python 3 Compatibility (basestring is deprecated)
- Solution MacOS python3 basestring HOT 2
- Not support unicode path
- can i access file with unc path?
- Have a solution to keep the sort of the html's content when I use ls() command ?
- Timestamp not preserved in downloaded file HOT 3
- Python3 adaptions
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from easywebdav.