jkeifer / drf-chunked-upload Goto Github PK
View Code? Open in Web Editor NEWModule for django-rest-framework to enable uploads using chunks via REST.
License: Other
Module for django-rest-framework to enable uploads using chunks via REST.
License: Other
Hi, when would the code in the ci-test
branch be merged into master
? No rush, but it's been nearly a month and I was wondering if the code would make it to a stable release. Thanks!
Currently the python manage.py migrate
command will not create a table for the ChunkedUpload
model, even if the abstract setting is set to false.
Possibly related: https://docs.djangoproject.com/en/3.2/topics/migrations/#dependencies
The EXT of the file has been changed. Is it necessary to change the EXT back when downloading
Hi, I've set up this module with a project I'm working on and am trying to upload a file using a Python script (relevant portion attached below):
with open(file, 'rb') as file_raw:
while chunk := file_raw.read(chunk_size):
r = requests.put(upload_url, headers={
"Authorization": "Token {}".format(token),
"Content-Range": "bytes {}-{}/{}".format(current_chunk * chunk_size + 1,
current_chunk * chunk_size + len(chunk),
total_file_size)
},
data={'file': chunk})
However, I'm getting an "AttributeError: 'str' object has no attribute 'size'" response. The trace points to the following code snippet:
I'm guessing the error is occurring because the chunk
object is being read in as a string and not in binary, so there is no size attribute. I don't know why the file is being read as a string; is there anything to fix in my code? Any help would be appreciated!
Using the basic setup:
from drf_chunked_upload.views import ChunkedUploadView
class MyChunkView(ChunkedUploadView):
pass
I'm hitting a NoReverseMatch
error. This seems to be cause on the serializer.py
where it tries to to find a chunkedupload-detail
URL and doesn't seem to find the one in tests.py
(assuming this is the reverse target). I've had to get around it just creating my own url and view e.g.
urls.py
path(
"uploadchuncks/<uuid:pk>/",
ChunkedUploadDetailView.as_view(),
name="chunkedupload-detail",
),
views.py
class ChunkedUploadDetailView(RetrieveAPIView):
queryset = ChunkedUpload.objects.all()
serializer_class = ChunkedUploadSerializer
Curious if anyone else is experiencing this?
Packages:
Django==3.2.16
'Cannot use ModelSerializer with Abstract Models.'
ValueError: Cannot use ModelSerializer with Abstract Models.
When deleting a ChunkedUpload object, the deletion fails because the deletion function continuously calls itself.
It only occurs when calling the .delete()
function on a ChunkedUpload object (i.e. chunked_upload.delete()
) or if a single ChunkedUpload object is deleted in the admin panel. For some reason it doesn't occur when bulk-deleting (i.e. checking each item in the ChunkedUpload table and delete using bulk actions).
When the upload is in progress, the filename is the upload UUID with the extension .part
. Once the upload finishes, the filename should be renamed to match the filename sent by the user.
I tried to set the DRF_CHUNKED_UPLOAD_STORAGE_CLASS
to S3Boto3Storage
(from django-storage
).
# drf-chunked-upload Config
# https://github.com/jkeifer/drf-chunked-upload
DRF_CHUNKED_UPLOAD_ABSTRACT_MODEL = False
from storages.backends.s3boto3 import S3Boto3Storage
DRF_CHUNKED_UPLOAD_STORAGE_CLASS = S3Boto3Storage
But, I got an error while "uploading the second chunk"
AttributeError: File was not opened in write mode.
Internal Server Error: /api/v1/polls/files/4fcb004a-4155-4a19-9822-8fe6a8f3f1f7/
Traceback (most recent call last):
File "/home/jpg/.local/share/virtualenvs/generic-django-example-Kc6WXfau/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/home/jpg/.local/share/virtualenvs/generic-django-example-Kc6WXfau/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/jpg/.local/share/virtualenvs/generic-django-example-Kc6WXfau/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/home/jpg/.local/share/virtualenvs/generic-django-example-Kc6WXfau/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
return self.dispatch(request, *args, **kwargs)
File "/home/jpg/.local/share/virtualenvs/generic-django-example-Kc6WXfau/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch
response = self.handle_exception(exc)
File "/home/jpg/.local/share/virtualenvs/generic-django-example-Kc6WXfau/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception
self.raise_uncaught_exception(exc)
File "/home/jpg/.local/share/virtualenvs/generic-django-example-Kc6WXfau/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
raise exc
File "/home/jpg/.local/share/virtualenvs/generic-django-example-Kc6WXfau/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch
response = handler(request, *args, **kwargs)
File "/home/jpg/.local/share/virtualenvs/generic-django-example-Kc6WXfau/lib/python3.9/site-packages/drf_chunked_upload/views.py", line 59, in put
return self._put(request, pk=pk, *args, **kwargs)
File "/home/jpg/.local/share/virtualenvs/generic-django-example-Kc6WXfau/lib/python3.9/site-packages/drf_chunked_upload/views.py", line 221, in _put
chunked_upload = self._put_chunk(request, pk=pk, *args, **kwargs)
File "/home/jpg/.local/share/virtualenvs/generic-django-example-Kc6WXfau/lib/python3.9/site-packages/drf_chunked_upload/views.py", line 194, in _put_chunk
chunked_upload.append_chunk(chunk, chunk_size=chunk_size)
File "/home/jpg/.local/share/virtualenvs/generic-django-example-Kc6WXfau/lib/python3.9/site-packages/drf_chunked_upload/models.py", line 106, in append_chunk
self.file.write(subchunk)
File "/home/jpg/.local/share/virtualenvs/generic-django-example-Kc6WXfau/lib/python3.9/site-packages/storages/backends/s3boto3.py", line 168, in write
raise AttributeError("File was not opened in write mode.")
AttributeError: File was not opened in write mode.
Packages used
django~=3.2.0
drf-chunked-upload~=0.5.1
django-storages[boto3]~=1.13.2
Need to add something to the base view that is something like response_serializer_class
and make it the same as the serializer_class
by default.
I'm getting the following response from drf-chunked-upload:
{'detail': 'Offsets do not match', 'offset': 55000000}
The weird thing is, I am sending a chunk with the content range header start set to the offset provided:
{'Authorization': 'Token [redacted]', 'Content-Range': 'bytes 55000000-55999999/1733612877'}
I'm guessing an off-by-one mistake here on my part, but the code clearly compares the "start" regex group of the Content-Range header and the offset stored in the incomplete chunked upload:
drf-chunked-upload/src/drf_chunked_upload/views.py
Lines 188 to 192 in 9d5bd7f
Any idea why this would be occurring?
As it says, add support for django 1.8.x UUID field, but keep current compatibility but checking for UUID field and gracefully falling back to current implementation.
hey @jkeifer ! I'm implementing a restful chunked upload, and hoping to give your module a try! Do you have a dummy example to go from?
I'm testing the module.
But I can't makemigrations
after list the module in INSTALLED_APPS
and define
DRF_CHUNKED_UPLOAD_ABSTRACT_MODEL = False
in project settings.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.