Comments (11)
Ok I figured out the issue, resumable uploads is fully supported without companion - apologies for my false claims! Even more amazingly, it is supported exactly how I wanted it! (we locally store the successfully uploaded chunks within a file descriptor object and call listParts to compare!)
The issue was from my end, for my listParts endpoint I was returning a object containing the array! I went so far as to locally develop uppy haha - this was where I saw that yes, the companion code from above was being run but it was not the source of my error.
After two companion code blocks, it would go on to the listParts I mentioned earlier, it expects an array for the .find method to work and hence this was the falling point. Will update the code so it can be a good starting point for people in the future!
Thanks for your patience and apologies for any confusion! π
from uppy.
This has been fixed: #4526
When using tus, you may not need Golden Retriever. The difference is after a browser refresh the user as to select the file again with tus and if the file fits in storage, it will be automatically loaded by Golden Retriever.
from uppy.
Checkout this option, it will likely fix your issue: https://uppy.io/docs/aws-s3-multipart/#shouldusemultipartfile
from uppy.
I'm not sure this is the place for Next.js specific issues. But if you have a reproducible example in codesandbox/stackblitz, then we may be able to help.
from uppy.
@uppy/golden-retriever
and uppy/aws-s3
(-multipart
) can work together yes.
By this I mean, will uppy golden retriever not store the file but store successfully uploaded chunks for me?
It stores the files, it's up to the uploader to decide what needs to be uploaded.
from uppy.
Apologies didn't see #4269 (comment)
Any plans to implement S3 support? Would be helpful if docs were clearer on which uploader golden retriever is actually intended for?
From what I just read, TUS innately supports resumable uploads, so if we were using TUS would golden retriever even be needed?
from uppy.
Thanks for the speedy response, appreciate the help!
My example wasn't great, my project is using S3-multipart and I tried integrating Golden Retriever last night. I will update the repo to test out with dashboard shortly!
I would like to achieve Resumable Uploads with S3 Multipart and looking at #2121 it looks like there has been some progress.
I was looking at the source code of multipart uploader to see why it may not be working in my project, and this line (201) of code stood out to me:
} else if (this.#isRestoring) { this.options.companionComm.restoreUploadFile(this.#file, { uploadId: this.options.uploadId, key: this.options.key }) this.#resumeUpload() }
Is companion required to support resumable S3 multipart uploads?
As users may be uploading large files, I do not want them to be stored with GR, what I would prefer is that successfully uploaded chunk metadata is stored instead, so if the same file is uploaded, we can pick up chunk metadata from where we left.
from uppy.
Ok I just updated the example code to include multipart and golden retriever, made into a hook like in my project.
The issue persists, something is going wrong, even when I try to pause and resume a upload - I think nextjs is bundling stuff funny.
You can test the example with 20mb files as I set the chunk size as 5mb.
There are two scenarios when we refresh the page 50% through upload:
-
The ghost file appears, if we try to continue, there is a crash - I don't think this is from uppy though, I think its next 14 "webpack" / turbopack
-
We try to a fresh upload with the same file, the upload starts from the beginning!
I have put the sw.js in my next public folder, so this should bypass webpack bundling. I have also done a dynamic import of Golden retriever so it should be available as required in the sw.js client?
from uppy.
Ofcourse, I just wanted to know if resumable multipart s3 uploads is supported, before I bother tackling the nextjs specific issues?!
By this I mean, will uppy golden retriever not store the file but store successfully uploaded chunks for me?
from uppy.
Ok I made some good progress!
I ran a react & express server with Golden Retriever, without the service worker. For me not to store the files on client, this is necessary.
This time the ghost file appeared and then I had a listParts problem, so when using golden retriever to reupload a previously uploaded file, listParts is called, this is good!
When I fixed this, I got back to this
This error comes from here, and because my companion is undefined, everything following is undefined. So a companion is required to support resumable uploads!
function _resumeUpload2() {
this.options.companionComm.resumeUploadFile(_classPrivateFieldLooseBase(this, _file)[_file], _classPrivateFieldLooseBase(this, _chunks)[_chunks], _classPrivateFieldLooseBase(this, _abortController)[_abortController].signal).then(_classPrivateFieldLooseBase(this, _onSuccess)[_onSuccess], _classPrivateFieldLooseBase(this, _onReject)[_onReject]);
}
Will try with a companion next haha! thanks, so I don't think its a next issue! but a companion problem! Will learn more about this!
from uppy.
I would just like to add that for me to resume a multipart s3 upload, I don't believe a fully fledged companion server should be a necessity!
Surely it should be possible to continue an upload from storing client side which chunks were successfully sent? We could call listParts to verify it is a valid upload!
Many thanks to the Uppy team regardless! I appreciate what you have built!!
from uppy.
Related Issues (20)
- Version 3.22 Invalid target option given to Dashboard:StatusBar. HOT 6
- File checksum not matching after uploading with XHR HOT 4
- Compressor - Progress event or built in progress bar HOT 1
- ImageUploader - duplicates itself in DOM when using Uppy/react Dashboard HOT 5
- Typescript cannot detect UppyFile error property HOT 1
- @uppy/xhr-upload: upload fails if server returns 204 empty
- @uppy/tus: Undocumented dependency on Nodeβs Buffer API HOT 4
- Option to customize rendering of item names in dashboard (virtual scroll prevents doing it manually) HOT 1
- Dashboard - Option to increase number of selected images visible per row HOT 2
- Confirm Delete for Files
- Failed to parse source map from @transloadit\prettier-bytes\src\prettierBytes.ts HOT 7
- autoOpenFileEditor option is ignored (broken?) past @uppy/dashboard 3.7.4 and @uppy/react HOT 16
- Total Progress incorrectly reflected when keeping files in state. HOT 4
- Unsplash doesn't load further pages
- aws-s3-multipart: Self-signed request does NOT escape path according to RFC 3986 HOT 2
- Create well-defined error codes HOT 1
- "Link" is not translated (Url module) HOT 2
- Plugin titles are sometimes configured through options, something in locale
- Tus resumes unrelated uploads, causing conflicts or lost metadata HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from uppy.