Comments (3)
I have since tried to DELETE
the resumable upload by crc32
when I destroy my write/validate streams but it also doesn't affect the underlying streams, so the handle stays open.
from nodejs-storage.
Not sure if this is the same behaviour I'm seeing or not.
I'm seeing streams aren't closing when using WebStream apis in Node.
To me, it feels like the same problem, the handler on the web stream side completes correctly, but the event is not emitted on the GCS lib side.
This has the effect of any means of closing the stream does not work... Either destroy or end... Neither actually closes the handle.
from nodejs-storage.
Here's a little example of what I mean.
I tested this on node v20.10.0 using ts-node.
You can get the CSV from here
https://github.com/datablist/sample-csv-files/raw/main/files/people/people-2000000.zip
import { join } from 'path';
import { createReadStream } from 'fs';
import { Writable, Readable } from 'stream'
import { TransformStream } from 'stream/web'
import { Storage } from '@google-cloud/storage'
const _sourcePathname = join(process.cwd(), 'test/people-2000000.csv');
const _storage = new Storage({ keyFilename: './secrets/gcp.json' });
const _bucket = _storage.bucket('journeywise_test_bucket');
const sourceFile = createReadStream(_sourcePathname, { encoding: 'utf-8' })
const _readable = Readable.toWeb(sourceFile);
const destinationFile = _bucket.file('output.csv');
const destinationStream = destinationFile.createWriteStream();
const _writable = Writable.toWeb(destinationStream);
const assertSize = (assertSize: number) => {
let size = 0;
return new TransformStream({
transform(chunk, ctrl) {
size += chunk.length;
ctrl.enqueue(chunk);
},
flush() {
console.log('Closed stream. Size match?', size == assertSize);
}
})
}
_readable
.pipeThrough(assertSize(235121126))
.pipeTo(_writable)
.then(() => console.log('stream completed successfully'))
from nodejs-storage.
Related Issues (20)
- storage object retention lock: should disable object retention on the file failed HOT 1
- NodeError: Cannot call write after a stream was destroyed HOT 9
- storage public access prevention enforced public access prevention behavior: object cannot be made public via ACL failed HOT 2
- Cannot sign data without `client_email`: is there a way to generate a signed url without a private key? HOT 8
- error TS2305: Module '"stream"' has no exported member 'PipelineSource'. HOT 4
- storage requester pays existing bucket methods that accept userProject: file#createReadStream failed HOT 2
- Add Support for includeFolders option in ListObjects HOT 1
- Does not exist in type 'CredentialBody | ExternalAccountClientOptions' HOT 4
- Add support for multipart uploads and/or expose XMLMultiPartUploadHelper HOT 3
- Apparently wrong type of `CopyOptions.metadata` HOT 2
- Downloading file fails when running on Node version 20.11.0 'Iron' (LTS) HOT 9
- storage acls buckets: should make a bucket public failed HOT 3
- node_modules/@google-cloud/storage/build/cjs/src/file.d.ts:7:30 - Module '"stream"' has no exported member 'PipelineSource'. HOT 5
- file file archived generations: should copy file with old versions failed HOT 2
- How to specify proxy directly on the client? HOT 1
- How to Add custom value for the header `x-goog-api` using the module google cloud storage? HOT 2
- refactor!: Revamp `apiEndpoint` HOT 1
- Implement soft delete
- Error: Cannot find module 'node:url' HOT 2
- storage acls buckets: should make a bucket public failed HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nodejs-storage.