franciscop / backblaze Goto Github PK

We are using the https://www.npmjs.com/package/backblaze-b2 API behind this library, but making it more abstract and hopefully easier to manage your files with backblaze. This means to upload a file, you just need to do bucket.upload(localFileName), but so far it only works with normal files.

Feature request: allow bucket.upload(localFileName) to also accept large files for upload and use b2's APIs for that internally:

• startLargeFile()
• getUploadPartUrl()
• uploadPart()
• finishLargeFile()

You might want to check the file size prior to this with a fs.stat() to check the size first.

The API of bucket.upload(localFileName) should remain the same as much as possible, so when the end user does bucket.upload() they should not care whether it's a small or a large file.

Let's say you have a data.json in your bucket and you want to read it. Right now you need to do:

await bucket.download('./data.json');
const raw = await fs.readFile('./data.json');
const data = JSON.parse(raw);

Feature request: create a new method called bucket.read(remoteFileName) that will read the remote file straight into a variable, so that the above can be simplified to this:

// Note it's now called `.read()`
const raw = await bucket.read('./data.json');
const data = JSON.parse(raw);

Tip: if it cannot be read directly, you can try using a temp directory to download the file and then read it.

Working with folders could/should be made easier. For this, I propose this API:

Simply define a folder to upload the file:

// Relative uncomplete path
await bucket.upload(myFile, '/profile/');
// Output: { name: "/profile/3nNLR9AUdgjRWrVd.png", ... }

// List the given folder
await bucket.list('/profile/');

// Maybe? Remove a full folder
await bucket.remove('/profile/');

// Maybe? Download a full folder
await bucket.download('/profile/');
// Will download into `./profile/`

Later on, a full namespace could be done, but for now I think it's best to do only the proposed above:

// Use it concatenating with a `.folder()`
bucket.folder('profile').upload(myFile);
// Output: `/profile/3nNLR9AUdgjRWrVd.png`

// Create a separated variable to handle it
const profileBucket = bucket.folder('profile');
profileBucket.upload(myFile);
// Output: `/profile/3nNLR9AUdgjRWrVd.png`

Allow to pipe data to upload:

const bucket = Bucket();

// Option A:
fs.createReadStream('./demo.png')
  .pipe(bucket.upload)
  .on('end', file => {
    console.log(file);
  });

// Option B:
const file = await bucket.upload(fs.createReadStream('./demo.png'));

Note: I prefer option B if possible

When uploading a file, it autogenerates the name of the file. Or you can specify the full path of the file to name. Both are fine, but it'd be nice if you could also specify the "folder" to upload and it still autogenerates the name. Example:

// Current way
bucket.upload(file.path);  // kwergvckwsdb.png

// Proposed alternative, a path ending with a slash to denote it's a folder,
// but still autogenerate the filename
bucket.upload(file.path, "profile/");  // profile/kwergvckwsdb.png

If you restrict the key generated to a single bucket, even if it's the one that you want, fails:

...
    data: { code: 'unauthorized', message: '', status: 401 }
...

Probably because of this line:

      const list = await b2.listBuckets();
      const raw = list.data.buckets.find((b) => b.bucketName === name);
      bucket = { ...raw, baseURL };

So we should find a new way of getting the current bucket info instead of relying picking the right one out of a global list.