Code Monkey home page Code Monkey logo

minio-js's Introduction

MinIO JavaScript Library for Amazon S3 Compatible Cloud Storage Slack

NPM

The MinIO JavaScript Client SDK provides high level APIs to access any Amazon S3 compatible object storage server.

This guide will show you how to install the client SDK and execute an example JavaScript program. For a complete list of APIs and examples, please take a look at the JavaScript Client API Reference documentation.

This document presumes you have a working Node.js development environment, LTS versions v16, v18 or v20.

Download from NPM

npm install --save minio

Download from Source

git clone https://github.com/minio/minio-js
cd minio-js
npm install
npm run build
npm install -g

Using with TypeScript

minio>7.1.0 is shipped with builtin type definition, @types/minio is no longer needed.

Initialize MinIO Client

The following parameters are needed to connect to a MinIO object storage server:

Parameter Description
endPoint Hostname of the object storage service.
port TCP/IP port number. Optional, defaults to 80 for HTTP and 443 for HTTPs.
accessKey Access key (user ID) of an account in the S3 service.
secretKey Secret key (password) of an account in the S3 service.
useSSL Optional, set to 'true' to enable secure (HTTPS) access.
import * as Minio from 'minio'

const minioClient = new Minio.Client({
  endPoint: 'play.min.io',
  port: 9000,
  useSSL: true,
  accessKey: 'Q3AM3UQ867SPQQA43P2F',
  secretKey: 'zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG',
})

Quick Start Example - File Uploader

This example connects to an object storage server, creates a bucket, and uploads a file to the bucket. It uses the MinIO play server, a public MinIO cluster located at https://play.min.io.

The play server runs the latest stable version of MinIO and may be used for testing and development. The access credentials shown in this example are open to the public. All data uploaded to play should be considered public and non-protected.

file-uploader.mjs

import * as Minio from 'minio'

// Instantiate the MinIO client with the object store service
// endpoint and an authorized user's credentials
// play.min.io is the MinIO public test cluster
const minioClient = new Minio.Client({
  endPoint: 'play.min.io',
  port: 9000,
  useSSL: true,
  accessKey: 'Q3AM3UQ867SPQQA43P2F',
  secretKey: 'zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG',
})

// File to upload
const sourceFile = '/tmp/test-file.txt'

// Destination bucket
const bucket = 'js-test-bucket'

// Destination object name
const destinationObject = 'my-test-file.txt'

// Check if the bucket exists
// If it doesn't, create it
const exists = await minioClient.bucketExists(bucket)
if (exists) {
  console.log('Bucket ' + bucket + ' exists.')
} else {
  await minioClient.makeBucket(bucket, 'us-east-1')
  console.log('Bucket ' + bucket + ' created in "us-east-1".')
}

// Set the object metadata
var metaData = {
  'Content-Type': 'text/plain',
  'X-Amz-Meta-Testing': 1234,
  example: 5678,
}

// Upload the file with fPutObject
// If an object with the same name exists,
// it is updated with new data
await minioClient.fPutObject(bucket, destinationObject, sourceFile, metaData)
console.log('File ' + sourceFile + ' uploaded as object ' + destinationObject + ' in bucket ' + bucket)

Run the File Uploader

node file-uploader.mjs
Bucket js-test-bucket created successfully in "us-east-1".
File /tmp/test-file.txt uploaded successfully as my-test-file.txt to bucket js-test-bucket

Verify the object was created with mc:

mc ls play/js-test-bucket
[2023-11-10 17:52:20 UTC]  20KiB STANDARD my-test-file.txt

API Reference

The complete API Reference is available here:

Bucket Operations

File Object Operations

Object Operations

Presigned Operations

Bucket Notification Operations

Bucket Policy Operations

Examples

Bucket Operations

File Object Operations

Object Operations

Presigned Operations

Bucket Notification Operations

Bucket Policy Operations

Custom Settings

Explore Further

Contribute

GitHub Workflow Status

minio-js's People

Contributors

aldy505 avatar barinbritva avatar bexsoft avatar brendanashworth avatar colhodm avatar dcharbonnier avatar deekoder avatar dependabot[bot] avatar djwfyi avatar ebozduman avatar feorlen avatar fluxxu avatar garimakapoor avatar harshavardhana avatar jefvel avatar joaofranciscosantos avatar kaankabalak avatar kanagarajkm avatar kannappanr avatar koolhead17 avatar krishnasrinivas avatar loremaps avatar minio-trusted avatar nitisht avatar nomagick avatar poornas avatar prakashsvmx avatar trim21 avatar vadmeste avatar will3942 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

minio-js's Issues

Port missing from host header in signature v4

host:localhost
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20150714T052457Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
4894da28173885ba7688f2052b5fe936dc9c3effd2d88dcd1bc2cf91fe39ae2c 20ce0726d9529d133efcd2ab48310a2103daeac8958c379bc512935c7eb033ff

Make minio-js library isomorphic

Over the course of last couple of years, majority of the libraries in npmjs are being used for both server and client alike with a single code base.

We need to explore this possibility for minio-js in the near future.

multipart put with unicode throws an error

Error: Uncaught, unspecified "error" event.
at Error (native)
at DestroyableTransform.emit (events.js:87:13)
at done (/home/fkautz/src/minio-js/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:195:19)
at /home/fkautz/src/minio-js/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:136:9
at /source/minio.js:354:22
at /source/xml-parsers.js:67:16
at ConcatStream. (/home/fkautz/src/minio-js/node_modules/concat-stream/index.js:36:43)
at ConcatStream.emit (events.js:129:20)
at finishMaybe (/home/fkautz/src/minio-js/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:460:14)
at endWritable (/home/fkautz/src/minio-js/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:469:3)

put object does not url encode key

client.putObject(bucket, 'small/o?ne', 'text/plain', 11, stream, function(e) {

results in small/o

client.putObject(bucket, 'small/t#wo', 'text/plain', 11, stream, function(e) {

results in SignatureDoesNotMatch

client.putObject(bucket, 'small/世界', 'text/plain', 11, stream, function(e) {

results in SignatureDoesNotMatch

Figure out and fix all FIXMEs

  // FIXME: the below line causes weird failure in 'gulp test'
    // req.on('error', e => cb(e))
    req.end()
  }

 src/main/transformers.js:  // FIXME: cb should provide proper error object instead of string

Method not allowed on putObject() should be handled for minio servers

  method: null,
  statusCode: 405,
  statusMessage: 'Method Not Allowed',
  client: 
$ node put-object.js 
Error: Uncaught, unspecified "error" event.
    at Error (native)
    at DestroyableTransform.emit (events.js:87:13)
    at done (/home/harsha/repos/minio-js/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:195:19)
    at /home/harsha/repos/minio-js/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:136:9
    at /source/minio.js:365:15
    at /source/xml-parsers.js:67:16
    at ConcatStream.<anonymous> (/home/harsha/repos/minio-js/node_modules/concat-stream/index.js:36:43)
    at ConcatStream.emit (events.js:129:20)
    at finishMaybe (/home/harsha/repos/minio-js/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:460:14)
    at afterWrite (/home/harsha/repos/minio-js/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:340:3)

convert minio-js to a stateless library

Convert the APIs to follow this style:

        config := minio.Config{
                AccessKeyID:     "YOUR-ACCESS-KEY-HERE",
                SecretAccessKey: "YOUR-PASSWORD-HERE",
        }

        // Default is Signature Version 4. To enable Signature Version 2 do the following.
        // config.Signature = minio.SignatureV2

        reader, stat, err := minio.GetObject(config, "https://s3.amazonaws.com/bucket/object") {
        if err != nil {
                log.Fatalln(err)
        }  

Uncaught exception on localhost:9000

$ node list-buckets.js 
undefined
Error: Uncaught, unspecified "error" event.
    at Error (native)
    at Readable.emit (events.js:87:13)
    at /source/minio.js:165:18
    at /source/xml-parsers.js:59:16
    at ConcatStream.<anonymous> (/home/harsha/node_modules/minio/node_modules/concat-stream/index.js:36:43)
    at ConcatStream.emit (events.js:129:20)
    at finishMaybe (/home/harsha/node_modules/minio/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:499:14)
    at afterWrite (/home/harsha/node_modules/minio/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:378:3)
    at afterTick (/home/harsha/node_modules/minio/node_modules/concat-stream/node_modules/readable-stream/node_modules/process-nextick-args/index.js:11:8)
    at process._tickCallback (node.js:355:11)

No buckets on the server yet.

$ node list-buckets.js 
$

One out of two attempts leads to correct output.

PermanentRedirect error seen for unauthenticated requests to s3.amazonaws.com, we might have to turn off redirection like other libraries

$ node drop-incomplete-upload.js
{ requestId: null,
  code: 'PermanentRedirect',
  message: 'The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.',
  requestid: 'DD74B72BA25D3B5C',
  hostid: 'jsefjSYiZZvfXurrpBWUWLHt6BvT8wccslEm7FicQpo3iojP/YTlXq+pe87w9QGT' }
$

Similary for GetObject(), DropAllIncompleteUploads() - cannot reproduce on golang, python and java libraries.

When one doesn't have secret access keys specified with s3.amazonaws.com, we get an unspecified error event.

$ node list-buckets.js
undefined
Error: Uncaught, unspecified "error" event.
    at Error (native)
    at Readable.emit (events.js:87:13)
    at /source/minio.js:169:18
    at /source/xml-parsers.js:59:16
    at ConcatStream.<anonymous> (/home/harsha/repos/minio-js/node_modules/concat-stream/index.js:36:43)
    at ConcatStream.emit (events.js:129:20)
    at finishMaybe (/home/harsha/repos/minio-js/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:460:14)
    at endWritable (/home/harsha/repos/minio-js/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:469:3)
    at ConcatStream.Writable.end (/home/harsha/repos/minio-js/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:436:5)
    at IncomingMessage.onend (_stream_readable.js:505:10)
$

I know we have a list of these already from minio-py, noting this down here so that we don't forget.

validateBucketName should work for all Amazon S3 compatible bucket names

Following regex

https://github.com/minio/minio-js/blob/master/src/main/helpers.js#L79

  // starts and ends with letters or numbers
  var re1 = /^[a-z0-9]+[a-z0-9\-]*[a-z0-9]+$/
  if (bucket.match(re1) === null) {
    return false
  }

Is incomplete in handling all S3 compatible bucket names.

Follow Golang implementation.

func isValidBucket(bucket string) bool {
        if len(bucket) < 3 || len(bucket) > 63 {
                return false
        }
        if bucket[0] == '.' || bucket[len(bucket)-1] == '.' {
                return false
        }
        if match, _ := regexp.MatchString("\\.\\.", bucket); match == true {
                return false
        }
        // We don't support buckets with '.' in them
        match, _ := regexp.MatchString("^[a-zA-Z][a-zA-Z0-9\\-]+[a-zA-Z0-9]$", bucket)
        return match
}

Fix type errors for our custom exceptions.

 node remove-incomplete-upload.js

/source/minio.js:470
      throw new errors.isValidBucketNameException('Invalid bucket name: ' + bucket)
                ^
TypeError: Cannot read property 'isValidBucketNameException' of undefined
    at Client.removeIncompleteUpload (/source/minio.js:470:17)
    at Object.<anonymous> (/Users/harsha/Repositories/minio-js/dist/main/remove-incomplete-upload.js:28:10)
    at Module._compile (module.js:434:26)
    at Object.Module._extensions..js (module.js:452:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Function.Module.runMain (module.js:475:10)
    at startup (node.js:117:18)
    at node.js:951:3
$ node remove-incomplete-upload.js

/source/minio.js:476
      throw new errors.InvalidObjectNameException('Object name cannot be empty')
                ^
TypeError: Cannot read property 'InvalidObjectNameException' of undefined
    at Client.removeIncompleteUpload (/source/minio.js:476:17)
    at Object.<anonymous> (/Users/harsha/Repositories/minio-js/dist/main/remove-incomplete-upload.js:28:10)
    at Module._compile (module.js:434:26)
    at Object.Module._extensions..js (module.js:452:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Function.Module.runMain (module.js:475:10)
    at startup (node.js:117:18)
    at node.js:951:3

listObjects() exception on non-existent bucket

$ node list.js 
Error: Uncaught, unspecified "error" event.
    at Error (native)
    at DestroyableTransform.emit (events.js:87:13)
    at DestroyableTransform.onerror (_stream_readable.js:556:12)
    at DestroyableTransform.emit (events.js:107:17)
    at onwriteError (/home/harsha/repos/minio-js/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:250:10)
    at onwrite (/home/harsha/repos/minio-js/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:268:5)
    at WritableState.onwrite (/home/harsha/repos/minio-js/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:106:5)
    at afterTransform (/home/harsha/repos/minio-js/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:104:5)
    at TransformState.afterTransform (/home/harsha/repos/minio-js/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:79:12)
    at /home/harsha/repos/minio-js/test/minio.js:462:20

Improve our error messages for all exceptions.

// set minimum/maximum length of what Content-Length can be
+  setContentLength(min, max) {
+    if (min > max) {
+      throw new errrors("min can not be more than max")
+    }
+    if (min < 0) {
+      throw new errors("min has to be > 0")
+    }
+    if (max < 0) {
+      throw new errors("max has to be > 0")
+    }
+    this.policy.conditions.push(["content-length-range", min, max])
+  }

For invalid argument, lets add 'InvalidArgumentException'

makeBucket payload object is unnecessary for us-east and milkyway

Sending data with only "xmlns" header is not necessary and we should avoid it. Since the same data is being sent to Minio server too.

value: function makeBucket(bucket, cb) {                                                                                           
       if (!helpers.validBucketName(bucket)) {                                                                                          
         return cb('invalid bucket name');                                                                                              
       }                                                                                                                                

       var region = helpers.getRegion(this.params.host);                                                                                
       if (region === 'milkyway' || region === 'us-east-1') {                                                                           
         region = null;                                                                                                                 
       }                                                                                                                                
       var createBucketConfiguration = [];                                                                                              
       createBucketConfiguration.push({                                                                                                 
         _attr: {                                                                                                                       
           xmlns: 'http://s3.amazonaws.com/doc/2006-03-01/'                                                                             
         }                                                                                                                              
       });                                                                                                                              
       if (region) {                                                                                                                    
         createBucketConfiguration.push({                                                                                               
           LocationConstraint: helpers.getRegion(this.params.host)                                                                      
         });                                                                                                                            
       }                                                                                                                                
       var payloadObject = {                                                                                                            
         CreateBucketConfiguration: createBucketConfiguration                                                                           
       };                                                                                                                               

       var payload = Xml(payloadObject); 

python, golang make_bucket do not send this additional header

All arguments for all functions should have native types

For example in Golang in PresignedGetObject

func (a API) PresignedGetObject(bucket, object string, expires time.Duration) (string, error) {

Expires argument is time.Duration which is a native type under Golang ecosystem, we should make sure that wherever possible we should take native types.

Received an exception when attempting to download object

 client.getObject(bucket, 'hello', function(e, r) {
   console.log(e)
   r.pipe(process.stdout)
 })
/source/minio.js:694
    return cb({
           ^
TypeError: Cannot read property 'pipe' of undefined
    at /home/fkautz/src/test-minio-js/index.js:86:5
    at parseError (/source/minio.js:694:12)
    at ClientRequest.<anonymous> (/source/minio.js:409:16)
    at ClientRequest.g (events.js:199:16)
    at ClientRequest.emit (events.js:107:17)
    at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:419:21)
    at HTTPParser.parserOnHeadersComplete (_http_common.js:111:23)
    at TLSSocket.socketOnData (_http_client.js:310:20)
    at TLSSocket.emit (events.js:107:17)
    at readableAddChunk (_stream_readable.js:163:16)

endPoint is called url

var s3client = new Minio({
  url: 'https://<your-s3-endpoint>',
  accessKey: 'YOUR-ACCESSKEYID',
  secretKey: 'YOUR-SECRETACCESSKEY'
})

should be

var s3client = new Minio({
  endPoint: 'https://<your-s3-endpoint>',
  accessKey: 'YOUR-ACCESSKEYID',
  secretKey: 'YOUR-SECRETACCESSKEY'
})

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.