Code Monkey home page Code Monkey logo

skipper's Introduction

Skipper

Streaming Multipart File Upload Parsing

Skipper is an opinionated variant of Connect's body parser designed to support streaming upload of monolithic files to a compatible blob receiver, while still allowing application code to run in a timely manner; without writing .tmp files to disk.

This module may or may not be included as a part of the stable release of Sails v0.10-- need more documentation, examples, and "receivers" (currently receivers for S3 and local disk exist.)

Usage

With Sails

Install it with npm using the --save flag. This will automatically add it to your app's package.json file.

npm install --save

Skipper intends to be a drop-in replacement for the Connect Body Parser which Sails uses by default (via Express.js).

Therefore, the next step is to disable it and hook up Skipper.

Change Config

Do this by adding the line bodyParser: require('skipper') to the express object in /myApp/config/express.js. It should look something like this.

/**
 * Configure advanced options for the Express server inside of Sails.
 *
 * For more information on configuration, check out:
 * http://sailsjs.org/#documentation
 */

module.exports.express = {

 // ... much comment.  so amaze. wow
 
	// Defaults to a slightly modified version of `express.bodyParser`, i.e.:
	// If the Connect `bodyParser` doesn't understand the HTTP body request 
	// data, Sails runs it again with an artificial header, forcing it to try
	// and parse the request body as JSON.  (this allows JSON to be used as your
	// request data without the need to specify a 'Content-type: application/json'
	// header)
	// 
	// If you want to change any of that, you can override the bodyParser with
	// your own custom middleware:
	bodyParser: require('skipper')

};


/**
 * HTTP Flat-File Cache
 * 
 * These settings are for Express' static middleware- the part that serves
 * flat-files like images, css, client-side templates, favicons, etc.
 *
 * ... more comments ...
 */
module.exports.cache = {

	// The number of seconds to cache files being served from disk
	// (only works in production mode)
	maxAge: 31557600000
};
Create an API

Now that it's hooked up, we need to generate a new api for serving/storing the files. Do this using the sails command line tool.

dude@littleDude:~/node/myApp$ sails generate api file

debug: Generated a new controller `file` at api/controllers/FileController.js!
debug: Generated a new model `File` at api/models/File.js!

info: REST API generated @ http://localhost:1337/file
info: and will be available the next time you run `sails lift`.

dude@littleDude:~/node/myApp$ 
Write Controller Actions

Refer to myApp/api/controllers/FileController.js below and modify it as you see fit.

// myApp/api/controllers/FileController.js

/**
 * FilesController.js 
 *
 * @description ::
 * @docs        :: http://sailsjs.org/#!documentation/controllers
 */

module.exports = {

  index: function (req,res){

    res.writeHead(200, {'content-type': 'text/html'});
    res.end(
      '<form action="http://localhost:1337/files/upload" enctype="multipart/form-data" method="post">'+
      '<input type="text" name="title"><br>'+
      '<input type="file" name="avatar" multiple="multiple"><br>'+
      '<input type="submit" value="Upload">'+
      '</form>'
    )
  },
  upload: function (req,res){

var Writable = require('stream').Writable;
var fs = require('fs');

    function newReceiverStream (options) {
        options = options || {};

        // Default the output path for files to `/dev/null` if no `id` option
        // is passed in (for testing purposes only)
        var filePath = options.id || '/dev/null';

        var receiver__ = Writable({objectMode: true});

        // This `_write` method is invoked each time a new file is received
        // from the Readable stream (Upstream) which is pumping filestreams
        // into this receiver.  (filename === `__newFile.filename`).
        receiver__._write = function onFile (__newFile, encoding, done) {
        var outs = fs.createWriteStream(filePath, encoding);
        __newFile.pipe(outs);

        // Garbage-collect the bytes that were already written for this file.
        // (called when a read or write error occurs)
        function gc (err) {
          console.log('************** Garbage collecting file `'+__newFile.filename+'` located @ '+filePath+'...');
          fs.unlink(filePath, function (gcErr) {
            if (gcErr) return done([err].concat([gcErr]));
            return done(err);
          });
        }


        __newFile.on('error', function (err) {
          console.log('***** READ error on file '+__newFile.filename, '::',err);
        });
        outs.on('error', function failedToWriteFile (err) {
        	console.log('OH THE BITS',__newFile,'enc',encoding,'dun',done);
          gc(err);
        });

        outs.on('finish', function successfullyWroteFile () {
          done();
        });

        };

        return receiver__;
    }
    
var streamOptions = {id:'/home/dude/node/fileUploadExample/assets/images/shitBird.jpeg'};
    req.file('avatar').upload( newReceiverStream(streamOptions) , function (err, files) {
          if (err) return res.serverError(err);
        
          res.json({
            message: files.length + ' file(s) uploaded successfully!',
            files: files
          });

        });
  }

};
With Express

You're on your own, homie... nah, it's comin'

Status

Currently, this project is in an openly released alpha stage, but under active development.

More Resources

License

MIT © 2014 Mike McNeil, Scott Gress, Balderdash & contributors

Sails is free and open-source under the MIT License.

image_squidhome@2x.png

githalytics.com alpha

skipper's People

Contributors

mikermcneil avatar sgress454 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.