Code Monkey home page Code Monkey logo

combohandler's Introduction

Combo Handler

Build Status

This is a simple combo handler for Node.js, usable either as Connect middleware or as an Express server. It works just like the combo handler service on the Yahoo! CDN, which you'll be familiar with if you've used YUI.

The combo handler is compatible with the YUI Loader, so you can use it to host YUI, or you can use it with any other JavaScript or CSS if you're willing to construct the combo URLs yourself.

The combo handler itself doesn't perform any caching or compression, but stick Nginx or something in front of it and you should be ready to rock in production.

Installation

Install using npm:

npm install combohandler

Or just clone the GitHub repo:

git clone git://github.com/rgrove/combohandler.git

Usage

The combohandler module provides a configurable Connect middleware that can be used to add combo handling capability to any Connect-based request handler (like Express).

The combohandler/lib/server module creates a standalone Express server instance, or augments an existing server, to perform combo handling for a set of configurable routes.

As Express middleware

The combo handler middleware can be used as application-wide middleware for all routes:

var combo = require('combohandler');
app.use(combo.combine({rootPath: '/local/path/to/files'}));

Or as route middleware for a specific route:

app.get('/foo', combo.combine({rootPath: '/local/path/to/foo'}), combo.respond);

In either case, the middleware will perform combo handling for files under the specified local rootPath when requested using a URL with one or more file paths in the query string:

http://example.com/<route>?<path>[&path][...]

For example:

http://example.com/foo?file1.js
http://example.com/foo?file1.js&file2.js
http://example.com/foo?file1.js&file2.js&subdir/file3.js

Attempts to traverse above the rootPath or to request a file that doesn't exist will result in a BadRequest error being bubbled up.

Here's a basic Express app that uses the combo handler as route middleware for multiple routes with different root paths:

var combo   = require('combohandler'),
    express = require('express'),

    app = express();

app.configure(function () {
  app.use(express.errorHandler());
});

// Return a 400 response if the combo handler generates a BadRequest error.
app.use(combo.errorHandler());

// Given a root path that points to a YUI 3 root folder, this route will
// handle URLs like:
//
// http://example.com/yui3?build/yui/yui-min.js&build/loader/loader-min.js
//
app.get('/yui3', combo.combine({rootPath: '/local/path/to/yui3'}), combo.respond);

app.listen(3000);

combo.respond

The respond method exported by require('combohandler') is a convenience method intended to be the last callback passed to an express route. Unless you have a very good reason to avoid it, you should probably use it. Here is the equivalent callback:

function respond(req, res) {
    res.send(res.body);
}

This method may be extended in the future to do fancy things with optional combohandler middleware.

combo.errorHandler

The errorHandler export encapsulates the convention of sending BadRequest errors with an optional errorMaxAge config. By default, BadRequest errors are served with a 5 minute max-age header.

To explicitly disable caching (via Pragma: no-cache and Cache-Control: private,no-store headers), pass null in the options object:

app.use(combo.errorHandler({
    errorMaxAge: null
}));

Any other value (including zero) for errorMaxAge is interpreted as the desired duration in seconds.

Creating a server

If you just want to get a server up and running quickly by specifying a mapping of routes to local root paths, use the combohandler/lib/server module. It creates a barebones Express server that will perform combo handling on the routes you specify:

var comboServer = require('combohandler/lib/server'),
    app;

app = comboServer({
    roots: {
        '/yui3': '/local/path/to/yui3'
    }
});

app.listen(3000);

Augmenting an existing server

If you already have an existing Express server instance and just want to add some combo handled routes to it easily, you can augment your existing server with combo handled routes:

var comboServer = require('combohandler/lib/server');

comboServer({
    roots: {
        '/yui3': '/local/path/to/yui3'
    }
}, myApp); // Assuming `myApp` is a pre-existing Express server instance.

From the command line

If installed globally via npm -g install, the CLI executable combohandler is provided. If you're operating from a local clone, npm link in the repository root and you're off to the races. To start the default single-process server, it's as simple as

combohandler
# combohandler now running until you hit Ctrl+C

Of course, the default output leaves something to be desired: that is to say, any output.

Root Configuration

At the very least, you need to provide some route-to-rootPath mappings for your CLI combohandler.

When passed in the --rootsFile option, the JSON file contents should follow this pattern:

{
    "/yui3": "/local/path/to/yui3"
}

When passed as individual --root parameters, the equivalent to the JSON above looks like this:

combohandler --root /yui3:/local/path/to/yui3 [...]

To run the standalone server in production mode, set the NODE_ENV variable to production before running it:

    NODE_ENV=production combohandler --root /yui3:/path/to/yui3

CLI Usage

Usage: combohandler [options]

General Options:
  -h, --help        Output this text
  -v, --version     Prints combohandler's version

Combine Options:
  -p, --port        Port to listen on.                                    [8000]
  -a, --server      Script that exports an Express app [combohandler/lib/server]
  -r, --root        String matching the pattern '{route}:{rootPath}'.
                        You may pass any number of unique --root configs.
  -f, --rootsFile   Path to JSON routes config, *exclusive* of --root.
  -b, --basePath    URL path to prepend when rewriting relative url()s.     ['']
  -w, --webRoot     Filesystem path to base rewritten relative url()s from. ['']
                    Use this instead of --basePath when using route parameters.
                    Overrides behaviour of --basePath.
  -m, --maxAge      'Cache-Control' and 'Expires' value, in seconds.  [31536000]
                    Set this to `0` to expire immediately, `null` to omit these
                    headers entirely.

Cluster Options:
  --cluster         Enable clustering of server across multiple processes.
  -d, --pids        Directory where pidfiles are stored.       [$PREFIX/var/run]
  -n, --workers     Number of worker processes.          [os.cpus.length, max 8]
  -t, --timeout     Timeout (in ms) for process startup/shutdown.         [5000]

  --restart         Restart a running master's worker processes.       (SIGUSR2)
  --shutdown        Shutdown gracefully, allows connections to close.  (SIGTERM)
  --status          Logs status of master and workers.
  --stop            Stop server abruptly, not waiting for connections. (SIGKILL)

The --port and --server options may also be set via npm package config settings:

npm -g config set combohandler:port 2702
npm -g config set combohandler:server /path/to/server.js

Unlike the --server option, a path specified in this manner must be absolute.

Clustered!

With the advent of node v0.8.x, the core cluster module is now usable, and combohandler now regains the capability it once had. Huzzah! said the villagers.

To run a clustered combohandler from the CLI, just add the --cluster flag:

combohandler --cluster --root /yui3:/path/to/yui3

To clusterize combohandler from a module dependency, combohandler/lib/cluster is your friend:

var comboCluster = require('combohandler/lib/cluster');
var app = comboCluster({
    pids: '/path/to/piddir',
    server: './myserver.js',
    roots: {
        '/yui3': '/local/path/to/yui3'
    }
});
app.listen(2702);

Optional Middleware

Rewriting URLs in CSS files

Because the combo handler changes the path from which CSS files are loaded, relative URLs in CSS files need to be updated to be relative to the combohandled path. Set the basePath or webRoot configuration option to have the combohandler default middleware do this automatically.

// This static route can be used to load images and other assets that shouldn't
// be combined.
//
app.use('/public', express.static(__dirname + '/public'));

// This route will combine requests for files in the public directory, and will
// also automatically rewrite relative paths in CSS files to point to the
// non-combohandled static route defined above.
//
app.get('/combo', combo.combine({
    rootPath: __dirname + '/public',
    basePath: '/public'
}), combo.respond);

// The equivalent config as the previous route, except using webRoot
app.get('/combo', combo.combine({
    rootPath: __dirname + '/public',
    webRoot : __dirname
}), combo.respond);

Alternatively, you can use the built-in cssUrls middleware as a separate route callback. cssUrls must always be placed after the default combine middleware when used in this fashion.

// This route provides the same behaviour as the previous example, providing
// better separation of concerns and the possibility of inserting custom
// middleware between the built-in steps.
app.get('/combo',
    combo.combine({
        rootPath: __dirname + '/public'
    }),
    combo.cssUrls({
        basePath: '/public'
    }),
    combo.respond);

Finally, the cssUrls middleware has the ability (disabled by default) to rewrite @import paths in the same manner as url() values. As @import is considered an anti-pattern in production code, this functionality is strictly opt-in and requires passing true as the rewriteImports property in the middleware options object.

// Automagically
app.get('/combo', combo.combine({
    rootPath: __dirname + '/public',
    webRoot : __dirname,
    rewriteImports: true
}), combo.respond);

// As explicit middleware
app.get('/combo',
    combo.combine({ rootPath: __dirname + '/public' }),
    combo.cssUrls({ basePath: '/public', rewriteImports: true }),
    combo.respond);

basePath or webRoot?

In the simplest case, basePath and webRoot reach the same result from different directions. basePath allows you to rewrite a single well-known path under any root, whereas webRoot will handle any number of paths under a well-known root.

In general, if you are using both optional middleware, you should prefer webRoot over basePath.

Dynamic Paths via Route Parameters

To enable resolution of dynamic subtree paths under a given rootPath, simply add a route parameter to both the route and the rootPath config.

app.get('/combo/yui/:version', combo.combine({
    rootPath: '/local/path/to/yui/:version/build'
}), combo.respond);

Given this config, any YUI release tarball you explode into a versioned subdirectory of /local/path/to/yui/ would be available under a much shorter URL than the default config provides:

    http://example.com/combo/yui/3.9.1?yui/yui-min.js&yui-throttle/yui-throttle-min.js
    // vs
    http://example.com/combo/yui?3.9.1/build/yui/yui-min.js&3.9.1/build/yui-throttle/yui-throttle-min.js

If the built-in dynamicPath middleware is used manually, it must be inserted before the default combine middleware.

Using as a YUI 3 combo handler

With a tiny bit of configuration, you can tell YUI to use your custom combo handler instead of the Yahoo! combo handler. Here's an example:

<script src="http://example.com/combo/yui3?build/yui/yui-min.js"></script>
<script>
YUI({
    comboBase: 'http://example.com/combo/yui3?',
    combine  : true,
    root     : 'build/'
}).use('node', function (Y) {
    // YUI will now automatically load modules from the custom combo handler.
});
</script>

License

Copyright (c) 2012 Yahoo! Inc. All rights reserved.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

combohandler's People

Contributors

elementstorm avatar ericf avatar evocateur avatar kara-ryli avatar natecavanaugh avatar rgrove avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

combohandler's Issues

errors and cpu spin on non-existent path

comboloader has been working great for me, for use with yui2 and yui3. Compared to the yui php combo loader, it works out of the box.

However I do see one problem. When the config includes a non-existent path comboloader goes into a spin, spewing out error messages along with high cpu use. This continues until I manually kill the process.

Including the logs:

aaditya➜lib/node/combohandler(master✗)» spark2 -v                                                                                                                                                                                [13:31:01]
... starting
... detected config.js
... loading config `/usr/local/lib/node/.npm/combohandler/0.1.2/package/./config.js'
...   --roots { '/combo/static/vendor/lib/yui3': '/Users/aaditya/work/id/src/id/vaitarna/vaitarna/public/static/vendor/lib/3.4.0x',
  '/combo/static/vendor/lib/yui2': '/Users/aaditya/work/id/src/id/vaitarna/vaitarna/public/static/vendor/lib/2.9.0',
  '/combo/static/vendor/lib/epsilon': '/Users/aaditya/work/id/src/id/vaitarna/vaitarna/public/static/epsilon',
  '/combo/static/vendor/lib/d1/d2/gallery': '/Users/aaditya/work/id/src/id/vaitarna/vaitarna/public/static/vendor/lib/gallery',
  '/combo/2in3/static/vendor/lib/2in3': '/Users/aaditya/work/id/src/id/vaitarna/vaitarna/public/static/vendor/lib/2in3' }
... detected app.js
... starting with (1) workers
... child spawned 87224

node.js:134
        throw e; // process.nextTick error, or 'error' event on first tick
        ^
Error: ENOENT, No such file or directory '/Users/aaditya/work/id/src/id/vaitarna/vaitarna/public/static/vendor/lib/3.4.0x'
    at Object.lstatSync (fs.js:396:18)
    at Object.realpathSync (fs.js:627:23)
    at Object.combine (/usr/local/lib/node/.npm/combohandler/0.1.2/package/index.js:22:21)
    at /usr/local/lib/node/.npm/combohandler/0.1.2/package/lib/server.js:33:26
    at Object. (/usr/local/lib/node/.npm/combohandler/0.1.2/package/app.js:2:18)
    at Module._compile (module.js:407:26)
    at Object..js (module.js:413:10)
    at Module.load (module.js:339:31)
    at Function._load (module.js:298:12)
    at require (module.js:351:19)
... Child Died (87224), respawning...
... child spawned 87225

node.js:134
        throw e; // process.nextTick error, or 'error' event on first tick
        ^
Error: ENOENT, No such file or directory '/Users/aaditya/work/id/src/id/vaitarna/vaitarna/public/static/vendor/lib/3.4.0x'
    at Object.lstatSync (fs.js:396:18)
    at Object.realpathSync (fs.js:627:23)
    at Object.combine (/usr/local/lib/node/.npm/combohandler/0.1.2/package/index.js:22:21)
    at /usr/local/lib/node/.npm/combohandler/0.1.2/package/lib/server.js:33:26
    at Object. (/usr/local/lib/node/.npm/combohandler/0.1.2/package/app.js:2:18)
    at Module._compile (module.js:407:26)
    at Object..js (module.js:413:10)
    at Module.load (module.js:339:31)
    at Function._load (module.js:298:12)
    at require (module.js:351:19)
... Child Died (87225), respawning...
... child spawned 87226

node.js:134
        throw e; // process.nextTick error, or 'error' event on first tick

CSS url rewriting is broken (on windows)

The CSS url rewrite does not seem to work.
Here is my web directory:

C:\work\if\server\web>dir
 Volume in drive C has no label.
 Volume Serial Number is 82A9-DF6B

 Directory of C:\work\if\server\web

01/10/2014  06:06 AM    <DIR>          .
01/10/2014  06:06 AM    <DIR>          ..
01/02/2014  07:15 AM    <DIR>          css
01/10/2014  06:06 AM    <JUNCTION>     if [C:\work\jetty\webapps\if]
01/02/2014  07:49 AM    <DIR>          js
01/04/2014  11:58 AM    <DIR>          lib
01/10/2014  06:05 AM    <JUNCTION>     yui [C:\work\jetty\webapps\yui]
01/10/2014  06:06 AM    <JUNCTION>     yui-gallery [C:\work\jetty\webapps\yui-gallery]
               0 File(s)              0 bytes
               8 Dir(s)  35,535,917,056 bytes free

C:\work\if\server\web>

_dirname is C:\work\if\server
The project loads yui/datatable-sort/assets/skins/sam/datatable-sort.css, which contains the following line:

url(../../../../assets/skins/sam/sprite.png)

Try 1:

app.get('/combo',
  combo.combine({
    rootPath: __dirname + '/web/',
    basePath: '/web'
  }),
  combo.respond);

The combined css rewrites the aforementioned url directive to

url(C:\web\yui\assets\skins\sam\sprite.png) 

Try 2:

app.get('/combo',
  combo.combine({
    rootPath: __dirname + '/web/',
    webRoot: __dirname
  }),
  combo.respond);

Which yields exactly the same url directive:

url(C:\web\yui\assets\skins\sam\sprite.png) 

Of course, it does not work.

Relax URL validation

Strange enough YUI 3.1.1 seed makes the following request, appending an unnecessary & to the URL:

http://yui.yahooapis.com/combo?3.1.1/build/widget/assets/skins/sam/widget.css&3.1.1/build/widget/assets/skins/sam/widget-stack.css&3.1.1/build/overlay/assets/skins/sam/overlay.css&

Their service ignores the ampersand, even when it is inserted multiple times, and fulfils successfully the request.

Given this behaviour is part of Yahoo's service, would you be willing to relax the validation, perhaps by removing the following lines from combohandler.js:

        if (fileTypes.indexOf('') > -1) {
            // Most likely a malformed URL, which will just cause
            // an exception later. Short-cut to the inevitable conclusion.
            return next(new BadRequest('Truncated query parameters.'));
        }

Will this be replaced by YLS/RLS?

I wonder if this module will be replaced by YLS/RLS in the future?

I need a custom combo handler for serving my private modules. Is this the only current solution?

CSS Url Loading/Rewriting for YUI as well as CSS in general not working *FIX INCLUDED*

Hello,

I've ran into some serious issues with this node module regarding the handling of CSS file loading and URL rewriting; When there are skin files included within YUI, it will attempt to pull from a machine's absolute directory(at least, on Windows). Another issue is anything with a data: path in a CSS URL would also have an error as well as the absolute path to the directory itself in a CSS URL.

I am on a Windows 8.1 machine btw, if that helps at all.

I fixed these but am not 100% certain the fix will apply to Unix/Linux machines. Haven't had an opportunity to test yet.

Please see the js fiddles I've created with the files that address these issues and apply them to the node NPM directory. Otherwise these files must be replaced every time npm install is run to fix.

cssUrls.js fixes: http://jsfiddle.net/9t1t14rz/
combohandler.js fixes: http://jsfiddle.net/b4zc8m3n/

Thanks for your time and for the great extension!

-Rob

Support YUI "shorthand" module syntax

"Classic" combo requests for YUI library follow the "module_name/module_name.js" pattern. "Shorthand" requests will omit the doubled part of the request.

Classical: combo?3.13.0/yui/yui.js
Shorthand: combo?3.13.0/yui.js

This should be achievable with some route config to enable handling missing files gracefully.

Support combining files across multiple roots

It would be useful to support combining files which reside in different root paths in one HTTP request.

app.get('/js', combo.combine({rootPath: [
    __dirname + '/public/js',
    __dirname + '/shared/js'
]}), function (req, res) {
    res.send(res.body, 200);
});
http://example.com/js?models/photo.js&widget/lightbox.js

One tricky aspect of this is having ambiguous file paths when a file of the same path resides in both roots.

Update HISTORY.md for 0.3.1 release.

It's funny how unit tests and code coverage only help so much. While I noodle about a more comprehensive integration test method, with these changes combohandler will actually work in a customized capacity.

v0.3.0...master

custom combo url template (a la stockpile)

YUI-Stockpile supports a custom template to build the combo url thru the loader that does not rely on the querystring (few reasons behind that [1]). E.g:

comboBase: "/combo~",
comboSep:  "~"

I see few options here:

  • expose parseQuery routine so it can be hacked
  • provide some basic configurations to specify comboSep and comboBase, and use those to parse the url

[1] http://www.yuiblog.com/blog/2012/11/06/managing-your-javascript-modules-with-yui-3-stockpile-2/

Serving binary assets

Looks like combohandler doesn't serves binary files (images for example) properly, even though proper mime-type is provided via config. I.E. images are served broken due to pollution of original file.
Using node v0.10.18.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.