Browserify: split vendor/app code into 2 bundles about blendid HOT 22 CLOSED

Thank you,

+1 for making this a standard feature. This is actually my use case. I'm
working on an application that should have 2-3 deployments per week and we
have poor/unreliable Internet connectivity and mobile devices. Splitting
the libraries will make caching kick in.

2015-01-19 18:46 GMT+02:00 Sébastien Lorber [email protected]:

See
https://github.com/sogko/gulp-recipes/tree/master/browserify-separating-app-and-vendor-bundles

This could be nice to include this by default to reduce build time.

Splitting oftenly modified code (app business code) from static code
(library code) is also interesting if you put your single page application
in production often: if you have 1 bundle, every production deployment make
all the browsers reload a big fat bundle. If you have 2 bundles but the
library bundle does not change the client can use the library bundle from
the browser cache (using etags) thus makes the app load faster to clients
on app deployments.

—
Reply to this email directly or view it on GitHub
#75.

Ioan Eugen Stan
0720 898 747

from blendid.

👍

from blendid.

I recently added support for multiple bundles and factoring out common dependencies. Is this what you're looking for? You can just use the require and external options of browserify to specify which libraries to expose to other bundles and to expect to be exposed by other bundles respectively. That should do it for you, right?

from blendid.

it seems to be yes. Except you seem to assume that the libs have an entry point I think.

In my app the libs are only packaged using require with no add

See my browserify build here:

"use strict";

var browserify   = require('browserify');
var gulp         = require('gulp');
var gutil        = require('gulp-util');
var handleErrors = require('../util/handleErrors');
var source       = require('vinyl-source-stream');
var watchify     = require("watchify");
var livereload   = require('gulp-livereload');
var gulpif       = require("gulp-if");


var buffer       = require('vinyl-buffer');
var uglify       = require('gulp-uglify');


var libs = [
    "ajax-interceptor",
    //"atom-react",
    "autolinker",
    "bounded-cache",
    "fuse.js",
    "highlight.js",
    "html-truncate",
    "htmlsave",
    "imagesloaded",
    "iscroll",
    "jquery",
    "keymaster",
    "lodash",
    "medium-editor",
    "mime-db",
    "mime-types",
    "moment",
    //"offline-js",
    "packery",
    "q",
    "rangy",
    "react",
    "react-date-picker",
    "spin.js",
    "steady",
    "store",
    "string",
    //"tether-tooltip",
    "uuid",
    "react-dnd"
];


// permits to create a special bundle for vendor libs
// See https://github.com/sogko/gulp-recipes/tree/master/browserify-separating-app-and-vendor-bundles
gulp.task('browserify-libs', function () {
    var b = browserify({
        debug: true
    });

    libs.forEach(function(lib) {
        b.require(lib);
    });

    return b.bundle()
        .on('error', handleErrors)
        .pipe(source('appLibs.js'))
        // TODO use node_env instead of "global.buildNoWatch"
        .pipe(gulpif(global.buildNoWatch, buffer()))
        .pipe(gulpif(global.buildNoWatch, uglify()))
        .pipe(gulp.dest('./build'));
});



// Inspired by http://truongtx.me/2014/08/06/using-watchify-with-gulp-for-fast-browserify-build/

gulp.task('browserify',['cleanAppJs','browserify-libs'],function browserifyShare(){
    var b = browserify({
        cache: {},
        packageCache: {},
        fullPaths: true,
        extensions: ['.jsx'],
        paths: ['./node_modules','./src/'],
        debug: true
    });
    b.transform('reactify');

    libs.forEach(function(lib) {
        b.external(lib);
    });

    // TODO use node_env instead of "global.buildNoWatch"
    if ( !global.buildNoWatch ) {
        b = watchify(b);
        b.on('update', function() {
            gutil.log("Watchify detected change -> Rebuilding bundle");
            return bundleShare(b);
        });
    }
    b.on('error', handleErrors);

    //b.add('app.js'); // It seems to produce weird behaviors when both using "add" and "require"

    // expose does not seem to work well... see https://github.com/substack/node-browserify/issues/850
    b.require('app.js',{expose: 'app'});

    return bundleShare(b);
});



function bundleShare(b) {
    return b.bundle()
        .on('error', handleErrors)
        .pipe(source('app.js'))
        .pipe(gulp.dest('./build'))
        // TODO use node_env instead of "global.buildNoWatch"
        .pipe(gulpif(!global.buildNoWatch, livereload()));
}

from blendid.

oooh interesting! I hadn't thought about a bundle with no entires. I can definitely update to accommodate that. Cool!

from blendid.

@greypants also @sogko added some code to do this automatically for all NPM/Bower dependencies:

https://github.com/sogko/gulp-recipes/blob/master/browserify-separating-app-and-vendor-bundles/gulpfile.js#L95

However I've had troubles with some libs using window.variable instead of module.exports: could not use require on them so I end up maintaining my own list.

Maybe you can add all dependencies to a lib bundle automatically, but provide a filtering mecanism to make some exceptions if it cause some troubles?

from blendid.

Actually, I don't think there's anything I need to change really. the entires option is not required. You can omit it and just require your vendor libraries, and specify an output destination.

bundleConfigs: [{
  dest: dest,
  outputName: 'vendor.js',
 require: ['jquery', 'underscore', 'etc']
}, {
  entries: src + '/javascript/page.js',
  dest: dest,
  outputName: 'page.js',
  external: ['jquery', 'underscore', 'etc']
}]

Lemme know if there's something else to do. Otherwise, I think the existing example works for your use case just fine.

from blendid.

yes that seems nice

from blendid.

@greypants : I've tried with require and external and it does not work for gulp production It works if I'm using node modules, but not with bower modules via browserify-shim, and as things are, not all of my modules are packaged in node with the version I require.

If I install the node packages things work, but maintaing two sets of dependenceis it's not something I would like to do.

P.S. I'm quite new with the JS tooling.
When running gulp production I get this (normal gulp build runs ok):

events.js:72
throw er; // Unhandled 'error' event
^
Error: Cannot find module 'backbone.marionette' from '/home/ieugen/MyProject'
at /home/ieugen/MyProject/node_modules/browserify/node_modules/resolve/lib/async.js:50:17
at process (/home/ieugen/MyProject/node_modules/browserify/node_modules/resolve/lib/async.js:119:43)
at /home/ieugen/MyProject/node_modules/browserify/node_modules/resolve/lib/async.js:128:21
at load (/home/ieugen/MyProject/node_modules/browserify/node_modules/resolve/lib/async.js:60:43)
at /home/ieugen/MyProject/node_modules/browserify/node_modules/resolve/lib/async.js:66:22
at /home/ieugen/MyProject/node_modules/browserify/node_modules/resolve/lib/async.js:21:47
at Object.oncomplete (fs.js:107:15)

In my package.json I have the following deffinitions:

"browserify": {
    "transform": [
      "coffeeify",
      "hbsfy",
      "jstify",
      "browserify-shim"
    ]
  },
 "browserify-shim": {
    "jquery": "$",
    "underscore": "_",
    "backbone": {
      "exports": "Backbone",
      "depends": [
        "jquery:$",
        "underscore:_"
      ]
    },
    "backbone.marionette": {
      "exports": "Marionette",
      "depends": [
        "backbone:Backbone"
      ]
    },
    "backbone.epoxy": {
      "exports": "Epoxy",
      "depends": [
        "backbone:Backbone"
      ]
    },
    "bootstrap": {
      "depends": [
        "jquery"
      ]
    }

from blendid.

@ieugen looks like you accidentally required backbone.marionette in a module instead of Marionette. You should be using the values specified in the exports property (when you've done so). Browserify-shim can get super confusing. Looks like there's some things to fix in your setup there.

from blendid.

@greypants : could you please be more explicit? I've treied with Marionett in the require field and it didn't work.

The following config gives error (see below) only for gulp production . Normal gulp invocation seems to ignore require and external.

var vendors = [
  'jquery',
  'underscore',
  'Backbone',
  'Marionette',
  'backbone.epoxy'
];
  bundleConfigs: [{
      dest: dest,
      outputName: 'global.js',
      // list of modules to make require-able externally
      require: vendors
    }, {
      entries: src + '/javascript/page.js',
      dest: dest,
      outputName: 'page.js',
      // list of externally available modules to exclude from the bundle
      external: ['jquery', 'underscore']
    }, {
      entries: src + '/javascript/main.js',
      dest: dest,
      outputName: 'main.js',
      // list of externally available modules to exclude from the bundle
      extensions: ['.coffee', '.hbs', ".tpl", ".tmpl"],
      external: ['$', '_', 'Backbone', 'Marionette']
    }

Error:

[19:59:54] Starting 'minifyCss'...

events.js:72
throw er; // Unhandled 'error' event
^
Error: Cannot find module 'Backbone' from '/home/ieugen/MyProject'
at /home/ieugen/MyProject/node_modules/browserify/node_modules/resolve/lib/async.js:50:17
at process (/home/ieugen/MyProject/node_modules/browserify/node_modules/resolve/lib/async.js:119:43)
at /home/ieugen/MyProject/node_modules/browserify/node_modules/resolve/lib/async.js:128:21
at load (/home/ieugen/MyProject/node_modules/browserify/node_modules/resolve/lib/async.js:60:43)
at /home/ieugen/MyProject/node_modules/browserify/node_modules/resolve/lib/async.js:66:22
at /home/ieugen/MyProject/node_modules/browserify/node_modules/resolve/lib/async.js:21:47
at Object.oncomplete (fs.js:107:15)

from blendid.

@ieugen I had this kind of problems too. I think this is because Backbone uses a global window variable instead of module.exports, which makes it incompatible with require of browserify. Maybe it can be solved by using some additionnal config? Like using browserify-shim or something? don't know.

In my project, I've kept these libs in my app bundle and not in the vender bundle to fix this problem. Try removing Backbone from the list and see what happens

from blendid.

@slorber, @greypants thanks for the help.

I went around the issue by moving some of the libraries to vendor (as node dependencies) and some in bower. Not all bower dependencies seem to work with require. It seems like there is still some time needed for the tooling to mature.

from blendid.

I think this is what you're looking for: https://github.com/eugeneware/debowerify

from blendid.

The problem is that you have to either:

1. Support hand-picked list of vendor libraries. Miss one depending on React and Browserify will write React to app bundle. This is getting bloody hard on big projects!
2. Rely on automatic listing. As soon as you develop frontend + backend app package.json has no clue how to distinguish them. You'll get your backend only deps on frontend which is blocker on itself, not even saying that some of them will break build...
   All this is just crazy and doesn't work at production scale. I'm now thinking of switching from Browserify to Webpack only for this reason.

from blendid.

@ivan-kleshnin I'm using an handpicked list and it's not so hard to maintain actually, it's just a list of 40 names. If one lib is not in the list, it ends up in the app bundle and it is not such a big deal.

How does webpack solve this problem?

from blendid.

If one lib is not in the list, it ends up in the app bundle and it is not such a big deal.

I disagree. I had three cases for the last months when I lost a hours of debugging because some libs don't work when their components are splitted aparts (because of state sharing) and error messages can be just weird. For example, flummox does not load flummox/component in its index file. I didn't know that and got a case where flummox was in my vendors bundle and flummox/component was in my app bundle. That failed with very obsure messages, just believe me. So the problem is not to write them up but to remember to update them and be aware of vendor speficifs about load mechanics. It's really a pain in the ass.

Actually, browserify has plugins to deal with it. I wasn't aware of them when posted prev. comment.
I don't know much about webpack because I don't use it (and would prefer to don't).

from blendid.

@ivan-kleshnin woah, good hint with the partioning plugins. 👍

from blendid.

@renestalder actually those plugins can split only by entry point.
There is no plugin yet that could help with factoring out vendor dependencies.
That's very sad.

from blendid.

Hello

i think this issue should be reopened; something is weird when packaging with browserify. if you export flummox in a separate bundle, you get confusing errors in the browser : "fluxMixin: Could not find Flux instance. Ensure that your component has either this.context.flux or this.props.flux."

does anyone has a working splitted browserify bundle including flummox ?

from blendid.

does anyone has a working splitted browserify bundle including flummox ?

@revolunet I migrated to Webpack and forgot all this crap like a bad dream 😉

from blendid.

@ivan-kleshnin yeah i went back from webpack due to other issues :)

from blendid.