stefanpenner / broccoli-concat-analyser Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
[node 11/03/16 10:29:12] return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
[node 11/03/16 10:29:12] ^
[node 11/03/16 10:29:12]
[node 11/03/16 10:29:12] Error: ENOENT: no such file or directory, open 'concat-stats-for/undefined-engine-vendor.js/engines-dist/messaging-ext/assets/engine-vendor.js'
[node 11/03/16 10:29:12] at Error (native)
[node 11/03/16 10:29:12] at Object.fs.openSync (fs.js:549:18)
[node 11/03/16 10:29:12] at Object.fs.readFileSync (fs.js:397:15)
[node 11/03/16 10:29:12] at /Users/syeung/Documents/Development/new/voyager-web_trunk/node_modules/broccoli-concat-analyser/index.js:27:22
[node 11/03/16 10:29:12] at Array.map (native)
[node 11/03/16 10:29:12] at summarize (/Users/syeung/Documents/Development/new/voyager-web_trunk/node_modules/broccoli-concat-analyser/index.js:26:38)
[node 11/03/16 10:29:12] at /Users/syeung/Documents/Development/new/voyager-web_trunk/node_modules/broccoli-concat-analyser/index.js:56:3
[node 11/03/16 10:29:12] at Array.forEach (native)
[node 11/03/16 10:29:12] at Object.<anonymous> (/Users/syeung/Documents/Development/new/voyager-web_trunk/node_modules/broccoli-concat-analyser/index.js:53:9)
[node 11/03/16 10:29:12] at Module._compile (module.js:410:26)
uglify-es
is unmaintained. Also broccoli-uglify-sourcemap
has switched to terser
as well.
This hopefully might fix some errors here as well, when uglify-es
failed to minify some code (reproducible with https://github.com/NYCPlanning/labs-zola)
I noticed the total zipped size of an asset (say vendor.js
) as shown in the analyser output is much bigger that the real one.
Which is understandable given how we compute that number: we just compute the sum of all (individually zipped) sizes of the modules it contains. However zip/deflate (especially its duplicate string elimination) will work much better on a large concatenated file containing all modules, than when applied to all single modules individually. Which explains the difference AFAICT.
Maybe we should compute the zipped size of the whole asset file, and "spread" that to its individual modules proportionally (like zipped module_size = total_zipped_size / total_minified_size * minified_module_size
)?
I think its inherent to the compression algorithm that we cannot accurately determine how much a given module will attribute to the total (concatenated) zipped size, but this could be a good enough approximation, that makes sure that the sum of all zipped module sizes equals the real zipped size of the concatenated file.
@stefanpenner the changes required for this are probably not very difficult, I can try to bring up a PR when we agree on this!
CONCAT_STATS=true ember s is not generating /concat-stats-for dir
v4.3.5
so given
ember-data/-private/system/store
ember-data/adapters/rest.js
We could create the following depth:
ember-data
adapters
rest.js
-private
system
store.js
Most likely though, merely adding 1 additional grouping (the top level) would drastically improve this.
ember-data
adapters/rest.js
-private/system/store.js
I believe this addon relies on vendor/ember/ember.debug.map
being present.
There is an outstanding issue here emberjs/ember.js#15464 that I believe is related. I'm not sure of a workaround atm, but thought this might be a good place to add insight into implications of sourcemaps not being available.
Hello guys,
I'm trying to run this on a project of mine but it breaks if I have the ember-cli-chart
addon installed.
To reproduce:
ember new testApp
ember install ember-cli-chart
CONCAT_STATS=true ember s
broccoli-concat-analyser ./concat-stats-for
The error is:
⠹ processing: ./concat-stats-for2-vendor.js.json > ./concat-stats-for2-vendor.js.out.jsonfs.js:641
return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
^
Error: ENOENT: no such file or directory, open 'concat-stats-for/2-vendor.js/home/t318791/projects/testApp/node_modules/chart.js/dist/Chart.js'
at Error (native)
at Object.fs.openSync (fs.js:641:18)
at Object.fs.readFileSync (fs.js:509:33)
at summary.files.Object.keys.map.relativePath (/home/t318791/.nvm/versions/node/v6.11.0/lib/node_modules/broccoli-concat-analyser/index.js:28:22)
at Array.map (native)
at summarize (/home/t318791/.nvm/versions/node/v6.11.0/lib/node_modules/broccoli-concat-analyser/index.js:27:38)
at entries.forEach.entry (/home/t318791/.nvm/versions/node/v6.11.0/lib/node_modules/broccoli-concat-analyser/index.js:57:3)
at Array.forEach (native)
at Object.<anonymous> (/home/t318791/.nvm/versions/node/v6.11.0/lib/node_modules/broccoli-concat-analyser/index.js:54:9)
at Module._compile (module.js:570:32)
I'm using
$ ember -v
ember-cli: 2.13.3
node: 6.11.0
os: linux x64
When viewing the visualization, I'm noticing that the application css is completely missing - there's no area for it at all. The app js, vendor js, and vendor css are all showing up fine.
I've tried this on both a large application with lots of css, and a small add-on with just a bit.
Tested with ember-cli 2.18 and 3.0, as well as broccoli-concat-analyser 2.1.0 and 3.0.0.
Let me know what further info you need. Thanks!
Throws errors like these on OS X with Node 7.10.0:
undefined:1625
throw new JS_Parse_Error(message, filename, line, col, pos);
^
SyntaxError: Unexpected token operator «*», expected punc «(»
at JS_Parse_Error.get (eval at <anonymous> (/usr/local/lib/node_modules/broccoli-concat-analyser/node_modules/uglify-js/tools/node.js:1:1), <anonymous>:86:23)
Let's say I want to include the very nice bundle analysis in my repo and/or as artifacts on my ci/cd environment..
Is there a way I can output a self-contained file?
at the moment, there aren't many files,
but without zipping, it'd be hard to send this to somebody.
but I lose the ability to zoom in on the smaller pieces -- so I think an SVG would be the perfect solution to this. :)
When uglifying, we end up being quite CPU bound, and large projects likely suffer.
Using:
We can likely improve throughput. That being said, it will require some testing and tuning for sure.
Could it possible to improve the current readme? It's not quite clear to me what the steps are to actually get the process output.
0 npm install -g broccoli-concat-analyser
1 using broccoli-concat (latest versions)
2 CONCAT_STATS=true ember s
3 broccoli-concat-analyser ./concat-stats-for
4 look at .out.json files in ./concat-stats-for for the process output
1 It's a dependency of ember-cli, so I can ignore that probably unless someone doesn't have it installed.
2 Where do you put CONCAT_STATS=true. This does not work, I just found out this is a linux style command. I'm on Windows.
3 What does this mean?
4 I can't find the file, probably because I'm not doing it correctly.
Having the module path easily available alongside the existing size values could be helpful, e.g. to exclude that module by adding it to ember-cli-funnel
's exclude list: https://github.com/kellyselden/ember-cli-funnel#usage
It appears that broccoli-concat-analyser doesn't include packages which have been imported using ember-auto-import. I'm unsure whether the issue is in this addon or ember-auto-import but thought I'd open it here first.
To reproduce the issue I've made a sample repo where I've done the following:
Running ember asset-sizes
we get the following size vendor.js
File sizes:
- dist/assets/vendor.js: 3.2 MB (683.25 KB gzipped)
But broccoli-concat-analyser outputs the following size for vendor.js
- dist/assets/vendor.js: 2.65 MB (218.44 KB gzipped)
There looks to be a large discrepancy there. Looking at the generated stats it appears the auto-imported lodash
has been excluded.
The reproduction repo is here and I've committed the generated broccoli-concat-analyser stats
Livereload server on https://localhost:49155
The Broccoli Plugin: [BroccoliMergeTrees: styles] failed with:
TypeError: Cannot read property 'length' of undefined
at /Users/bas/Documents/Projects/PLHW.ProjectHealthyFeet/applications/hf-app-lab/node_modules/broccoli-concat/lib/strategies/simple.js:128:40
at Array.reduce (native)
at SimpleConcat.fileSizes (/Users/bas/Documents/Projects/PLHW.ProjectHealthyFeet/applications/hf-app-lab/node_modules/broccoli-concat/lib/strategies/simple.js:127:7)
at Concat._doPatchBasedBuild (/Users/bas/Documents/Projects/PLHW.ProjectHealthyFeet/applications/hf-app-lab/node_modules/broccoli-concat/concat.js:152:33)
at Concat.build (/Users/bas/Documents/Projects/PLHW.ProjectHealthyFeet/applications/hf-app-lab/node_modules/broccoli-concat/concat.js:104:17)
at /Users/bas/Documents/Projects/PLHW.ProjectHealthyFeet/applications/hf-app-lab/node_modules/broccoli-plugin/read_compat.js:93:34
at tryCatch (/Users/bas/Documents/Projects/PLHW.ProjectHealthyFeet/applications/hf-app-lab/node_modules/rsvp/dist/rsvp.js:525:12)
at invokeCallback (/Users/bas/Documents/Projects/PLHW.ProjectHealthyFeet/applications/hf-app-lab/node_modules/rsvp/dist/rsvp.js:538:13)
at publish (/Users/bas/Documents/Projects/PLHW.ProjectHealthyFeet/applications/hf-app-lab/node_modules/rsvp/dist/rsvp.js:508:7)
at flush (/Users/bas/Documents/Projects/PLHW.ProjectHealthyFeet/applications/hf-app-lab/node_modules/rsvp/dist/rsvp.js:2415:5)
I've been using the analyzer for fun over at https://emberclear.io/bundle.html
:)
But the generated bundle is a smidge out of date, as recently (I believe) there has been some private api change that the analyzer was relying on?
Here is my package.json for reference: https://gitlab.com/NullVoxPopuli/emberclear/blob/master/packages/frontend/package.json
I'm not able to run this anymore. I get this:
processing: ./concat-stats-for11-app.js.json > ./concat-stats-for11-app.js.out.jsonzlib.js:138
throw new TypeError('"buffer" argument must be a string, Buffer, ' +
^
TypeError: "buffer" argument must be a string, Buffer, TypedArray, or DataView
at zlibBufferSync (zlib.js:138:11)
at Object.deflateSync (zlib.js:588:14)
at summary.files.Object.keys.map.relativePath (/Users/rwwagner/.nvm/versions/node/v8.7.0/lib/node_modules/broccoli-concat-analyser/lib/summarize.js:22:25)
at Array.map (<anonymous>)
at summarize (/Users/rwwagner/.nvm/versions/node/v8.7.0/lib/node_modules/broccoli-concat-analyser/lib/summarize.js:16:38)
at entries.forEach.entry (/Users/rwwagner/.nvm/versions/node/v8.7.0/lib/node_modules/broccoli-concat-analyser/lib/summarize-all.js:13:5)
at Array.forEach (<anonymous>)
at summarizeAll (/Users/rwwagner/.nvm/versions/node/v8.7.0/lib/node_modules/broccoli-concat-analyser/lib/summarize-all.js:9:11)
at Object.<anonymous> (/Users/rwwagner/.nvm/versions/node/v8.7.0/lib/node_modules/broccoli-concat-analyser/lib/cli.js:19:1)
at Module._compile (module.js:624:30)
uglify-es
does not throw an error itself, so in case of minification errors the root issue is swallowed and results in a TypeError: Cannot read property 'length' of undefined
in here https://github.com/stefanpenner/broccoli-concat-analyser/blob/master/lib/summarize.js#L54.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.