The main goal is to show unused dependencies on Node.js modules, more specifically, to show that Foo dependency declared in package.json is not being used because there is no require('foo').
License: Other
Goals:
Should we add a flag for also searching for usage of development Dependencies?
https://github.com/bucharest-gold/szero/blob/master/lib/searcher.js#L79-L82
replace with:
require('repl')._builtinLibs
thanks @lholmquist
working:
[hf@archT440 genet]$ szero . --ci=true
----------------------------------------------------------------------
[ Declaration and file lines ]
----------------------------------------------------------------------
Fidelity-require('fidelity'):
./index.js:9
./index.js:55
./index.js:68
./lib/console_reporter.js:3
./lib/console_reporter.js:18
./lib/file_reporter.js:4
./lib/file_reporter.js:19
./lib/flamegraph_reporter.js:4
./lib/flamegraph_reporter.js:10
processor-require('huilu/lib/cpuprofile-processor'):
./index.js:8
./index.js:122
profiler-require('v8-profiler'):
./index.js:6
./index.js:7
./index.js:40
./index.js:56
./index.js:65
huilu-require('huilu/from-stream'):
./lib/flamegraph_reporter.js:5
./lib/flamegraph_reporter.js:11
Table-require('cli-table2'):
./lib/table.js:3
./lib/table.js:5
./lib/table.js:6
./lib/table.js:54
----------------------------------------------------------------------
[ Declaration and amount ]
----------------------------------------------------------------------
Fidelity-require('fidelity') --> [ 9 ]
processor-require('huilu/lib/cpuprofile-processor') --> [ 2 ]
profiler-require('v8-profiler') --> [ 5 ]
huilu-require('huilu/from-stream') --> [ 2 ]
Table-require('cli-table2') --> [ 4 ]
----------------------------------------------------------------------
[ Unused dependencies ]
----------------------------------------------------------------------
None.
not working:
[hf@archT440 genet]$ npm run dependencyCheck
> [email protected] dependencyCheck /mnt/datadisk/genet
> szero . --ci=true
----------------------------------------------------------------------
[ Declaration and file lines ]
----------------------------------------------------------------------
Fidelity-require('fidelity'):
./index.js:9
./index.js:55
./index.js:68
./lib/console_reporter.js:3
./lib/console_reporter.js:18
./lib/file_reporter.js:4
./lib/file_reporter.js:19
./lib/flamegraph_reporter.js:4
./lib/flamegraph_reporter.js:10
processor-require('huilu/lib/cpuprofile-processor'):
./index.js:8
./index.js:122
profiler-require('v8-profiler'):
./index.js:6
./index.js:7
./index.js:40
./index.js:56
./index.js:65
huilu-require('huilu/from-stream'):
./lib/flamegraph_reporter.js:5
./lib/flamegraph_reporter.js:11
Table-require('cli-table2'):
./lib/table.js:3
./lib/table.js:5
./lib/table.js:6
./lib/table.js:54
----------------------------------------------------------------------
[ Declaration and amount ]
----------------------------------------------------------------------
Fidelity-require('fidelity') --> [ 9 ]
processor-require('huilu/lib/cpuprofile-processor') --> [ 2 ]
profiler-require('v8-profiler') --> [ 5 ]
huilu-require('huilu/from-stream') --> [ 2 ]
Table-require('cli-table2') --> [ 4 ]
----------------------------------------------------------------------
[ Unused dependencies ]
----------------------------------------------------------------------
None.
npm ERR! Linux 4.7.1-1-ARCH
npm ERR! argv "/home/hf/.nvm/versions/node/v4.5.0/bin/node" "/home/hf/.nvm/versions/node/v4.5.0/bin/npm" "run" "dependencyCheck"
npm ERR! node v4.5.0
npm ERR! npm v2.15.9
npm ERR! code ELIFECYCLE
npm ERR! [email protected] dependencyCheck: `szero . --ci=true`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] dependencyCheck script 'szero . --ci=true'.
npm ERR! This is most likely a problem with the genet package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! szero . --ci=true
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs genet
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!
npm ERR! npm owner ls genet
npm ERR! There is likely additional logging output above.
npm ERR! Please include the following file with any support request:
npm ERR! /mnt/datadisk/genet/npm-debug.log
Looks like we also need to add the license info section to the file output.
doing szero -v or szero --version should show the version of szero
[ Declaration and file lines ]
undefined:
./from-stream.js:5
./from-stream.js:5
./from-stream.js:41
./from-stream.js:4
./from-stream.js:4
./from-stream.js:29
./index.js:6
./index.js:6
./index.js:25
./index.js:49
./index.js:51
./index.js:55
./index.js:49
./index.js:7
./index.js:9
./index.js:10
./index.js:25
./index.js:44
./index.js:49
./index.js:51
./index.js:53
./index.js:60
./index.js:61
./lib/contextify.js:3
./lib/contextify.js:3
./lib/contextify.js:147
./lib/contextify.js:147
./lib/contextify.js:151
./lib/contextify.js:152
./lib/svg-template.js:3
./lib/svg-template.js:3
./lib/svg-template.js:6
./lib/svg-template.js:8
./lib/svg.js:3
./lib/svg.js:3
./lib/svg.js:31
./lib/svg.js:31
./lib/svg.js:6
./lib/svg.js:8
./lib/svg.js:17
./lib/svg.js:27
./lib/svg.js:30
./lib/svg.js:31
./lib/svg.js:32
./lib/svg.js:33
[ Declaration and amount ]
undefined --> [45]
[ Unused dependencies ]
This was a inserted bug from migration to Dep class which holds the name and version of dependency.
https://github.com/bucharest-gold/szero/blob/master/lib/reporter.js#L93
If your run szero project_name with a project that doesn't have a package.json, an error is thrown, which is correct i think, but the error should probably be better looking instead of the stack trace
the run method here: https://github.com/bucharest-gold/szero/blob/master/bin/cli.js#L8
It feels like it is trying to do much, maybe we can break out some of the things it is doing into smaller pieces
depends on #30
It looks like there is the possibility that the license report can be run twice when using the API.
https://github.com/bucharest-gold/szero/blob/master/bin/cli.js#L61 that block of code will run if the consoleReporter is true and the license flag is true. We should probably just return after doing the console/filereporter.
It might make sense to break this out, so this one run function isn't doing everything, lets focus on that for another release.
Found on huilu build:
https://travis-ci.org/bucharest-gold/huilu/jobs/173071772#L424
related to #42 , i wonder if showing the license info should be behind a flag? We are doing a GET for each of the dependencies, so for a project with lots of dependencies, it could take a while.
@helio-frota @jaredsmith @lance thoughts?
I'll create this as another issue, but we should also have the ability to output that license info in the public API(maybe also behind a flag).
with more flags being added, i wonder if it is time to add a more sophisticated parsing tool like commander, https://www.npmjs.com/package/commander.
i add the issue for adding a --help flag #32 , using commander would give us this for free,
Looks like it only had 1 dependency and that dependency has no dependencies
If we like this idea, then i can set it up for the next milestone
example
, cpuprofilify:
should be:
cpuprofilify
While running in Node 7, There is a test that produces the
(node:80749) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): package.json is require
(node:80749) DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.`
we should probably fix that. Not sure if it is the test or the lib that is the issue
----------------------------------------------------------------------
[ Unused dependencies ]
----------------------------------------------------------------------
concat-stream,concat-stream,concat-stream,concat-stream,
concat-stream,concat-stream,concat-stream,concat-stream,
concat-stream,concat-stream,concat-stream,concat-stream,
concat-stream,concat-stream,concat-stream,concat-stream,
concat-stream,concat-stream,concat-stream,concat-stream,
concat-stream,concat-stream,
....
....
Sometimes we need to focus only the "break CI feature" in case unused dependencies found, instead see the information about amount of use etc.
Something like:
szero . --ci --silent
----------------------------------------------------------------------
[ Declaration and file lines ]
----------------------------------------------------------------------
require('roi'):
x.js:4
roi:
x.js:4
x.js:7
----------------------------------------------------------------------
[ Declaration and amount ]
----------------------------------------------------------------------
require('roi') --> [ 1 ]
roi --> [ 2 ]
undefined <-------------------------------------------------------------------------
There is some directories we don't need to track when using szero, an example of that is to run szero inside a project that has bower dependencies.
require('../../js/transition.js')
require('../../js/alert.js')
require('../../js/button.js')
require('../../js/carousel.js')
require('../../js/collapse.js')
require('../../js/dropdown.js')
require('../../js/modal.js')
require('../../js/tooltip.js')
require('../../js/popover.js')
require('../../js/scrollspy.js')
require('../../js/tab.js')
require('../../js/affix.js')This code is from bootstrap dependency using bower and that syntax is equal node modules, but most of the time we don't want to track bower modules.
Actually the output of szero inside projects with bower returns:
bower_components/bootstrap/dist/js/)
bower_components/bootstrap/fs
bower_components/bootstrap/grunt
bower_components/bootstrap/grunt/bs-commonjs-generator.js
bower_components/bootstrap/grunt/bs-glyphicons-data-generator.js
bower_components/bootstrap/grunt/bs-lessdoc-parser.js
bower_components/bootstrap/grunt/bs-raw-files-generator.js
bower_components/bootstrap/grunt/btoa
bower_components/bootstrap/grunt/fs
bower_components/bootstrap/grunt/glob
bower_components/bootstrap/grunt/markdown-it
bower_components/bootstrap/grunt/path
bower_components/bootstrap/grunt/shelljs
bower_components/bootstrap/js/)
bower_components/bootstrap/js/affix.js
bower_components/bootstrap/js/alert.js
bower_components/bootstrap/js/button.js
bower_components/bootstrap/js/carousel.js
bower_components/bootstrap/js/collapse.js
bower_components/bootstrap/js/dropdown.js
bower_components/bootstrap/js/modal.js
bower_components/bootstrap/js/popover.js
bower_components/bootstrap/js/scrollspy.js
bower_components/bootstrap/js/tab.js
bower_components/bootstrap/js/tooltip.js
bower_components/bootstrap/js/transition.js
bower_components/bootstrap/load-grunt-tasks
socket.io
socket.io-redis
...
Need to add some jsdoc type stuff to document the API
I believe currently when the command is run, both reporters(console and file) are run.
I think this should change a little bit, since this will affect #16 . Here is my suggestion.
When run from the command line, it should default to just do console output and if some flag is provided, maybe like --reporter=file or something like that, then it will also do the file reporter.
When calling the api directly, console and file reporters are not run(i'll create a future issue about also returning a file in the api) and just the JSON object is returned.
@helio-frota any strong objections or agreements :)
Now that #42 is in there, we should add the ability for this information to be accessible in the programmatic API.
possibly behind a flag, so the user needs to opt-in
Currently we have an array full or deps and devDeps.
return Array.from(dependencies);
The problem is, this way ^ we can't create different sections on report.
More details
#45 (comment)
----------------------------------------------------------------------
[ Unused dependencies ]
----------------------------------------------------------------------
[object Object],[object Object]
We need to extract this code to have easy maintenance.
https://github.com/bucharest-gold/szero/blob/master/lib/reader.js#L23-L35
Also to easily exclude more unrelated directories.
Currently when we do a search for license, we get all the result from all module versions.
Look:
http://registry.npmjs.org/test-ccsv/
We need do it:
http://registry.npmjs.org/test-ccsv/1.1.1
To get the current version in use on package.json
Unused dependencies
lightbright
Unused devDependencies
roi
Missing dependencies
fidelity
depends on #30
require('../index.js')
require('../index')
This is the same file, but since node do a cache, we need to keep only one.
Suggestion to keep require('../index.js') or what is with the nearest path.
szero is using a Set so if we do a require('foo') inside A.js
When szero found another require('foo') inside B.js it will be discarded automatically due Set.
So we need also to solve the issue with relative path like:
require('../index.js')
require('../../index')
require('../../../index')
require('../../../../index.js')
Because the Set will will treat this as different elements.
Today szero mix require and usage.
It would be good to know only the unique require('something').
As a new section on output report.
Would you please consider adding licensing information about each dependency, so that it's easy to scan a program and tell which licenses are getting pulled in by each dependency?
require=="function"&&require;if(!u&&a)
how to reproduce:
requiredBy.filter(isLink),requiredBy.some(isN....
actual :
----------------------------------------------------------------------
[ Unused dependencies ]
----------------------------------------------------------------------
should be something:
----------------------------------------------------------------------
[ Unused dependencies ]
----------------------------------------------------------------------
none.
` ``
or not show this section ^
We need to create an entry point, like index.js, that exports the api so someone could use it in code
szero is tracking commented requires but it is not something expected since that modules are not being loaded from node and it is not using resources.
My test to check if a commented module is being tracked was to comment a require and checked if it is in the output of szero.
const indexOnRootDirectoryRequiredFromC = require('../index.js');
const indexOnRootDirectoryRequiredFromD = require('./d/index.js');
// const indexOnRootDirectoryRequiredFromD = require('../a/commentedModule.js');
/* const indexOnRootDirectoryRequiredFromE = require('../a/commentedModule1.js'); */and my output was
require('../index.js'),
require('./d/index.js'),
require('../a/commentedModule.js')
require('../a/commentedModule1.js')
so when running szero -h or szero --help show the usage
We should probably add a new method that returns the formatted string that eventually can be turned into a file
edited: i changed the the return type from buffer/stream, to just the formatted string. This allows the user to write it to the file however they want to
possible add a flag on the command line to allow for different naming of the szero.txt file that gets output using the fileReporter
Create a new part on report to show unused.
example:
require('concat-stream') --> [ 1 ]
concatStream --> [ 2 ]
require('through2') --> [ 1 ]
through --> [ 2 ]
Since the variable name is not the same as required dependency , the report shows this for unused dependencies:
----------------------------------------------------------------------
[ Unused dependencies ]
----------------------------------------------------------------------
concat-stream,through2
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.