Well:

JavaScript parsing fatigue 🐭

Well.

• it should parse ES6 and JSX (maybe ES7 also. Generally it should be compatible with JavaScript people write in the wild).
• it should generate standard JavaScript AST like the rest of parsers (because there is a lot existing code assuming data structure and I don't want to rewrite it just because I would change parser)
• AST should work nicely with recast
• AST should work nicely with tern. It seems that this can be tricky: ternjs/tern#565

Now I use flow-parser in production and esprima-fb in tests:
https://github.com/hex13/lupa/blob/master/plugins/test/javascriptSpec.js
(which is weird, BTW, that tests have other parser than production. But this issue should be rethought and rewritten. I've just experimented with various parsers and I'm not quite sure which is the best (by best I mean compatible with as many things as possible (i.e. compatible with ES6, JSX, Recast, Tern, Flow etc.). I am not trying to optimize right now. Maybe some parsers can be faster than others, but criterion for choice is compatibility, not speed.

AST analysis of fluent interface (chaining) must be rewritten from scratch. Too much assumptions about node types, too chaotic.

It doesn't analyse correctly code like that:

angular.directive('DirectiveSomething', function () {

}).directive('DirectiveSomethingElse', function () {

});

and it's getting lost in tree.

when parsing with Acorn:

#!/usr/bin/env node                                                                                                                                       
// ... rest of file

SyntaxError: Unexpected character '#' (1:0)
at Parser.pp.raise (/Users/lukasz/sandbox/lupa/node_modules/acorn/dist/acorn.js:920:13)

using of fs prevents Lupa to be used in browser environment

it doesn't render correctly paths with asterisk parts for related files in getRelatedFiles.

Only JavaScript has full support of plugins. Plugins for line count or labels is not yet assigned for e.g. CoffeeScript files:
https://github.com/hex13/lupa/blob/master/src/analysis.js#L37

It should insert default plugins (like line counter) for all files.

It also should look for filename, not only for extensions (because e.g. package.json is special .json file and deserves for special treatment. So does gulpfile.js. This is not normal .js.

We need way for detecting and analysing scopes, project wide.
-- global variable 1
-- global variable 2
-- module level scope (module foo.js)
------ scope 1
------ scope 2
-- module level scope (module bar.js)
------ scope 1 (function, block etc.)
------ scope 2 (other function or block etc.)
------ scope 3
---------- nested scope 1
---------- nested scope 2
etc.

we need way for identify scopes.

enables:

• go to definition
• resolve referencies by name (e.g. in dependency searching)
• other

Given directory structure like this:

if in module.js there is relative require which resolves to module, like this:
var project = require('../..')
It should resolve to package level (because two levels up from project/src/directory is project and returns main file for the package (let's say index.js).

Instead it returns some weird incorrect files (research in progress).

better README, description that will tell what Lupa is for (analysing projects, gather metadata, make code queries, autocompleting, navigation, draw graphs etc.).

some demos.

some visualizations etc.

• remove files from older version of Lupa
• remove old tests
• remove console.logs, commented code etc.
• remove old dependencies
• remove old examples from README

modules which perform actual analysis (utils for AST and plugin which analyses JavaScript) are tested
https://github.com/hex13/lupa/tree/master/plugins/test

but there are still no tests for

• various small helper modules, e.g.:
  https://github.com/hex13/lupa/blob/master/src/metadata.js
• this important file, for example:
  https://github.com/hex13/lupa/blob/master/src/analysis.js
  here is API which outer applications use to communicate with Lupa as a library. File is called analysis.js but it containts more of a "controller" or "orchestrator" for whole thing.

We must test if module analysis.js:

1. calls correct plugins based on config and on files itself (e.g. file extensions)
2. triggers event callbacks after analysis
3. returns correct data (some sort of integration test. Exact data correctness is tested in tests for plugins, but even if some plugin correctly performs analysis, this doesn't mean that library returns correct data as a whole when called by external application).

PROJECT IS CURRENTLY IN MAINTENANCE ONLY MODE. ONLY BUG FIXES ANS SMALL IMPROVEMENTS. READ WHOLE THREAD*

Overview and purpose of the project
Lupa will read text files (JS, HTML, CSS etc.) and will use various means to analyses these (i.e. regexps, AST, code running etc.). It is supposed to have plugin-based architecture. Every plugin (file reader) will process file(input) and return(output) plain javascript object in specific format. This data will be integrated elsewhere in project (plugin shouldn't know anything about such an integration. Plugin has to just process input and return output).

Then integrated data will be transformed appropriately (it will be configurable which transformations should be used)

Then transformed data will be send to the reporter/view and presented to user in, for example, form of
animated diagram (using HTML/JS and libraries like D3.js) representing dependencies between modules. Or tree with all found classes/objects/modules in object and some form metadata attached to it, e.g. property/method names for each class. Maybe some code excerpts (usage examples for example). This project is supposed to help programmers in better understanding of their projects.

General program flow will be:
--> Get Project Configuration (for example directory structure patterns, list of plugins etc.)
--> Analyse Files (plugins will be doing that)
--> Transform Data
--> Create output / code visualisation / report and return to user

Planned architecture will look like that:
- some method of project configuration
- plugins for reading analyse files.
- transformations/mappers to modify raw data into something more appropriate (for example making trees or graphs from plain old arrays).
- reporters/views - modules that will be responsible for creating output. e.g. rendering HTML from template and inject data in it.

for now it is preparatory phase. EVERYTHING COULD CHANGE. I made this project public but I need time for prepare Lupa's architecture so that contributions from other developers than me would make any sense (I think that I need few weeks of alone work on this project). Then there should be more established architecture and potential merging will be a lot easier.

(Maybe this all writing is purposeless. Maybe I will be one solely developer working on that. And maybe nobody will join and contribute... But I think that this whole project idea has great sense and sooner or later somebody will stumble on it and try to submit his/her pull request)

it doesn't detect expressions like this:
const Whatever = require('some-module').Whatever
probably because VariableDeclarator's init is MemberExpression then, not CallExpression.

Practically it's not a bug but quirks of JavaScript AST (if we treated this as a bug we should say also that JavaScript language as a whole is just a one big bug, or "A Fractal of Bad Design", and we should automatically switch to TypeScript, Go, Java or some other language which is easier to analyse).

Nevertheless it should be rewritten a little. And a code responsible for require analysis is here:
https://github.com/hex13/lupa/blob/master/plugins/javascript.js#L132