rse / astq Goto Github PK

After updating pegjs-otf with newer lodash version, update astq to latest pegjs-otf

High            Command Injection                                
  Package         lodash                                                  
  Patched in      >=4.17.21                                          
  Dependency of   asty-astq                                      
  Path            asty-astq > astq > pegjs-otf > lodash                                                                     
  More info       https://npmjs.com/advisories/1673

Would you accept a PR with TypeScript types descriptions (.d.ts) files for the API and the adapter ? I already have it, if so, I just need to add the jsdoc descriptions, tidy up a bit and add it to package.json. But before I wanted to ask you, thanks, keep it up!

I'm receiving the error message stating that my version of SpiderMonkey AST doesn't support parent node traversal from /lib/astq.node.js:362.
The query I'm using is in fact trying to traverse parent nodes of an AST built with Acorn. The query is //Identifier[@name=='startTimer'] ..// *

What version of SpiderMonkey do I need to install to get this feature? Or is there a different adapter I should try and use?

is it possible to allow a string literal as id ?
the language would change possibly to:

id ::= /[a-zA-Z_][a-zA-Z0-9_-]*/ | string

This is required when handling ASTs that use ids like #text

I have two adapters, one I've created for HTML and one from astq-ts. Both appear to work great except the last() or nth(-1), nth(-2), etc. functions. first() and nth(1), etc. functions do work.

Would be awesome to have a parameter in execute() so it stops on first match (and returns a single Node). IMO this feature is commonly supported for query APIs and useful on searches that takes time and only one (or the first one) node is needed. (DOM querySelectorAll() vs querySelector()). Think on somebody querying lots of files in a large project, if only one node is needed then this could save lots of time.

Do you think this would be a viable feature for this project API ?

A couple of nice to haves related to this:

• adapters being able to stop the search
• API for pagination or at least result size limit.

Thanks!, great library

Not sure if this is still activly maintained but your example does not work.

I modified it slightly to log the entire object and I can't get anything no matter what selector I use.

const acorn = require("acorn")
const ASTQ = require("astq")

let source = `
    class Foo {
        foo () {
            const bar = "quux"
            let baz = 42
        }
    }
`

let ast = acorn.parse(source, { ecmaVersion: 6 })

// console.log(JSON.stringify(ast, null, '  '))

let astq = new ASTQ()
const hey = astq.query(ast, astq.query(ast, `
    // VariableDeclarator [
           /:id   Identifier [ @name  ]
        && /:init Literal    [ @value ]
    ]
`)
// .forEach(function (node) {
//   console.log(`${node.id.name}: ${node.init.value}`)
// })

console.log('hey', hey)

// This also doesn't work
const hey = astq.query(ast, astq.query(ast, `Program`)

I think I never announce a couple of projects I've made using astq some time ago that I think the maintainers will be happy to see In particular this playground which uses monaco-editor so users can edit astq queries with syntax highlight, autocomplete, error handling inline help, and online evaluation against the target code which in my case is TypeScript /JavaScript/JSX/TSX. I thought the author would be happy seeing its creation being represented like this:

https://cancerberosgx.github.io/demos/cannabis/typescript-ast-query-editor/

(use the editor on the left and press ctrl-space for auto-completion of operators, modifiers, literals, etc. hover with the mouse to get help). Provoke some errors - I think more could be done with errors and suggestions).

the monaco configuration is mostly this
this is monaco definitions for those features: https://github.com/cancerberoSgx/cannabis/blob/master/typescript-ast-query-editor/src/editor/query/queryEditor.ts (you can try their playground and this config to play around...)

The playground has several examples of the real project that is a JavaScript/ TypeScript AST query library based on astq and although some uses custom helper functions of my library could be helpful for people understanding this syntax.

I must say that while I'm pretty familiar with AST representations of several languages I had a hard time learning astq syntax to write useful queries in a language like JavaScript
BTW what do you think of a playground like this but with a less complex target language like json/xml that shows real-life examples -I think that should be ideal for non expert users trying to learn the syntax. I would really want to master and document the syntax and BTW was it created by you ? was inspired on another work (form where get more examples) ? - I could easily reproduce that playground and contribute with something similar/simpler here - but first: I would like to know from @rse what's the status of this project but mostly if you are motivated or viable discuss/help me understand the limitations or advance queries that I didn't figure it out how to accomplish (in a single or finite number of queries).

In general I think that by adding 5 or 10 examples - real world targetting asts like JavaScript's could boost this project which IMO has great potential but is hard to grasp (the syntax not the API) for non expert users like me.The docs seems complete and formal, but I think this syntax is beautiful and complex enough to deserve a a getting started like documentation with examples - I understand the basics of several parser languages and their differences and still I found it hard to write simple queries since the syntax is rich and today whey you way queries you tink about html selectors, path, globs - which are pretty different.

BTW ( are you the author of this syntax ? Was it inspired in another works - if so or if there is something similar I would appreciate a reference. ) . I'm would like to contribute with some getting started and example document (but first wanted to ask about the project, limitations, and your motivation/viability, for describe and review my docs)

For example, this other project of mine https://cancerberosgx.github.io/demos/univac/playground/ is a playground to parse several common programming language code and shows their AST structure in different ways. It uses different parsing technologies form peg.js, tree-sitter, antlr4 so the structure of trees varies a lot between them. As a next step I would like to introduce the ability to query the asts using a language which can be relatively useful in all these trees / and language real-world queries (which include imperative, declarative, functional, parser definition, bash, strongly typed, data only, asm). Naturally I thought about this project and my doubt is if the syntax supports something like the following or must I solve it with helper functions:

In the context of C++/TypeScript/JavaScript/Java I would like to build one query (or two) that get me:

"the method declarations_(child-type)_ which
second (child index) parameter (child type) is of type (child-type) X (attribute)
name (child-type) contains Y (attribute)
members of (ancestor-type) of classes (type) that implements_(child-type)_ directly or indirectly the interface named_(attribute)_ U"

The difficult is recursion (directly or indirectly) . With multiple queries or a program the algorithm could be : find all interfaces extending interface I. then and recursively find all interfaces extending those and so and so until all are found. Then form those interfaces do nothing similar to query all the lasses implementing them directly or indirectly. My doubts are in general to this regard, is it possible to recurse like that using the syntax ? or I need to solve this with a helper function and probably a language extension or a second language on top of this?. my library already has helpers for this but slow compared with a solution based on astq

Going back to my typescript parser library It was very intuitive to add a custom adapter

• support interesting features like
• represent files and folders as nodes too
• TS/JS projects (tsconfig.json / package.json and project dependencies
• JSX/TSX syntax support
• a second language to support glob like patterns (so I can filter nodes using expressions like "src//service//specialFileName*/InterfaceDeclaration/**/**MethodSignature"
• previous implies a definition for unique node path - for performance I'm caching each node path but probably I'm already destroying your work performance nevertheless
• query Identifier references and even names in the local scope of a symbol, or the declaration of a symbol or the apparent type of a name or return value, etc (which is inferred by tsc
• so many values here needs to be calculated and some are expensive - so there's a thin cache layer for that but performance.

thanks for the library hoping you are still available/motivated, and with excuses for my long msg which is not an issue, my greetings from Uruguay, Seba

Hi Ralf,

is it possible that you could add a simple information to
astq/src/astq.js Line 132
about the node that caused the no suitable adapter error

It is extremely horrible to determine the root cause of the error without the node that causes it.
Thanks

Hey :),

Thanks for the lib. I tried to bundle it with browserify and I receivered the following error:

Error: Cannot find module './astq-util.js' from '.../node_modules/astq/lib'

Is there a chance that you could find few minutes to have a look at it?

Best regards,
Emil

Hallo Ralf,
danke für die lib. Ich hab' einen Adapter für den aktuellen CoffeeScript-AST gemacht. Der zukünftige CS-AST wird - aller Voraussicht nach - Babel-kompatibel sein.
Ich würde mir noch das eine oder andere Beispiel wünschen. Möglicherweise im WIKI ?
Insbesondere die Möglichkeiten des Ob und Wie man mit einem match oder einem result umgeht.
Also ich kann bspw. alle require-expressions finden und als Ergebnis erhalten. Dann kann ich die Elemente, die ich brauche selbst extrahieren. Netter wäre es, wenn ich die gesuchten Teil-Elemente direkt benennen könnte - so wie in deinem Beispiel mit der /:id und /:init-Notation.
Ist es möglich ein Ergebnis weiter zu filtern ?

// Assign [
    / Value / Call / Value / IdentifierLiteral [@value=='require']
  ]

Die interessanten Teile befinden sich nun in einem anderen Pfad im Assign-Knoten :
Assign / Variable / Value / IdentifierLiteral

Ein kleines Beispiel, wie man eigene (Extraktions)-Funktionen verwendet wäre auch hilfreich ?
Ich könnte da auch gerne was im WIKI formulieren - sobald ich verstanden habe wie es gemacht wird.

LG Andreas

It seems like nth is implemented by getting the parent, then getting the children on * axis, and finding the element in question.
This produces arguably unexpected cases where nth is called from within a filter on an axis with an id, e.g. I would expect

/ ExpressionStatement / CallExpression /:arguments * [
  nth(1)
]

running on the input ast of

a(1)

to find the 1st argument of the CallExpression (1-based indexing, ouch).
Instead, this query returns nothing, because the 1st child of the CallExpression on axis * is actually the function that was called, located on .callee.
What now? Always just +1 the parameter for nth in this case? Not pretty, and doesn't work on ast nodes with multiple variable-length child arrays.

While testing this, I've also found that

/ ExpressionStatement / CallExpression / *

finds 'Identifier', 'Literal', 'ExpressionStatement':
What is happening here? Why is ExpressionStatement, which is the parent of the CallExpression, being matched by this Query?
The readme defines the / Axis as:

• / for direct child nodes

Edit: Turns out this last part is likely caused by astq considering the parent property (which I populated on my mozast to get nth to work) as a valid child axis.

Would be great to have a working test demonstrating ASTQ#adapter and ASTQ@func methods (particularly the later has no example in readme). I think at some point you wanted to add a test (

I will try to add tests for both APIs (ASTQ#func and ASTQ#adapter methods), For the later I will need to define a custom AST (probably similar to snippet's #9 (comment)). Would you accept a PR with these ? Thanks