rse / astq Goto Github PK
View Code? Open in Web Editor NEWAbstract Syntax Tree (AST) Query Engine
Home Page: https://npmjs.com/astq
Abstract Syntax Tree (AST) Query Engine
Home Page: https://npmjs.com/astq
After updating pegjs-otf with newer lodash version, update astq to latest pegjs-otf
High Command Injection
Package lodash
Patched in >=4.17.21
Dependency of asty-astq
Path asty-astq > astq > pegjs-otf > lodash
More info https://npmjs.com/advisories/1673
Would you accept a PR with TypeScript types descriptions (.d.ts) files for the API and the adapter ? I already have it, if so, I just need to add the jsdoc descriptions, tidy up a bit and add it to package.json. But before I wanted to ask you, thanks, keep it up!
I'm receiving the error message stating that my version of SpiderMonkey AST doesn't support parent node traversal from /lib/astq.node.js:362.
The query I'm using is in fact trying to traverse parent nodes of an AST built with Acorn. The query is //Identifier[@name=='startTimer'] ..// *
What version of SpiderMonkey do I need to install to get this feature? Or is there a different adapter I should try and use?
is it possible to allow a string literal as id ?
the language would change possibly to:
id ::= /[a-zA-Z_][a-zA-Z0-9_-]*/ | string
This is required when handling ASTs that use ids like #text
I have two adapters, one I've created for HTML and one from astq-ts. Both appear to work great except the last()
or nth(-1)
, nth(-2)
, etc. functions. first()
and nth(1)
, etc. functions do work.
Would be awesome to have a parameter in execute() so it stops on first match (and returns a single Node). IMO this feature is commonly supported for query APIs and useful on searches that takes time and only one (or the first one) node is needed. (DOM querySelectorAll() vs querySelector()). Think on somebody querying lots of files in a large project, if only one node is needed then this could save lots of time.
Do you think this would be a viable feature for this project API ?
A couple of nice to haves related to this:
Thanks!, great library
Not sure if this is still activly maintained but your example does not work.
I modified it slightly to log the entire object and I can't get anything no matter what selector I use.
const acorn = require("acorn")
const ASTQ = require("astq")
let source = `
class Foo {
foo () {
const bar = "quux"
let baz = 42
}
}
`
let ast = acorn.parse(source, { ecmaVersion: 6 })
// console.log(JSON.stringify(ast, null, ' '))
let astq = new ASTQ()
const hey = astq.query(ast, astq.query(ast, `
// VariableDeclarator [
/:id Identifier [ @name ]
&& /:init Literal [ @value ]
]
`)
// .forEach(function (node) {
// console.log(`${node.id.name}: ${node.init.value}`)
// })
console.log('hey', hey)
// This also doesn't work
const hey = astq.query(ast, astq.query(ast, `Program`)
I think I never announce a couple of projects I've made using astq some time ago that I think the maintainers will be happy to see In particular this playground which uses monaco-editor so users can edit astq queries with syntax highlight, autocomplete, error handling inline help, and online evaluation against the target code which in my case is TypeScript /JavaScript/JSX/TSX. I thought the author would be happy seeing its creation being represented like this:
https://cancerberosgx.github.io/demos/cannabis/typescript-ast-query-editor/
(use the editor on the left and press ctrl-space for auto-completion of operators, modifiers, literals, etc. hover with the mouse to get help). Provoke some errors - I think more could be done with errors and suggestions).
the monaco configuration is mostly this
this is monaco definitions for those features: https://github.com/cancerberoSgx/cannabis/blob/master/typescript-ast-query-editor/src/editor/query/queryEditor.ts (you can try their playground and this config to play around...)
The playground has several examples of the real project that is a JavaScript/ TypeScript AST query library based on astq and although some uses custom helper functions of my library could be helpful for people understanding this syntax.
I must say that while I'm pretty familiar with AST representations of several languages I had a hard time learning astq syntax to write useful queries in a language like JavaScript
BTW what do you think of a playground like this but with a less complex target language like json/xml that shows real-life examples -I think that should be ideal for non expert users trying to learn the syntax. I would really want to master and document the syntax and BTW was it created by you ? was inspired on another work (form where get more examples) ? - I could easily reproduce that playground and contribute with something similar/simpler here - but first: I would like to know from @rse what's the status of this project but mostly if you are motivated or viable discuss/help me understand the limitations or advance queries that I didn't figure it out how to accomplish (in a single or finite number of queries).
In general I think that by adding 5 or 10 examples - real world targetting asts like JavaScript's could boost this project which IMO has great potential but is hard to grasp (the syntax not the API) for non expert users like me.The docs seems complete and formal, but I think this syntax is beautiful and complex enough to deserve a a getting started like documentation with examples - I understand the basics of several parser languages and their differences and still I found it hard to write simple queries since the syntax is rich and today whey you way queries you tink about html selectors, path, globs - which are pretty different.
BTW ( are you the author of this syntax ? Was it inspired in another works - if so or if there is something similar I would appreciate a reference. ) . I'm would like to contribute with some getting started and example document (but first wanted to ask about the project, limitations, and your motivation/viability, for describe and review my docs)
For example, this other project of mine https://cancerberosgx.github.io/demos/univac/playground/ is a playground to parse several common programming language code and shows their AST structure in different ways. It uses different parsing technologies form peg.js, tree-sitter, antlr4 so the structure of trees varies a lot between them. As a next step I would like to introduce the ability to query the asts using a language which can be relatively useful in all these trees / and language real-world queries (which include imperative, declarative, functional, parser definition, bash, strongly typed, data only, asm). Naturally I thought about this project and my doubt is if the syntax supports something like the following or must I solve it with helper functions:
In the context of C++/TypeScript/JavaScript/Java I would like to build one query (or two) that get me:
"the method declarations_(child-type)_ which
second (child index) parameter (child type) is of type (child-type) X (attribute)
name (child-type) contains Y (attribute)
members of (ancestor-type) of classes (type) that implements_(child-type)_ directly or indirectly the interface named_(attribute)_ U"
The difficult is recursion (directly or indirectly) . With multiple queries or a program the algorithm could be : find all interfaces extending interface I. then and recursively find all interfaces extending those and so and so until all are found. Then form those interfaces do nothing similar to query all the lasses implementing them directly or indirectly. My doubts are in general to this regard, is it possible to recurse like that using the syntax ? or I need to solve this with a helper function and probably a language extension or a second language on top of this?. my library already has helpers for this but slow compared with a solution based on astq
Going back to my typescript parser library It was very intuitive to add a custom adapter
thanks for the library hoping you are still available/motivated, and with excuses for my long msg which is not an issue, my greetings from Uruguay, Seba
Hi Ralf,
is it possible that you could add a simple information to
astq/src/astq.js Line 132
about the node that caused the no suitable adapter error
It is extremely horrible to determine the root cause of the error without the node that causes it.
Thanks
Hey :),
Thanks for the lib. I tried to bundle it with browserify
and I receivered the following error:
Error: Cannot find module './astq-util.js' from '.../node_modules/astq/lib'
Is there a chance that you could find few minutes to have a look at it?
Best regards,
Emil
Hallo Ralf,
danke für die lib. Ich hab' einen Adapter für den aktuellen CoffeeScript-AST gemacht. Der zukünftige CS-AST wird - aller Voraussicht nach - Babel-kompatibel sein.
Ich würde mir noch das eine oder andere Beispiel wünschen. Möglicherweise im WIKI ?
Insbesondere die Möglichkeiten des Ob und Wie man mit einem match
oder einem result
umgeht.
Also ich kann bspw. alle require
-expressions finden und als Ergebnis erhalten. Dann kann ich die Elemente, die ich brauche selbst extrahieren. Netter wäre es, wenn ich die gesuchten Teil-Elemente direkt benennen könnte - so wie in deinem Beispiel mit der /:id
und /:init
-Notation.
Ist es möglich ein Ergebnis weiter zu filtern ?
// Assign [
/ Value / Call / Value / IdentifierLiteral [@value=='require']
]
Die interessanten Teile befinden sich nun in einem anderen Pfad im Assign
-Knoten :
Assign / Variable / Value / IdentifierLiteral
Ein kleines Beispiel, wie man eigene (Extraktions)-Funktionen verwendet wäre auch hilfreich ?
Ich könnte da auch gerne was im WIKI formulieren - sobald ich verstanden habe wie es gemacht wird.
LG Andreas
Lines 75 to 86 in ea8d5c8
It seems like nth
is implemented by getting the parent, then getting the children on *
axis, and finding the element in question.
This produces arguably unexpected cases where nth
is called from within a filter on an axis with an id, e.g. I would expect
/ ExpressionStatement / CallExpression /:arguments * [
nth(1)
]
running on the input ast of
a(1)
to find the 1st argument of the CallExpression (1-based indexing, ouch).
Instead, this query returns nothing, because the 1st child of the CallExpression on axis *
is actually the function that was called, located on .callee
.
What now? Always just +1
the parameter for nth
in this case? Not pretty, and doesn't work on ast nodes with multiple variable-length child arrays.
While testing this, I've also found that
/ ExpressionStatement / CallExpression / *
finds 'Identifier', 'Literal', 'ExpressionStatement'
:
What is happening here? Why is ExpressionStatement, which is the parent of the CallExpression, being matched by this Query?
The readme defines the /
Axis as:
/
for direct child nodesEdit: Turns out this last part is likely caused by astq considering the parent
property (which I populated on my mozast to get nth
to work) as a valid child axis.
Would be great to have a working test demonstrating ASTQ#adapter and ASTQ@func methods (particularly the later has no example in readme). I think at some point you wanted to add a test (
Line 53 in 9764beb
I will try to add tests for both APIs (ASTQ#func and ASTQ#adapter methods), For the later I will need to define a custom AST (probably similar to snippet's #9 (comment)). Would you accept a PR with these ? Thanks
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.