Comments (11)
Actually no need to do the tree edit. I don't know why I included that in the example. Here is a simpler example showing an incorrect match:
# Initialize test query (match identifier with all caps)
query_text = '((identifier) @test (#match? @test "^[A-Z]$"))'
python_query = python_language.query(query_text)
# Original Source Code
orig_src = b'def myfunc():\n pass\n'
def read_callable_bytes(byte_offset, point):
byte = orig_src[byte_offset:byte_offset + 1]
return byte
# Parse original tree and query
orig_tree = parser.parse(read_callable_bytes)
orig_captures = python_query.captures(orig_tree.root_node)
# Print results
print(f"orig_sexp:{orig_tree.root_node.sexp()}")
print(f"orig_text:{orig_tree.root_node.text}")
print(f"orig_captures:{orig_captures}")
Output:
orig_sexp:(module (function_definition name: (identifier) parameters: (parameters) body: (block (pass_statement))))
orig_text:None
orig_captures:[(<Node type=identifier, start_point=(0, 4), end_point=(0, 10)>, 'test')]
from py-tree-sitter.
In this case - a read callable does not have the ability to index/remember the source input, so a query match predicate can't execute on that
from py-tree-sitter.
Is there a fundamental reason why read callable is not used for query? It seems inefficient to have to pass in the entire source code on every keystroke.
from py-tree-sitter.
Also, can you point me to where the code is for handling the match predicate?
from py-tree-sitter.
Is there a fundamental reason why read callable is not used for query? It seems inefficient to have to pass in the entire source code on every keystroke.
You cannot get the text at a specific point for the match to execute unlike using a buffer, sure, some read callbacks might be able to, but it's not a guarantee hence why it's omitted
Also, can you point me to where the code is for handling the match predicate?
py-tree-sitter/tree_sitter/binding.c
Line 1883 in da6716b
from py-tree-sitter.
@amaanq let me know what you think of this PR. I tested this out and it works great for me. I don't see why this wouldn't work for any type of read callback.
from py-tree-sitter.
@amaanq any love or feedback on this PR?
from py-tree-sitter.
Hey @MrPrezident I believe upstream (Rust/Wasm) doesn't keep text from sources that use a "read_callable" as well, so I'd like to implement it there first before here - I'm also a bit concerned about the performance cost about keeping the text in certain cases when conditionally switching the input based on external programmatic sources (e.g. a test in Rust will run forever until a timeout is reached - wouldn't that be expensive to keep? https://github.com/tree-sitter/tree-sitter/blob/6019b7c84cd5a7f580d57bb981315afd9afec826/cli/src/tests/parser_test.rs#L676
from py-tree-sitter.
I believe upstream (Rust/Wasm) doesn't keep text from sources that use a "read_callable" as well, so I'd like to implement it there first before here
Hmm.. ok. I can try to look into that.
I'm also a bit concerned about the performance cost about keeping the text in certain cases when conditionally switching the input based on external programmatic sources
but it's not keeping the text, it's just keeping the callback, or am I missing something?
from py-tree-sitter.
Well it collects the bytes from the callable right, that's what I meant sorry for not clarifying, like in that timeout test it's possible that it might accumulate a lot of bytes before the timeout is hit.
Maybe a flag in core to enable keeping the source text when using a callable would be nice
from py-tree-sitter.
Yeah but that test you are talking about is doing parsing, not querying. I didn't make any changes that would affect what happens to the callback during parsing.
from py-tree-sitter.
Related Issues (20)
- Remove/replace the usage of distutils HOT 1
- Missing aarch64 packages for 0.21.x HOT 1
- Make the Node class hashable. HOT 1
- [tree-sitter-python] No matching distribution found for tree-sitter-python HOT 3
- Python nil pointer crash while iterating over query captures
- Results returned by `Query.matches()` and `Query.captures()` are different HOT 2
- FutureWarning: Language(path, name) is deprecated. HOT 1
- Deprecation warning `Language.build_library is deprecated. Use the new bindings instead` HOT 4
- Tree sitter gets stuck HOT 3
- Can't pip install tree-sitter-python on mac os with apple silicon HOT 1
- way to get parser for typescript/tsx HOT 2
- How to get the location of a variable name in the whole function?
- Does that mean it's a conversion error? HOT 2
- How to build 0.22.x from source HOT 1
- TypeError on initializing Language with tree_sitter_python after following example code in readme. HOT 2
- Replacement of `build_library`
- Fails to load language on 32 bit HOT 6
- Post Source to pypi HOT 1
- cannot install the tree-sitter-python using pip HOT 1
- How to audit binary wheels?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from py-tree-sitter.