nom rewrite and AST node locations about hcl-rs HOT 6 CLOSED

Hi there!

Yes, I'm currently evaluating the replacement of pest with nom, mainly to see how easy it is to replicate the current parser and if it makes implementing some improvements easier. I'm planning support for emitting the complete AST of an HCL document at one point, mainly to support use cases like #112. This would also include span information.

I'm still undecided if I want this to be part of the general purpose parser that just emits structural elements, expressions and templates (like it does right now). The overhead might harm performance. If this is the case, I might end up with two parsers, one for the common use case, and a specialized one that emits the full AST including comments and whitespace. These two could share a lot of code of course, but the API and data types emitted would differ.

I'm interested to hear what your use case would be for the location spans!

from hcl-rs.

I base my parsing on hcl::structure::body::Body and having location spans would enrich my error messages

from hcl-rs.

Our use case is very much like a linter. We're building a security scanner for HCL files. So:

1. We deserialize to get the attributes, blocks, as data
2. We lint the data with a datalog-like language (OPA/rego)
3. Then when finding a bad value we want to point the user into the line:col coordinate to check for these issues

As a security linter we're focused on the values inside these files, and the composition of it, meaning the syntax might be perfectly fine.

Having location spans can support any linting usecase and not just a security linting example.
Hope that makes sense - LMK if you'd like some more detail of exactly how we do it 👍

from hcl-rs.

Thanks for the feedback, this makes complete sense to me. I'll try to make span information available as part of the nom rewrite. I was already aware of nom_locate and will probably use it to track the spans.

from hcl-rs.

Great!
If I can be of any help -- feel free to pull me in :)

from hcl-rs.

@jondot Hey there, I'm happy to announce that I released the first version of hcl-edit, which also includes span information (accessible via the hcl_edit::repr::Span trait). There are still rough edges and usage examples and a lot of docs are missing, but the parser is functional and even over 100% faster than the original one in hcl-rs.

I'll continue to improve this crate and will also integrate it with hcl-rs, but this will take some more work. Happy to receive some initial feedback if you want to try it out :).

https://docs.rs/hcl-edit

from hcl-rs.