gleam-lang / tree-sitter-gleam Goto Github PK

I noticed some strange behavior with the syntax highlighting of records vs records with functions. Consequently, I tried to introduce this test:

fn thing() {
  let local_record = Bar(thing: fn(x) { x + 1 })
  local_record.thing(5)
  // ^ variable
  //           ^ property
}

However, this test fails with this error:

  ✗ functions.gleam
    Failure - row: 44, column: 15, expected highlight 'property', actual highlights: 'function'

So the syntax highlighter is able to determine that local_record is a variable, not a module, but it is not able to extend that inference to imply that therefore the second part of the field_access must be a property instead of a function.

use #(a, b) <- blah

:)

When editing a source file, using attributes can cause odd syntax highlighting failures. For instance, using the @external attribute to mark an external function will cause the rest of the source file to be parsed incorrectly unless the function defines a fallback implementation.

A solution to this problem would likely be to attach attributes to legal parent nodes, such as functions and statement blocks. This would allow tree-sitter to more consistently "complete" a tree as soon as it matches a syntax node.

When I have code like:

    http.Get, [".well-known", "webfinger"] ->
      case host {
        value if value == config.connection.host -> web_finger_page(req, config)
        _ -> wisp.not_found()
      }

in a case statement, the syntax highlighting fails on the second . in config.connection.host.

Louis was able to reproduce it and I did my best to check with the latest commit from this repo via an entry in the Helix languages.toml.

Grateful for Gleam, this project and Helix :)

Hello! I found an instance in which a function name does not get highlighted. It's not clear to me why this is right away

https://github.com/gleam-lang/stdlib/blob/f9d57db847ff2a8471c9136bd309c870b610d04d/src/gleam/uri.gleam#L325

It looks like it stopped working at this commit: b7ae2c9

The new updates seem to be working really well! One spot where I'm seeing a parsing error is with @external.

It seems like the fn ... block is failing to parse, I'm guessing due to missing curly braces. If I add those, it parses successfully, but I obviously then get a syntax error from gleam.

It seems like the function isn't nested below the attribute so I'm not sure how easy this would be to change.

Thank you!

Looks like we may be able to add this to GitHub to get better support for Gleam.

https://twitter.com/importantshock/status/1496884996317011968

Create an issue on your repo and CC me (/patrickt on GitHub) or /dcreager.

Is this something the grammar and queries are ready for?

Hey,
when utilizing the panic() function within code files in nvim and hx editors, a syntax highlighting issue occurs as you can see in image below:

If I change panic() to panic as "Error" everything looks fine:

Thanks

Hello!

The latest Gleam release has added a new syntax, the <> operator.

Could we update this library to support it please? 🙏

https://github.com/gleam-lang/gleam/blob/6202f1b7b2aa15767b5a45410e870060befb4fa3/compiler-core/src/parse.rs#L847

https://github.com/gleam-lang/gleam/blob/6202f1b7b2aa15767b5a45410e870060befb4fa3/compiler-core/src/parse.rs#L2735

Thanks!

Hello!

The syntax for adding a message to panic and todo is now this

todo as "handle the case when..."
panic as "this is unreachable as..."

Currently only a string is permitted on the right hand side, but perhaps in future an expression would be supported. It may be good to have the tree sitter support parsing an expressions there to be more permissive than the compiler parser.

https://github.com/gleam-lang/stdlib/blob/c3f4f1b46443d70f0919d5646b3d4a4cc93d34d8/src/gleam/float.gleam#L247

After this float literal there is no more syntax highlighting.

Gleam features a pub modifier for constants, types, functions (internal and external), type definitions, and type aliases. Additionally, it also features an opaque modifier for type definitions and type aliases.

Presently, an construct with a modifier is represented as a different base AST node than the same construct without a modifier. For example:

const foo = "bar"

is represented as

(constant
 name: (identifier)
 value: (string
  (quoted_content)))

and the version with the public modifier:

pub const foo = "bar"

is represented as

(public_constant
 name: (identifier)
 value: (string
  (quoted_content)))

In the interest of usability, we should move to make constructs with and without a modifier share the same base AST node while the construct with the modifier should have an additional child node for the modifier.

e.g.

pub const foo = "bar"

should be represented as

(constant
 (visiblity_modifier)
 name: (identifier)
 value: (string
  (quoted_content)))

This behavior also exists in tree-sitter-rust.

Highlighting seems to be broken in the gleam core language tests:

https://github.com/gleam-lang/gleam/blob/0f20f6f46fa9c9e4baa6f264dd49e59ecbd9b0a5/test/language/src/main.gleam#L84

It looks like it stops after 0., so it could be that floats without values after the decimal aren't being parsed properly

Hello there!

Starting with Gleam v0.32 the syntax for unqualified importing a type will be import modulename.{type TypeName}.

The existing import modulename.{Name} will refer to the value constructor, not the type.

Thanks!

Hello there!

Gleam's parser has been changed to only permit let and use directly within functions and block. Code such as this will now be rejected:

let x = let y = 1

This is a subset of the previously accepted syntax so the current tree sitter will still parse all valid programs, but perhaps perhaps it wants to be restricted also?

With some code like so

<<code:int-size(8)-unit(2), reason:utf8>>

which gives

(bit_string
  (bit_string_segment
    value: (identifier
    options: (bit_string_segment_options
      (bit_string_segment_option_int)
      (bit_string_segment_option_size
        (integer))
      (bit_string_segment_option_unit)))
  (bit_string_segment
    value: (identifier)
    options: (bit_string_segment_options
      (bit_string_segment_option_utf8)))))))

I'd like the named nodes within the options field of bit_string_segment to be a little more generic so I can capture them in a highlight query and give them a special highlight. Something like

; captures unit, "size", "utf8", "int", etc.
(bit_string_segment_option) @function.builtin

so it might be parsed like so instead:

(bit_string
  (bit_string_segment
    value: (identifier
    options: (bit_string_segment_options
      (bit_string_segment_option)
      (bit_string_segment_option
        (integer))
      (bit_string_segment_option)))
  (bit_string_segment
    value: (identifier)
    options: (bit_string_segment_options
      (bit_string_segment_option)))))))

The nodes come out to be less specific than they are currently, but I think that's ok because you could do something like:

((bit_string_segment_option) @utf8
 (#eq? @utf8 "utf8"))

What do you think, would you be open to a PR that makes a refactor like this?

Hello! I'm curious if there are any plans to integrate this grammar with Neovim via nvim-treesitter. If not, I'm happy to help out with this 😄 Thanks!

Hello!

The external type syntax has been reworked.

// Previous
pub external type One

// New
pub type One

Thank you

I'll preface this with "I don't really know if there will be a resolution here". Also far from an expert in tree-sitter.

This is neovim specifically, but I imagine any indent query will run into the same issue. The grammar defines repeat1 for case_clauses, which I am guessing means 1 or more? That's the node used to determine the beginning of indentation. Before any clauses have been added, there is nothing to match on to signify that we should start indentation.

It's not valid Gleam to have an empty case, so changing that doesn't seem correct?

I guess this is more of a question/discussion issue. This functionality does work in VSCode, but obviously that is completely different. Just noting that it's not particularly unusual behavior.

Is there anything that can be changed in the grammar to handle this? If it's also a specific indent query issue, I can try to look more into that.

This might also end up just being a shortcoming of the grammar and indentation, but was just curious to get some info from more knowledgeable people :)

Hello!

This is a feature added in v0.26.0

An example of it can be found here:

https://github.com/gleam-lang/gleam/blob/72756d5e123853fc66c8810f783cc137be05a5f4/test/language/src/main.gleam#L114-L125

Hello!

v0.25 introduces a new feature called use. It looks like this:

use <- expression
use a <- expression
use a, b, c <- expression

Hello!

The upcoming Gleam release as 2 syntax changes. panic (which can go anywhere an expression goes, like todo but without the string argument), and let assert <pattern> = <expression>, which is the new syntax for assert <pattern> = <expression>.

Thanks!

When opening a gleam file with the latest commit, I get the following error:

This query could not be parsed: QueryError { row: 96, column: 3, offset: 2086, message: "try", kind: NodeType }

The Version 0.32.0 works fine, it has broken after that commit.
Example file: https://github.com/philipgiuliani/glubs/blob/55edb1e95d6b8ec66daa0e8aa2a13a008aeafad2/src/glubs/webvtt.gleam
Editor: helix 23.10 (f6021dd0)

Hello!

Gleam has some reserved keywords. Would it make sense to add these to the grammar? So they get highlighted appropriately if used.

https://github.com/gleam-lang/gleam/blob/0008b79d2eedfa2dd4d13efd67fa063e7ada285d/compiler-core/src/parse.rs#L2792-L2793

Thanks,
Louis

With tree-sitter 0.20.6 as used by the CI, the tests all pass for me. However when upgrading to tree-sitter 0.21.0 and later, some of the highlight tests fail.

  attributes:
    ✓ Target attribute
    ✓ Attribute with multiple values
  cases:
    ✓ Case examples
    ✓ Case examples
    ✓ Pattern matching binaries with 'as'
    ✓ Case with boolean negation in a guard
  constants:
    ✓ Constants
    ✓ Public constants
    ✓ Scientific notation
  custom_types:
    ✓ Parser example custom types
    ✓ Other custom type examples
    ✓ Public custom type definitions
    ✓ Public opaque custom type definitions
  destructuring:
    ✓ Case with spread
  expressions:
    ✓ Bit-string expression
    ✓ Boolean Negation
    ✓ Integer Negation
    ✓ Concatenation
    ✓ Todo and panic 'as' with string expressions
    ✓ Nested field access
  external_functions:
    ✓ External functions
    ✓ Public external functions
    ✓ External function with attribute syntax
  external_types:
    ✓ External types
    ✓ Public external types
  functions:
    ✓ Function examples
    ✓ Public function examples
    ✓ Basic functions
    ✓ Cases
    ✓ Let expressions
    ✓ Complex binary expressions
    ✓ Complex nesting of field and tuple access
    ✓ Unusual function invocations
    ✓ Various discard variables
    ✓ Weird lists
    ✓ Comment in string
  imports:
    ✓ Imports
    ✓ Unqualified imports
    ✓ Aliased imports
    ✓ Type imports
    ✓ Discard module imports
  pipes:
    ✓ Pipes
  statements:
    ✓ Use
  strings:
    ✓ Escape sequences
  targets:
    ✓ Target groups
    ✓ Target group edge cases
  type_aliases:
    ✓ Type aliases
    ✓ Public type aliases
    ✓ Public opaque type aliases
  whole_files:
    ✓ Excerpt from stdlib's base.gleam
    ✓ Excerpt from stdlib's bool.gleam
    ✓ Trailing commas
syntax highlighting:
Warning: you should add a `highlights` entry pointing to the highlights path in `tree-sitter` language list in the grammar's package.json
See more here: https://tree-sitter.github.io/tree-sitter/syntax-highlighting#query-paths

    ✓ bit_strings.gleam (21 assertions)
    ✗ constants.gleam
      Failure - row: 5, column: 18, expected highlight 'warning', actual highlights: 'string.escape'
    ✗ destructuring.gleam
      Failure - row: 0, column: 13, expected highlight 'variable.parameter', actual highlights: 'variable'
    ✗ expressions.gleam
      Failure - row: 9, column: 3, expected highlight 'operator', actual highlights: 'punctuation.delimiter'
    ✗ functions.gleam
      Failure - row: 0, column: 7, expected highlight 'function', actual highlights: 'variable'
    ✗ modules.gleam
      Failure - row: 4, column: 22, expected highlight 'module', actual highlights: 'variable'
    ✗ records.gleam
      Failure - row: 8, column: 12, expected highlight 'variable.parameter', actual highlights: 'variable'
    ✗ reserved.gleam
      Failure - row: 0, column: 0, expected highlight 'error', actual highlights: 'variable'

There's something called out in the changelog of tree-sitter 0.21 which might be the issue:

Breaking

• Remove the apply-all-captures flag, make last-wins precedence the default
  
  NOTE: This change might cause breakage in your grammar's highlight tests.
  Just flip the order around of the relevant queries, and keep in mind that the
  last query that matches will win.

I am not very experienced with tree sitter at all. I am currently having this experience in the Helix editor:

fn render_home(request: Request) -> Response {<-- pressing enter here
  let body = element.to_string_builder(layout.login())
  wisp.html_response(body, 200)
}

fn render_home(request: Request) -> Response {
<-- leaves the cursor here with no indentation
  let body = element.to_string_builder(layout.login())
  wisp.html_response(body, 200)
}

Which feels a bit off when you're used to it indention properly in other languages.

I copied the indents.scm file from zed into my ~/.config/helix/runtime/queries/gleam folder and it seems to behave better in this case. I don't know if they are right or not or how complex this file could get but it seems to help me a bit. I've only just done it so I'm not sure if there are negative impacts that I've not experienced yet.

Edit: I should add that I appreciate others know far more about this than I do and there might be good reasons why we don't have an idents file defined.

Howdy! As you know, I am working on integrating this grammar with neovim https://github.com/J3RN/tree-sitter-gleam/issues/1, during which I realized there is no node for string escape sequences. Without this node, I cannot highlight escape sequences similar to what I did with the nvim-treesitter Elixir highlight queries.

For example, tree-sitter-elixir parses:

"foo\nbar"

as the following tree of nodes:

(string
  quoted_start: """
  (quoted_content)
  (escape_sequence)
  (quoted_content)
  quoted_end: """)

I'm curious what you think!

Hello!

A new syntax has been added to Gleam, the attribute syntax

@target(erlang)
pub fn main() { todo }

Attributes proceed imports or definitions in modules. The compiler only accepts @target(erlang) and @target(javascript) but I believe the tree sitter should accept any name that would be a valid function name in place of target and any comma-delimited sequence of optionally labelled constant expressions within the ().

@deprecated(since: "1.2.0", replacement: wobble)
pub fn wibble() { todo }

Assuming wrapper accepts a callback with a generic argument (eg fn (event) -> Nil):

I can do this:

fn test() {
  wrapper(fn (event : ClickEvent) {  do_something(event) })
}

but I also can do this:

fn test() {
  use event : ClickEvent <- wrapper()
  do_something(event)
}

TS parsing will break down on such code - removal of : ClickEvent fixes it, but it's necessary.