Crashing (possibly while editing markdown) about tree-sitter-haskell HOT 7 CLOSED

hey @414owen, what's the reason for this being an assert?

  assert(to_copy <= TREE_SITTER_SERIALIZATION_BUFFER_SIZE);

I'd say there's a good chance that's what you're hitting here, @lf-

and also

chonkpad

😁

from tree-sitter-haskell.

Huh, that's cool. I didn't know editors highlighted markdown blocks at all.
I'll have a proper look tomorrow evening, but is it possible a realloc failed?

from tree-sitter-haskell.

@414owen I assumed it's not related to the markdown, but the "pretty big haskell file". though that size limit should only be hit as a consequence of nesting, shouldn't it?

in any case, the backtrace shows the error to be caused by __assert_fail, called from scanner_serialize, where this buffer size assert is, so I think that's a safe bet

from tree-sitter-haskell.

the scanner in the Python grammar simply cuts off the buffer at the size limit, so should we maybe replace the assert with that?

from tree-sitter-haskell.

Right, gotcha.

https://github.com/tree-sitter/tree-sitter-haskell/pull/62/files#diff-bbcf6c785b007d9655b443abc204cc3cc32aa03cea190cb865fe44252dc6f1dfL1618

What we had before was presumably undefined behaviour, so I added the assert.

A few other projects seem to not copy the buffer at all if it's too large.
The docs say For your parser to work correctly, the serialize function must store its entire state, and deserialize must restore the entire state. Whether or not his means we'll be buggy either way, I'm not sure. @maxbrunsfeld would it be possible to add a line to the docs about what to do if you simply don't have enough buffer space?

One thing we could do, is we could store uint8_ts for the indents, instead of uint16_t. That way, we'd be able to support files that have twice as many lexically nested layout-dependent constructs, but we'd only support indents only up to 255 chars.

I'll make a PR where we just don't copy the buffer at all. Not sure what @lf- will see visually in their markdown file after that.

@lf- I know you can't reveal the contents of your file, but maybe you could you tell us what the many layout-defining constructs are?

from tree-sitter-haskell.

The odd thing is that it is what I would call a pretty normal markdown file. I could lorem ipsum it and mechanically rename everything in the code blocks if you'd like. Frustratingly there's not an obvious repro, it just happened while writing in this file. Also, I'm very intrigued at why this parser is getting invoked with the particular stack trace that it is: called under pop-up menu completion (i use coc, which shouldn't do anything with treesitter there, so I'm very intrigued).

I would kill to have symbols or at least context? parser state? anything for all this, so I might figure out how to convince Nix to build a neovim and luajit and the treesitter plugin with symbols. No promises though, given it's work code and kinda a yak shave.

from tree-sitter-haskell.

mysterious!

from tree-sitter-haskell.