Comments (12)
I share the sentiment, I find that this is one of the fundamental flaws of TextMate. When including another grammar, there should be a straight-forward way to specify a bail out end pattern.
The fundamental problem here is that we want to be good open source citizens and we should not deviate from TM's implementation to avoid fracturing the TM grammar comunity. However, if we do decide to add this, we should try to add it in a minimal invasive way, perhaps as an optional property when including another grammar.
cc @hediet
from vscode-textmate.
I support your thoughts exactly, this is a "between a rock and a hard place" problem.
I think there is a way forward, but I could use some help.
As far as I know, Textmate isn't a specification. There's no central comprehensive documentation, there's a nearly complete lack of standards when it comes to textmate-scoping names, and there's no working group for resolving design flaws or specifying "correct" behavior. I documented some of the obscure features on my own, but lack anywhere meaninful to put it. I've also had many conversations with grammar maintainers and old Atom contributors about standardizing textmate scopes, but we also lacked a central place to have a conversation or publish any agreed-upon result.
A central spec-repo (similar to WebAssembly's) could convert a realatively small feature like bailouts from bad-fracturing-change to a helpful-new-addition that other textmate engines could agree upon and take advantage of.
I can do a lot of the work towards that, bringing the feature documentation, formalizing existing pseudo-standards, contacting contributors/maintainers of major Textmate implementations, but I would need the backing/support of vscode-textmate to add/start with some form of legitmacy.
from vscode-textmate.
Yeah I don't consider https://macromates.com/ "comprehensive" by any means. I don't think it even mentions while
or applyEndPatternLast
. The convention guide fails for all but the most basic/sterotypical scopes, and like you say it's closer to a personal opinion rather than a standard. The only textmate spec I'm aware of is behavior comparisons against the original implementation.
@jeff-hykin may I see that please?
Sure! This could use updating/enhancing, but here's an example of some documention for while
https://github.com/jeff-hykin/better-cpp-syntax/blob/fe873cdfacd1df7072e7b8c95df3df369c1ffcaa/documentation/library/textmate_while.md
I think I have one somewhere for applyEndPatternLast
but can't find it at the moment. I've got a few unfinished unreleased "How to use Textmate (without wanting to jump off a bridge)" drafts that would-be/are comprehensive. Without anywhere to really publish them, and with such a limited audience I haven't worked on the drafts in over a year though.
Some of the other missing stuff is $base vs $self, injections, referencing "begin" capture groups inside the "end" pattern, using "repository" in a nested pattern, then referencing the outer repository vs inner repository, \G behavior, scope matching (like the negative operator that VS code doesn't support), and probably some other stuff I'm forgetting.
from vscode-textmate.
To add some context, matter123 and myself made an npm package textmate-bailout several years ago that does this kind of thing, albeit in a very expensive way (which is the only way as of current). I still use it in multiple syntax highlighters. We quite literally duplicate entire grammars, just to add our end pattern in front of all the embedded grammars' end-patterns to make sure they bail-out.
It would be an improvement in quality to use the npm package for markdown, html, latex, etc. They would become the largest syntax files by far though as they'd literally contain copies of every language, likely multiple copies.
If instead there was a built in feature for this bailout step, it would cut the C++ grammar file down by more than 50%. It would be a non-standard textmate feature though.
from vscode-textmate.
@jeff-hykin I think such effort would be beneficial. Please note that we don't have resources at the moment to assist with that.
Also note that there are subtle bugs in our implementation.
I'd rather try to bring tree sitter tokenization support to VS Code than spending a significant amount of energy on improving textmate support.
from vscode-textmate.
@zm-cttae it does already
the problem is that the inner embedded language can push the end pattern outwards
which is a feature, not a bug
tho not being able to disable it is the issue
but then if that setting did exist, it could cause just as many problems as it fixes
as it can be quite hard to deduce if the end pattern is inside a string etc without knowing the context around it
from vscode-textmate.
I'd rather try to bring tree sitter tokenization support to VS Code
Woah! As per microsoft/vscode#50140 I thought tree-sitter support was a non-starter. If VS Code is even considering adding support, thats huge news to me. I'd be happy for all effort to be spent towards tree sitter support.
from vscode-textmate.
I thought TextMate was just a spec
but seems to also be a program?
idk if this is the official documentation
v1: https://macromates.com/manual/en/
v2: https://macromates.com/textmate/manual/
it has its opinions about scope names. which vscode dark seems to follow exactly?
https://macromates.com/manual/en/language_grammars#naming-conventions
I documented some of the obscure features on my own
@jeff-hykin may I see that please?
begin
- while
rules are more strict than begin
- end
but also much more limited
prob not useable
I don't think injections can be used to solve this problem either
as they are only good for expanding rather than halting
from vscode-textmate.
I believe these were new features meant to be documented in Textmate 2 but Apple didn't dedicate any resources to that yet.
This is all we have and will have for a long time:
https://macromates.com/manual/en/language_grammars#example_grammar
from vscode-textmate.
Why isn't this being done by a hoisted variable that stores the end pattern when embedding begins?
from vscode-textmate.
It should be possible to filter the setter for that end pattern hoisted variable.
You'd limit the condition to a begin
for embedded languages only.
from vscode-textmate.
Another relevant technique: push a "stack" array of patterns for end conditions.
Include whether they end a language embed in the stack data.
from vscode-textmate.
Related Issues (20)
- yaml highlighting/textmate recognises some integers as strings (or vice versa?) HOT 1
- Absolute rule reference use local scope HOT 2
- Scoping Issues For Relative Rule References HOT 1
- Cant type `>` after `{` if `{}` and `<>` are defined as 'brackets' HOT 1
- Can't define recursive regex match expressions HOT 4
- Add Option to Format in often used match expression parts HOT 1
- Vscodes TextMate doesn't support the any scope selector `*` in `injections` HOT 2
- lacks of some type export on latest version
- Unable to exclude elements with "textmate" scope selectors in custom themes HOT 1
- Textmate engine bug for `\k<>` backreferences HOT 2
- Please add back IRawGrammar as top level export!
- YAML Syntax interprets IP-Style Strings as numbers HOT 1
- Tagging releases HOT 4
- Cannot read property 'INITIAL' of undefine.
- Scopes on Recursive Regex Cause Problems
- Nested `begin`/`while` rules do not continue past begin match HOT 3
- Misclassification of PHP syntax tokens HOT 3
- Matching multiline text HOT 2
- How to find all local variables? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vscode-textmate.