Making syntax highlighting in Textadept providing features supported in SciTE Python lexer about textadept HOT 7 CLOSED

Excerpt from the updated lexers/lua.lua file :

local singleQuoteDelimitedTextBlock	= lexer.range("'")
local sQstr = singleQuoteDelimitedTextBlock
--
local doubleQuoteDelimitedTextBlock	= lexer.range('"')
local dQstr = doubleQuoteDelimitedTextBlock
--
local doubleSquareBracketWithOptionalIntermediateEqualSignDelimitedTextBlock_Level3andMore = lpeg.Cmt(
		"[" * lpeg.C(lpeg.P("=")^3) * "["	
	,	function(input, index, eq) local _, e = input:find("]" .. eq .. "]", index, true)
									return (e or #input) + 1 end	)		
local Lstr3 = doubleSquareBracketWithOptionalIntermediateEqualSignDelimitedTextBlock_Level3andMore
--
local doubleSquareBracketWithOptionalIntermediateEqualSignDelimitedTextBlock_Level2andMore = lpeg.Cmt(
		"[" * lpeg.C(lpeg.P("=")^2) * "[" 
	,	function(input, index, eq) local _, e = input:find("]" .. eq .. "]", index, true)
									return (e or #input) + 1 end	)
Lstr2 = doubleSquareBracketWithOptionalIntermediateEqualSignDelimitedTextBlock_Level2andMore
--
local doubleSquareBracketWithOptionalIntermediateEqualSignDelimitedTextBlock_Level1andMore = lpeg.Cmt(
		"[" * lpeg.C(lpeg.P("=")^1) * "["	
	,	function(input, index, eq) local _, e = input:find("]" .. eq .. "]", index, true)
									return (e or #input) + 1 end	)
Lstr1 = doubleSquareBracketWithOptionalIntermediateEqualSignDelimitedTextBlock_Level1andMore
--
local longstring = lpeg.Cmt(
		"[" * lpeg.C(lpeg.P("=")^0) * "["	
	,	function(input, index, eq) local _, e = input:find("]" .. eq .. "]", index, true)
									return (e or #input) + 1 end	)
Lstr0 = longstring
--	^^^---vvv	^^^--vvv	^^^--vvv 	^^^--vvv
lex:add_rule(	'string',  
--			lex:tag(lexer.STRING, sq_str + dq_str) 
--		 	lex:tag(lexer.STRING .. ".longstring"			, longstring	)
			lex:tag(lexer.STRING .. ".bracketed3xEQstr"	, Lstr3	)
		+	lex:tag(lexer.STRING .. ".bracketed2xEQstr"	, Lstr2	)
		+	lex:tag(lexer.STRING .. ".bracketed1xEQstr"	, Lstr1	)
		+	lex:tag(lexer.STRING .. ".bracketed0xEQstr"	, Lstr0	)
		+	lex:tag(lexer.STRING .. ".singlequotestring"	, sQstr	)
		+	lex:tag(lexer.STRING .. ".doublequotestring"	, dQstr	) 
	)		--[[  			 ^-- !!! line   ORDER   matters !!! ]] 
			-- 	START with highest-level category, then GO the hierarchy-levels DOWN

Below how it looks like after applying appropriate color style specifications in themes/light.lua file:

Implementation of highlighting Python-style triple quote strings can be done as follows:

local tripleSingleQuoteDelimitedTextBlock = lexer.range([[''']])
local tSQstr = tripleSingleQuoteDelimitedTextBlock

local tripleDoubleQuoteDelimitedTextBlock = lexer.range([["""]])
local tDQstr = tripleDoubleQuoteDelimitedTextBlock

... ... ...

		+	lex:tag(lexer.STRING .. ".triplesinglequoted", tSQstr )
		+	lex:tag(lexer.STRING .. ".tripledoublequoted", tDQstr ) 

Update:
In between I have simplified the code above ... For the bracketed string levels zero up to two it is sufficient to use lexer.range() instead of using lpeg.Cmt().

from textadept.

@orbitalquark : feel free to close this issue already now or close it later after updating the lexer files for Lua and Python and the theme files with the additional options available for styling the different kinds of strings and comments. If you decide to implement this above in Lua it would be a good idea to provide styling for the different levels of block comments too :) .

I enjoyed much your detailed guidance and the fast responses - thanks.

Maybe the outcome of this here will be useful also for others who want to make use of the already available features to make their coding world more colorful. Maybe it makes sense to equip the default themes with comment lines containing the additional available styling so it could be as easy as removing the leading -- to enable them (along with customization of the colors and the style)?

OK ... all of this above requires some work and results in larger code size ... no idea if it is worth the effort ... so I would let you to decide about closing this issue. From my point of view it is OK to close it now.

from textadept.

This can be done, but you'd have to modify the existing lexers from Scintillua (https://github.com/orbitalquark/scintillua) to do so. For example, the Lua lexer supports distinguishing between quoted strings and long strings:
https://github.com/orbitalquark/scintillua/blob/6c45f3bc2c3256195282318a8b34549b3041c4c0/lexers/lua.lua#L39
You can change your theme to explicitly highlight long strings as something different than normal strings:

styles.string_longstring = ...

If you don't change your theme, long strings fall back to being highlighted as normal strings (thanks to the 'strings.' prefix).

from textadept.

What would be the shortest path to know about how to and about the pitfalls of creating new tags for syntax highlighting in the language lexers in order to become able to provide the requested features?

I have already checked out the .lua lexer files. It should be relatively easy for me, especially with some guidance helping to gain the know-how about the general principles how this all works.

from textadept.

https://orbitalquark.github.io/textadept/api.html#lexer should have everything you need to know.

from textadept.

OK ... I have succeed to highlight single quoted, double quoted and in double square braces embedded longstrings with different styles, but ... WHERE can I see how the conversion of the . dot in the lexer specification is turned into an underscore in the name of the substyle defined in the theme?

themes/light.lua:

lexers/lua.lua:

The next step would be to highlight different levels of longstrings. I have some trouble to find in the documentation a clear description how the function need to be designed and how it works, so that I can specify [[ text ]], [=[ text ]=], [==[ text ]==], ... as different kinds of string literals named for example longstring, longstringLevel1, longstringLevel2, ...
What do the function parameter input, index and eq represent? Do the names of the parameter matter? What is the underlying principle of the mechanism used here?
Would it be sufficient to use a function only for longstringLevel3orMore specifying ^3 and simple [[, [=[, ... with appropriate closing patterns for the the other levels?
How to assure that the highlighting of the highest level long string has priority over lower in it embedded long string patterns?

from textadept.

OK ... I have succeed to highlight single quoted, double quoted and in double square braces embedded longstrings with different styles, but ... WHERE can I see how the conversion of the . dot in the lexer specification is turned into an underscore in the name of the substyle defined in the theme?

In code it's here:

In the API documentation (https://orbitalquark.github.io/textadept/api.html#view.styles), the suggestion is to use the "Tools > Show Style" menu item to see the name of the style under the caret. This would help discover the underscore.

The next step would be to highlight different levels of longstrings. I have some trouble to find in the documentation a clear description how the function need to be designed and how it works, so that I can specify [[ text ]], [=[ text ]=], [==[ text ]==], ... as different kinds of string literals named for example longstring, longstringLevel1, longstringLevel2, ... What do the function parameter input, index and eq represent?

https://www.inf.puc-rio.br/~roberto/lpeg/#matchtime

eq is the captured '='s string.

Do the names of the parameter matter?

No.

What is the underlying principle of the mechanism used here?

Detect any '=' between two '['s, and search for that many '='s between two ']'s later in the string. Return the position of the match if it exists.

Would it be sufficient to use a function only for longstringLevel3orMore specifying ^3 and simple [[, [=[, ... with appropriate closing patterns for the the other levels?

You could, but then you wouldn't match 4 '='s. The existing code can match any number of '='s.

How to assure that the highlighting of the highest level long string has priority over lower in it embedded long string patterns?

Good question. I'm not sure how you'd highlight nested longstrings in the current framework, or if it's even possible.

from textadept.