The SemanticSyntaxHighlighter uses TokenInfo results from the
language server to provide 'semantic' syntax highlighting, which
provides more fin-grained text spans results.
For example, the SemanticSyntaxHighlighter can color function calls,
member fields references and user-defined types in different colors.
With the simple lexer-only syntax highlighter, all of these tokens were
given the same text highlighting span type.
Since we have to provide immediate highlighting feedback to the user
after each edit and before we get the result for the language server,
we use a heuristic which computes the diff between the current tokens
and the last known tokens with compete semantic information
(We use LibDiff for this).
This heuristic is not very performant, and starts feeling sluggish with
bigger (~200 LOC) files.
A possible future improvement would be only computing the diff for
tokens in text ranges that have changes since the last commit.
And use them to highlight javascript in HTML source.
This commit also changes how TextDocumentSpan::data is interpreted,
as it used to be an opaque pointer, but everyone stuffed an enum value
inside it, which made the values not unique to each highlighter;
that field is now a u64 serial id.
The syntax highlighters don't need to change their ways of stuffing
token types into that field, but a highlighter that calls another
nested highlighter needs to register the nested types for use with
token pairs.
SPDX License Identifiers are a more compact / standardized
way of representing file license information.
See: https://spdx.dev/resources/use/#identifiers
This was done with the `ambr` search and replace tool.
ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
