We have a new, improved string type coming up in AK (OOM aware, no null
state), and while it's going to use UTF-8, the name UTF8String is a
mouthful - so let's free up the String name by renaming the existing
class.
Making the old one have an annoying name will hopefully also help with
quick adoption :^)
This moves all code comprehension-related code to a new library,
LibCodeComprehension.
This also moves some types related to code comprehension tasks (such as
autocomplete, find declaration) out of LibGUI and into
LibCodeComprehension.
Previously, the end position of the Type and Name nodes was incorrect.
It was incorrectly set to the start position of the next unconsumed
token.
This commit adds a previous_token_end() method and uses it to correctly
get the end position for these node types.
When the preprocessor encounters an #include statement it now adds
the preprocessor definitions that exist in the included header to its
own set of definitions.
We previously only aggregated the definitions from headers after
processing the source, which was less correct. (For example, there
could be an #ifdef that depends on a definition from another header).
We now call Preprocessor::process_and_lex() and pass the result to the
parser.
Doing the lexing in the preprocessor will allow us to maintain the
original position information of tokens after substituting definitions.
We previously stored the entire ASTNode vector in each parser state,
and this vector was copied whenever a state was loaded or saved.
We don't actually need to store the whole nodes list in each state
because a new state can only add new nodes to this list, and won't
mutate existing nodes.
It would suffice to only hold a vector of the nodes that were created
while parsing in the current state to keep a reference to them.
This reduces the time it takes on my machine for the c++ language
server to handle a file that #includes <LibGUI/Widget.h> from ~4sec to
~0.7sec.
There's no need to store parser error messages for states with
depth > 0, as they will eventually be popped from the states stack and
their error messages will never be displayed to the user.
Profiling shows that this change reduces the % of backtraces that
contain the store_state & load_state functions from ~95% to ~70%.
Empirically this change reduces the time it takes on my machine for the
c++ language server to handle a file that #includes <LibGUI/Widget.h>
from ~14sec to ~4sec.
This adds a new ASTNode type called 'NamedType' which inherits from
the Type node.
Previously every Type node had a name field, but it was not logically
accurate. For example, pointer types do not have a name
(the pointed-to type may have one).
We can now handle access-specifier tags (for example 'private:') when
parsing class declarations.
We currently only consume these tags on move on. We'll need to add some
logic that accounts for the access level of symbols down the road.
Previously, we had a special ASTNode for class members,
"MemberDeclaration", which only represented fields.
This commit removes MemberDeclaration and instead uses regular
Declaration nodes for representing the members of a class.
This means that we can now also parse methods, inner-classes, and other
declarations that appear inside of a class.
SPDX License Identifiers are a more compact / standardized
way of representing file license information.
See: https://spdx.dev/resources/use/#identifiers
This was done with the `ambr` search and replace tool.
ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
This type represents templatized names, and replaces our previous use
of 'TemplatizedType' and 'TemplatizedFunctionCall'.
Also, we now parse function calls as secondary expressions.
After we moved to storing the text of each token in the token itself,
we no longer have to store the source of the program in the Parser.
This makes more sense because the parser should deal with tokens, not
with raw source code.
We now also look at the available declarations from included header
files when autocompleting names.
Additionally, you can now request autocomplete on an empty token, which
brings up all available names, starting from the inner-most scope.
This parser will be used by the C++ langauge server to provide better
auto-complete (& maybe also other things in the future).
It is designed to be error tolerant, and keeps track of the position
spans of the AST nodes, which should be useful later for incremental
parsing.
