mirror of
https://github.com/LadybirdBrowser/ladybird.git
synced 2025-10-20 06:59:47 +00:00
00182a2405
This ports the lexer to UTF-16 and deals with the immediate fallout up to the AST. The AST will be dealt with in upcoming commits. The lexer will still accept UTF-8 strings as input, and will transcode them to UTF-16 for lexing. This doesn't actually incur a new allocation, as we were already converting the input StringView to a ByteString for each lexer. One immediate logical benefit here is that we do not need to know off- hand how many UTF-8 bytes some special code points occupy. They all happen to be a single UTF-16 code unit. So instead of advancing the lexer by 3 positions in some cases, we can just always advance by 1. |
||
|---|---|---|
| .. | ||
| Randomized | ||
| AssertionHandler.cpp | ||
| AssertionHandler.h | ||
| CMakeLists.txt | ||
| JavaScriptTestRunner.h | ||
| JavaScriptTestRunnerMain.cpp | ||
| Macros.h | ||
| Results.h | ||
| TestCase.h | ||
| TestMain.cpp | ||
| TestResult.h | ||
| TestRunner.h | ||
| TestRunnerUtil.h | ||
| TestSuite.cpp | ||
| TestSuite.h | ||