This also refactors interpreter creation to follow
InitializeHostDefinedRealm, but I couldn't fit it in the title :^)
This allows us to follow the spec much more closely rather than being
completely ad-hoc with just the parse node instead of having all the
surrounding data such as the realm of the parse node.
The interpreter creation refactor creates the global execution context
once and doesn't take it off the stack. This allows LibWeb to take the
global execution context and manually handle it, following the HTML
spec. The HTML spec calls this the "realm execution context" of the
environment settings object.
It also allows us to specify the globalThis type, as it can be
different from the global object type. For example, on the web, Window
global objects use a WindowProxy global this value to enforce the same
origin policy on operations like [[GetOwnProperty]].
Finally, it allows us to directly call Program::execute in perform_eval
and perform_shadow_realm_eval as this moves
global_declaration_instantiation into Interpreter::run
(ScriptEvaluation) as per the spec.
Note that this doesn't evalulate Source Text Modules yet or refactor
the bytecode interpreter, that's work for future us :^)
This patch was originally build by Luke for the environment settings
object change but was also needed for modules. So I (davidot) have
modified it with the new completion changes and setup for that.
Co-authored-by: davidot <davidot@serenityos.org>
Instead of making it a void function, checking for an exception, and
then receiving the relevant result via VM::last_value(), we can
consolidate all of this by using completions.
This allows us to remove more uses of VM::exception(), and all uses of
VM::last_value().
This includes:
- Parsing proper LabelledStatements with try_parse_labelled_statement()
- Removing LabelableStatement
- Implementing the LoopEvaluation semantics via loop_evaluation() in
each IterationStatement subclass; and IterationStatement evaluation
via {For,ForIn,ForOf,ForAwaitOf,While,DoWhile}Statement::execute()
- Updating ReturnStatement, BreakStatement and ContinueStatement to
return the appropriate completion types
- Basically reimplementing TryStatement and SwitchStatement according to
the spec, using completions
- Honoring result completion types in AsyncBlockStart and
OrdinaryCallEvaluateBody
- Removing any uses of the VM unwind mechanism - most importantly,
VM::throw_exception() now exclusively sets an exception and no longer
triggers any unwinding mechanism.
However, we already did a good job updating all of LibWeb and userland
applications to not use it, and the few remaining uses elsewhere don't
rely on unwinding AFAICT.
This is another major milestone on our journey towards removing global
VM exception state :^)
Does pretty much exactly what it says on the tin: updating
ASTNode::execute() to return a Completion instead of a plain value. This
will *also* allow us to eventually remove the non-standard unwinding
mechanism and purely rely on the various completion types.
This meant that if some program threw an uncaught exception VM still
had unwind_until set. This caused any further programs to not execute
correctly.
This will be fixed more thoroughly once we use Completions in the AST.
Fixes#10323
Before this we used an ad-hoc combination of references and 'variables'
stored in a hashmap. This worked in most cases but is not spec like.
Additionally hoisting, dynamically naming functions and scope analysis
was not done properly.
This patch fixes all of that by:
- Implement BindingInitialization for destructuring assignment.
- Implementing a new ScopePusher which tracks the lexical and var
scoped declarations. This hoists functions to the top level if no
lexical declaration name overlaps. Furthermore we do checking of
redeclarations in the ScopePusher now requiring less checks all over
the place.
- Add methods for parsing the directives and statement lists instead
of having that code duplicated in multiple places. This allows
declarations to pushed to the appropriate scope more easily.
- Remove the non spec way of storing 'variables' in
DeclarativeEnvironment and make Reference follow the spec instead of
checking both the bindings and 'variables'.
- Remove all scoping related things from the Interpreter. And instead
use environments as specified by the spec. This also includes fixing
that NativeFunctions did not produce a valid FunctionEnvironment
which could cause issues with callbacks and eval. All
FunctionObjects now have a valid NewFunctionEnvironment
implementation.
- Remove execute_statements from Interpreter and instead use
ASTNode::execute everywhere this simplifies AST.cpp as you no longer
need to worry about which method to call.
- Make ScopeNodes setup their own environment. This uses four
different methods specified by the spec
{Block, Function, Eval, Global}DeclarationInstantiation with the
annexB extensions.
- Implement and use NamedEvaluation where specified.
Additionally there are fixes to things exposed by these changes to eval,
{for, for-in, for-of} loops and assignment.
Finally it also fixes some tests in test-js which where passing before
but not now that we have correct behavior :^).
Since there are only a number of statements where labels can actually be
used we now also only store labels when necessary.
Also now tracks the first continue usage of a label since this might not
be valid but that can only be determined after we have parsed the
statement.
Also ensures the correct error does not get wiped by load_state.
The old name is the result of the perhaps somewhat confusingly named
abstract operation OrdinaryFunctionCreate(), which creates an "ordinary
object" (https://tc39.es/ecma262/#ordinary-object) in contrast to an
"exotic object" (https://tc39.es/ecma262/#exotic-object).
However, the term "Ordinary Function" is not used anywhere in the spec,
instead the created object is referred to as an "ECMAScript Function
Object" (https://tc39.es/ecma262/#sec-ecmascript-function-objects), so
let's call it that.
The "ordinary" vs. "exotic" distinction is important because there are
also "Built-in Function Objects", which can be either implemented as
ordinary ECMAScript function objects, or as exotic objects (our
NativeFunction).
More work needs to be done to move a lot of infrastructure to
ECMAScriptFunctionObject in order to make FunctionObject nothing more
than an interface for objects that implement [[Call]] and optionally
[[Construct]].
We need both a GlobalObject and Realm now, but can get the former from
the latter (once initialized).
This also fixes JS execution in LibWeb, as we failed to set the Realm of
the newly created Interpreter in this function.
This is where the spec wants to have it. Requires a couple of hacks as
currently everything that needs a Realm actually has a GlobalObject, so
we need to go via the Interpreter.
The test262 tests under RegExp/property-escapes/generated will invoke
Reflect.apply with up to 10,000 arguments at a time. In LibJS, when the
call stack reached VM::call_internal, we transfer those arguments from
a MarkedValueList to the execution context's arguments Vector.
Because these types differ (MarkedValueList is a Vector<Value, 32>), the
arguments are copied rather than moved. By changing the arguments vector
to a MarkedValueList, we can properly move the passed arguments over.
This shaves about 2 seconds off the following test262 test (from 15sec):
RegExp/property-escapes/generated/General_Category_-_Decimal_Number.js
This removes all usages of the non-standard put helper method and
replaces all of it's usages with the specification required alternative
or with define_direct_property where appropriate.
These represent the outermost scope in the environment record
hierarchy. The spec says they should be a "composite" of two things:
- An ObjectEnvironmentRecord wrapping the global object
- A DeclarativeEnvironmentRecord for other declarations
It's not yet clear to me how this should work, so this patch only
implements the first part, an object record wrapping the global object.
To better follow the spec, we need to distinguish between the current
execution context's lexical environment and variable environment.
This patch moves us to having two record pointers, although both of
them point at the same environment records for now.
This patch adds FunctionEnvironmentRecord as a subclass of the existing
DeclarativeEnvironmentRecord. Things that are specific to function
environment records move into there, simplifying the base.
Most of the abstract operations related to function environment records
are rewritten to match the spec exactly. I also had to implement
GetThisEnvironment() and GetSuperConstructor() to keep tests working
after the changes, so that's nice as well. :^)
This patch makes the following renames:
- get_from_scope() => get_from_environment_record()
- put_to_scope() => put_into_environment_record()
- delete_from_scope() => delete_from_environment_record()
This patch makes the following name changes:
- ScopeObject => EnvironmentRecord
- LexicalEnvironment => DeclarativeEnvironmentRecord
- WithScope => ObjectEnvironmentRecord
This counter is increased each time a synchronous execution sequence
completes, and will allow us to emulate the abstract operations
AddToKeptObjects & ClearKeptObjects efficiently.
"let" and "const" go in the lexical environment.
This fixes one part of #4001 (Lexically declared variables are mixed up
with global object properties)
When using VM::set_variable() to put the created ScriptFunction onto a
ScopeObject, we would previously unexpectedly reach the global object as
set_variable() checks each traversed scope for an existing Variable with
the given name - which would cause a leak of the inner function past the
outer function (we even had a test expecting that behaviour!). Now we
first declare functions (as DeclarationKind::Var) before setting them.
This will need some more work to make hoisting across non-lexical scopes
work, but it fixes this specific issue for now.
Fixes#6766.
Instead of having to run queued promise jobs in LibWeb in various
places, this allows us to consolidate that into one function - this is
very close to how the spec describes it as well ("at some future point
in time, when there is no running execution context and the execution
context stack is empty, the implementation must [...]").
Eventually this will also be used to log unhandled exceptions, and
possibly other actions that require JS execution to have ended.
SPDX License Identifiers are a more compact / standardized
way of representing file license information.
See: https://spdx.dev/resources/use/#identifiers
This was done with the `ambr` search and replace tool.
ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
Almost a year after first working on this, it's finally done: an
implementation of Promises for LibJS! :^)
The core functionality is working and closely following the spec [1].
I mostly took the pseudo code and transformed it into C++ - if you read
and understand it, you will know how the spec implements Promises; and
if you read the spec first, the code will look very familiar.
Implemented functions are:
- Promise() constructor
- Promise.prototype.then()
- Promise.prototype.catch()
- Promise.prototype.finally()
- Promise.resolve()
- Promise.reject()
For the tests I added a new function to test-js's global object,
runQueuedPromiseJobs(), which calls vm.run_queued_promise_jobs().
By design, queued jobs normally only run after the script was fully
executed, making it improssible to test handlers in individual test()
calls by default [2].
Subsequent commits include integrations into LibWeb and js(1) -
pretty-printing, running queued promise jobs when necessary.
This has an unusual amount of dbgln() statements, all hidden behind the
PROMISE_DEBUG flag - I'm leaving them in for now as they've been very
useful while debugging this, things can get quite complex with so many
asynchronously executed functions.
I've not extensively explored use of these APIs for promise-based
functionality in LibWeb (fetch(), Notification.requestPermission()
etc.), but we'll get there in due time.
[1]: https://tc39.es/ecma262/#sec-promise-objects
[2]: https://tc39.es/ecma262/#sec-jobs-and-job-queues
For various statements the spec states:
Return NormalCompletion(empty).
In those cases we have been returning undefined so far, which is
incorrect.
In other cases it states:
Return Completion(UpdateEmpty(stmtCompletion, undefined)).
Which essentially means a statement is evaluated and its completion
value returned if non-empty, and undefined otherwise.
While not actually noticeable in normal scripts as the VM's "last value"
can't be accessed from JS code directly (with the exception of eval(),
see below), it provided an inconsistent experience in the REPL:
> if (true) 42;
42
> if (true) { 42; }
undefined
This also fixes the case where eval() would return undefined if the last
executed statement is not a value-producing one:
eval("1;;;;;")
eval("1;{}")
eval("1;var a;")
As a consequence of the changes outlined above, these now all correctly
return 1.
See https://tc39.es/ecma262/#sec-block-runtime-semantics-evaluation,
"NOTE 2".
Fixes#3609.
