Commit graph

13 commits

Author SHA1 Message Date
Tim Ledbetter
168ea88cb2 test-web: Add an option to randomize the test order 2025-07-28 09:01:56 +02:00
Tim Ledbetter
bd83f5bde6 test-web: Treat files in images directories as resources for WPT tests 2025-07-20 16:26:57 +12:00
Jelle Raaijmakers
59a867d3e3 Tests: Enable all screenshot tests on all platforms
Some checks are pending
CI / Linux, x86_64, Fuzzers, Clang (push) Waiting to run
CI / Linux, x86_64, Sanitizer, GNU (push) Waiting to run
CI / macOS, arm64, Sanitizer, Clang (push) Waiting to run
CI / Linux, x86_64, Sanitizer, Clang (push) Waiting to run
Package the js repl as a binary artifact / Linux, arm64 (push) Waiting to run
Package the js repl as a binary artifact / macOS, arm64 (push) Waiting to run
Package the js repl as a binary artifact / Linux, x86_64 (push) Waiting to run
Run test262 and test-wasm / run_and_update_results (push) Waiting to run
Lint Code / lint (push) Waiting to run
Label PRs with merge conflicts / auto-labeler (push) Waiting to run
Push notes / build (push) Waiting to run
With the newly supported fuzzy matching in our test-web runner, we can
now define the expected maximum color channel and pixel count errors per
failing test and set a baseline they should not exceed.

The figures I added to these tests all come from my macOS M4 machine.
Most discrepancies seem to come from color calculations being slightly
off.
2025-07-17 12:59:11 +01:00
Jelle Raaijmakers
0d856a8fa7 Tests: Implement fuzzy screenshot comparisons in test-web
We now read WPT's `<meta name="fuzzy">` tags if present in tests and
apply them while comparing screenshots.
2025-07-17 12:59:11 +01:00
Jelle Raaijmakers
e4b2253b63 Tests: Replace load-reference-page debug action with internals method
WPT reference tests can add metadata to tests to instruct the test
runner how to interpret the results. Because of this, it is not enough
to have an action that starts loading the (mis)match reference: we need
the test runner to receive the metadata so it can act accordingly.

This sets our test runner up for potentially supporting multiple
(mis)match references, and fuzzy rendering matches - the latter will be
implemented in the following commit.
2025-07-17 12:59:11 +01:00
Jelle Raaijmakers
0f642ecb5c LibGfx: Replace Bitmap::visually_equals() with ::diff()
This produces more granural information on the actual pixel errors
present between two bitmaps. It now also asserts that both bitmaps are
the same size, since we should never compare two differently sized
bitmaps anyway.
2025-07-17 12:59:11 +01:00
Jelle Raaijmakers
616a2af23d Tests: Enable screenshot tests on arm64
Of the available 71 screenshot tests that we have, 42 fail on macOS
arm64. Let's make it possible to skip those and run the remaining
succeeding screenshot tests on arm64 anyway.
2025-07-16 08:36:14 +02:00
Tim Ledbetter
e1f0284fba test-web: Wait longer before completing crash tests
Previously, a lot of imported WPT crash tests wouldn't wait long
enough for a crash to occur.
2025-07-14 13:05:17 -04:00
Andreas Kling
dab1fd265d test-web: Dump stacking context tree in layout test output
This will allow us to test (and catch regressions in) stacking context
tree construction and updates, etc.
2025-07-09 14:36:08 +02:00
Jelle Raaijmakers
ead0a2c78a Everywhere: Rename serenity_main to ladybird_main
No functional changes.
2025-07-08 09:17:16 -04:00
Tim Ledbetter
a959d4dca2 test-web: Match WPT runner reftest-wait behavior
When a test has a `reftest-wait` class, we now wait for fonts to load
and then for 2 `requestAnimationFrame` callbacks. This matches the
behavior of the WPT runner and allows more imported WPT tests to work
correctly.
2025-07-02 18:44:21 +02:00
Timothy Flynn
39da2d9a2f LibWebView+UI: Migrate some UI process init to LibWebView
No need to do this setup in every UI's main().
2025-06-11 07:26:32 -04:00
Timothy Flynn
9a5b31ccd1 LibWebView+Tests+UI: Migrate headless-browser to test-web
Now that headless mode is built into the main Ladybird executable, the
headless-browser's only purpose is to run tests. So let's move it to the
testing directory and rename it to test-web (a la test-js / test-wasm).
2025-06-10 12:04:59 -04:00
Renamed from UI/Headless/Test.cpp (Browse further)