Commit graph

45 commits

Author SHA1 Message Date
ayeteadoe
f468f702d5 CMake: Update default BUILD_PRESET arg to Release in WPT.sh
The presets were recently refactored and 'default' was
renamed to 'Release', but keeping it in sync with
any consumer shell scripts was missed.
2025-07-19 21:03:51 +02:00
Tim Ledbetter
1e30fd5c6a Meta: Add WPT.sh bisect for finding WPT regressions automatically 2025-07-17 10:51:12 +02:00
Tim Ledbetter
117ff3778a Meta: Ensure the correct log file path is used with WPT.sh compare
Previously, using abolute paths with `WPT.sh compare` would fail, as
directories were always assumed to be relative to the working directory.
2025-07-17 10:51:12 +02:00
Tim Ledbetter
e26d848dde Meta: Exit gracefully when importing multiple WPT tests is interrupted 2025-07-15 18:10:07 +02:00
Tim Ledbetter
d6f419af53 Meta: Add --wpt-base-url option to the WPT importer
This allows files to be downloaded locally by first running:
`./Meta/WPT.sh serve` then importing with
`--wpt-base-url http://web-platform.test:8000`.
2025-07-15 18:10:07 +02:00
Tim Ledbetter
c427f75b02 Meta: Include the --install-fonts argument when running WPT
This ensures the Ahem test font is installed.
2025-07-07 11:57:41 +02:00
Andrew Kaster
ffd600a7f5 Meta+RequestServer: Remove local download of ca-certificates
We haven't required a local copy of the ca-certificates since switching
to OpenSSL as the backend for TLS. Remove the script to download the
PEM file, and update the tests to use the system's CA certificates.
2025-07-07 09:10:05 +02:00
Tim Ledbetter
53adf3be1e Meta: Ensure local WPT repository is up to date before importing
The current import process assumes that the local WPT repository is up
to date. Let's make sure that's true before importing a new test.
2025-07-02 12:02:09 +01:00
Tim Ledbetter
ea29877385 Meta: Add --force option to WPT import script to update existing tests 2025-06-22 23:53:14 +02:00
Sam Atkins
a0cef5a7da Meta: Only run WPT.sh cleanup routines on Linux
These use the `readarray` command which isn't available in the older
version of Bash available on macOS. However, they're also not needed on
macOS, so let's skip them entirely.

Ref: https://github.com/LadybirdBrowser/ladybird/issues/5118
2025-06-18 13:52:46 +01:00
Timothy Flynn
9a5b31ccd1 LibWebView+Tests+UI: Migrate headless-browser to test-web
Now that headless mode is built into the main Ladybird executable, the
headless-browser's only purpose is to run tests. So let's move it to the
testing directory and rename it to test-web (a la test-js / test-wasm).
2025-06-10 12:04:59 -04:00
Timothy Flynn
c011dc766f LibWebView+WebDriver+UI: Migrate headless browsing to main Ladybird exe
We currently create a separate headless-browser application to serve two
purposes:

1. Allow headless browsing to take a screenshot of a page or print its
   layout tree / internal text.
2. Run the LibWeb test framework.

This patch migrates (1) to the main Ladybird executable. The --headless
flag enables this mode. This matches the behavior of other browsers, and
means we have one less executable to ship at distribution time.

We want to avoid creating too many AppKit / Qt facilities in headless
mode. So this involves some shuffling of application init to ensure we
don't create them until after we've parsed the command line arguments.
Namely, we avoid creating the NSApp in AppKit and QCoreApplication in
Qt. Doing so also requires that we don't create the application event
loop until we've parsed the command line as well, because the loop we
create depends on whether we're creating those UI facilities.
2025-06-10 12:04:59 -04:00
Timothy Flynn
7ce88eb4cd Everywhere: Document use of ladybird.py over ladybird.sh 2025-05-29 16:24:17 -04:00
Tim Ledbetter
7f4916b806 Meta: Ensure wptreport browser_version field is always populated
This was previously done by passing the `LADYBIRD_GIT_VERSION`
environment variable to `wpt`, but this doesn't work in chunked mode.
2025-05-15 14:02:48 +02:00
Tim Ledbetter
e2db9790b0 Meta: Use human readable timestamp in WPT run folder name
This makes it easier to figure out when a given run started.
2025-05-15 12:11:20 +02:00
Ali Mohammad Pur
43defa90cc Meta: Make WPT.sh log args relative to cwd as of script invocation
Some checks are pending
CI / Lagom (arm64, Sanitizer_CI, false, macos-15, macOS, Clang) (push) Waiting to run
CI / Lagom (x86_64, Fuzzers_CI, false, ubuntu-24.04, Linux, Clang) (push) Waiting to run
CI / Lagom (x86_64, Sanitizer_CI, false, ubuntu-24.04, Linux, GNU) (push) Waiting to run
CI / Lagom (x86_64, Sanitizer_CI, true, ubuntu-24.04, Linux, Clang) (push) Waiting to run
Package the js repl as a binary artifact / build-and-package (arm64, macos-15, macOS, macOS-universal2) (push) Waiting to run
Package the js repl as a binary artifact / build-and-package (x86_64, ubuntu-24.04, Linux, Linux-x86_64) (push) Waiting to run
Run test262 and test-wasm / run_and_update_results (push) Waiting to run
Lint Code / lint (push) Waiting to run
Label PRs with merge conflicts / auto-labeler (push) Waiting to run
Push notes / build (push) Waiting to run
This restores the behaviour from before adding parallel instance
support.
2025-05-08 23:50:04 +01:00
Ali Mohammad Pur
011982c501 Meta: Make WPT.sh run nproc/2 testrunners in parallel
Some checks are pending
CI / Lagom (arm64, Sanitizer_CI, false, macos-15, macOS, Clang) (push) Waiting to run
CI / Lagom (x86_64, Fuzzers_CI, false, ubuntu-24.04, Linux, Clang) (push) Waiting to run
CI / Lagom (x86_64, Sanitizer_CI, false, ubuntu-24.04, Linux, GNU) (push) Waiting to run
CI / Lagom (x86_64, Sanitizer_CI, true, ubuntu-24.04, Linux, Clang) (push) Waiting to run
Package the js repl as a binary artifact / build-and-package (arm64, macos-15, macOS, macOS-universal2) (push) Waiting to run
Package the js repl as a binary artifact / build-and-package (x86_64, ubuntu-24.04, Linux, Linux-x86_64) (push) Waiting to run
Run test262 and test-wasm / run_and_update_results (push) Waiting to run
Lint Code / lint (push) Waiting to run
Label PRs with merge conflicts / auto-labeler (push) Waiting to run
Push notes / build (push) Waiting to run
This is the "intended" way of parallelism with wpt, but instead of
requiring N different systems (or VMs), this does it all on one system
with the power of namespaces.
2025-05-07 15:48:21 +01:00
Ali Mohammad Pur
79c32f33e1 Meta: Reformat WPT.sh 2025-05-07 15:48:21 +01:00
Jelle Raaijmakers
a9ef5b4924 Meta: Increase max open file descriptors in WPT.sh
On some macs, the default maximum number of file descriptors is 256.
This quickly makes the WPT runner run out of descriptors, so let's check
the active value and increase the soft limit if necessary.
2025-03-25 16:24:06 -04:00
Tim Ledbetter
79da6d48c1 Meta: Strip query string from WPT test variants before importing them
The output of `WPT.sh list-tests` includes test variants, which vary
only by their query string. Since we don't care about this when
importing tests, ignore any query strings and ensure duplicates are
removed from the given test paths.
2024-12-29 01:35:13 +00:00
Tim Ledbetter
971aacc603 Meta: Don't suppress the output of the WPT importer script
Previously, messages printed to stdout by `import-wpt-test.py` were
being swallowed.
2024-12-29 01:35:13 +00:00
Tim Ledbetter
b4751826c6 Meta: Construct test list after parsing other arguments in WPT.sh
Previously, running `WPT.sh compare foo.log` would fail because the log
file name was included in the list of test paths.
2024-12-29 00:50:02 +00:00
Tim Ledbetter
34f78ca152 Meta: Add a --debug-process option to WPT.sh
This passes the `--debug-process` option to WebDriver.
2024-12-20 23:42:02 +01:00
Tim Ledbetter
93a1c45eb2 Meta: Don't halt if importing a test fails in WPT.sh 2024-12-11 16:30:20 -07:00
Tim Ledbetter
cb8a1f6bb6 Meta: Add the ability to import entire test directories with WPT.sh
This uses the `WPT.sh list-tests` command to find which tests are in
each given path.
2024-12-11 16:30:20 -07:00
Tim Ledbetter
a4f1838798 Meta: Add a list-tests command to WPT.sh
This outputs a list of all the tests that would be run in the given
test directories.
2024-12-11 16:30:20 -07:00
Tim Ledbetter
02268e9c60 Meta: Use headless-browser by default when running tests with WPT.sh
This change removes the `--headless` option, which is now the default
behavior and adds the `--show-window` option to force tests to run in a
visible browser window.
2024-11-15 07:14:07 -05:00
sideshowbarker
0a773b408a Tests: Allow wpt.live URLs for both “WPT.sh run” and “WPT.sh import”
This change allows you to give http[s]://wpt.live/ URLs to the WPT.sh
script for both the “WPT.sh run” and “WPT.sh import” commands.

That facilitates the use case where you’ve navigated to a wpt.live URL
in a browser, and you want to just directly copy-paste the URL in order
to either run the test in Ladybird, or import the test into the repo.

Otherwise, without this change, when using WPT.sh, you’re limited to
needing to specify either a WPT path fragment or filesystem pathname —
which doesn’t allow for easy copy-paste directly from wpt.fyi.
2024-11-09 15:32:51 +01:00
Tim Ledbetter
3e57f472f5 Meta: Add an option to WPT.sh to import WPT tests
This invokes the existing `import-wpt-test.py` script, then uses
`headless-browser` to create an expectation file.
2024-11-04 14:02:44 +01:00
sideshowbarker
7d3ad89603 Tests: Allow full relative or absolute pathnames for test files to run
This change allows the test list given to “WPT.sh run” to include full
filesystem relative or absolute pathnames. That facilitates using tab
completion in the shell to browse for pathnames, and also facilitates
copy-paste of full filesystem pathnames. For example:

./Meta/WPT.sh run Tests/LibWeb/WPT/wpt/dom/historical.html
./Meta/WPT.sh run /opt/ladybird/Tests/LibWeb/WPT/wpt/dom/historical.html

Otherwise, without this change, the test list can’t include full
filesystem pathnames, but is instead limited to only path fragments that
specify WPT subdirectory pathnames — which doesn’t allow for tab
completion on pathnames in the shell, nor copy-paste of full pathnames.
2024-11-04 06:35:26 +00:00
Tim Ledbetter
ed658154d2 Meta: Set LADYBIRD_GIT_VERSION environment variable in WPT.sh
This ensures the `browser_version` field of any wptreport format log
files include the current git commit hash.
2024-10-29 13:30:12 +00:00
Tim Ledbetter
d79fcceb8a Meta: Replace realpath usage with cross-platform function in WPT.sh 2024-10-29 07:30:32 -04:00
Tim Ledbetter
0a3a75cf35 Meta: Use correct WPT log file path when an absolute path is given 2024-10-28 10:22:48 -04:00
Tim Ledbetter
c544d1fa1f Meta: Use correct certificate path when invoking WPT
We now ensure that the directory for the currently selected build
preset is used as the base certificate path.
2024-10-28 10:22:48 -04:00
sideshowbarker
6b82ab06fd Meta: Make WPT.sh echo the “wpt run” invocation
To help people in troubleshooting problems when running the WPT.sh
script, this change makes the script echo to stdout the complete
“wpt run” invocation (including all the flags and path args).
2024-10-26 11:27:33 +02:00
Timothy Flynn
14c3bff5da Meta: Add a flag to WPT.sh to run Ladybird headlessly
We have more work to do before we can run WPT headlessly by default
(i.e. handling alerts). But for now, we can run it headlessly locally
with the --headless flag.
2024-10-22 04:24:31 +01:00
Timothy Flynn
bd8ab33593 Meta: Add a command to WPT.sh to execute wpt serve
Just a handy command for local testing.
2024-10-12 15:02:41 +02:00
Shannon Booth
7ea17d2fea Meta: Use a common function for getting build directory
Between WPT.sh and ladybird.sh.

This is useful to me as I set my default build configuration to Debug,
and have been hacking around with the WPT script to align with this
configuration.
2024-10-09 11:31:51 +01:00
Shannon Booth
16f68ab1bd Meta: Make LADYBIRD_SOURCE_DIR consistent between WPT.sh and ladybird.sh
ladybird.sh allows the source directory to be overriden to point to
another source directory. I am not sure if anyone is actually using this
behaviour in practise, but let's make the behaviour at least common
between the two scripts with a helper function.
2024-10-09 11:31:51 +01:00
Tim Ledbetter
35ab0a2db6 Meta: Support all available logging formats in WPT.sh script
This change allows the user to specify the format of the log file to be
generated by the `WPT.sh` script. Multiple logging arguments may now be
specified.

The supported logging arguments are: `--log-raw`, `--log-unittest`,
`--log-xunit`, `--log-html`, `--log-mach`, `--log-tbpl`,
`--log-grouped`, `--log-chromium`, `--log-wptreport` and
`--log-wptscreenshot`. These arguments act the same as the equivalent
arguments supported by `wpt run`.

The short `--log` argument may also be used as an alias for `--log-raw`.
2024-09-14 15:04:22 +02:00
Tim Ledbetter
a859b3610d Meta: Use QT_QPA_PLATFORM offscreen in WPT.sh
This has better compatibility with MacOS.
2024-08-17 07:43:29 +02:00
Tim Ledbetter
961ad3c8e9 Meta: Use correct binary paths for MacOS in WPT.sh 2024-08-17 07:43:29 +02:00
Tim Ledbetter
29886b804d Meta: Use cross-platform method to get number of CPUs in WPT.sh 2024-08-17 07:43:29 +02:00
Tim Ledbetter
819f966419 Meta: Ensure required hosts are present before running tests in WPT.sh
The Web Platform Tests runner requires that some hostnames point to
localhost when running the tests locally. We now append these hostnames
to `/etc/hosts` if they aren't already present.
2024-08-17 07:43:29 +02:00
Tim Ledbetter
c9caa4262e Meta: Add a script to run the Web Platform tests locally
This change adds a script that can run the Web Platform Tests and
compare different test runs.
2024-08-13 14:10:56 +02:00