mirrors/ladybird
0
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2025-09-14 05:22:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9

Tests: Enable all screenshot tests on all platforms
Some checks are pending
CI / Linux, x86_64, Fuzzers, Clang (push) Waiting to run
Details
CI / Linux, x86_64, Sanitizer, GNU (push) Waiting to run
Details
CI / macOS, arm64, Sanitizer, Clang (push) Waiting to run
Details
CI / Linux, x86_64, Sanitizer, Clang (push) Waiting to run
Details
Package the js repl as a binary artifact / Linux, arm64 (push) Waiting to run
Details
Package the js repl as a binary artifact / macOS, arm64 (push) Waiting to run
Details
Package the js repl as a binary artifact / Linux, x86_64 (push) Waiting to run
Details
Run test262 and test-wasm / run_and_update_results (push) Waiting to run
Details
Lint Code / lint (push) Waiting to run
Details
Label PRs with merge conflicts / auto-labeler (push) Waiting to run
Details
Push notes / build (push) Waiting to run
Details

Browse source 
With the newly supported fuzzy matching in our test-web runner, we can
now define the expected maximum color channel and pixel count errors per
failing test and set a baseline they should not exceed.

The figures I added to these tests all come from my macOS M4 machine.
Most discrepancies seem to come from color calculations being slightly
off.
This commit is contained in:
Jelle Raaijmakers 2025-07-16 12:14:09 +02:00 • committed by Tim Ledbetter
commit 59a867d3e3
Notes: github-actions[bot] 2025-07-17 12:00:22 +00:00
44 changed files with 44 additions and 57 deletions
Download patch file Download diff file Expand all files Collapse all files

1
Tests/LibWeb/Screenshot/input/border-radius.html
View file

@ -1,6 +1,7 @@
<!DOCTYPE html>
<html>
<link rel="match" href="../expected/border-radius-ref.html" />
<meta name="fuzzy" content="maxDifference=0-8;totalPixels=0-28801">
<head>
<style>
.box {

1
Tests/LibWeb/Screenshot/input/canvas-arcs-and-ellipses.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/canvas-arcs-and-ellipses-ref.html" />
<meta name="fuzzy" content="maxDifference=0-5;totalPixels=0-567">
<style>
canvas {
border: 1px solid black;

1
Tests/LibWeb/Screenshot/input/canvas-fillstyle-gradients.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/canvas-fillstyle-gradients-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-6">
<style>
* {
margin: 0;

1
Tests/LibWeb/Screenshot/input/canvas-filters.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/canvas-filters-ref.html"/>
<meta name="fuzzy" content="maxDifference=0-6;totalPixels=0-17784">
<style>
* {
margin: 0;

1
Tests/LibWeb/Screenshot/input/canvas-implict-moves-and-lines.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/canvas-implict-moves-and-lines-ref.html" />
<meta name="fuzzy" content="maxDifference=0-2;totalPixels=0-106">
<canvas id="canvas" width="300" height="300"></canvas>
<style>
#canvas {

1
Tests/LibWeb/Screenshot/input/canvas-path-rect.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/canvas-path-rect-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-148">
<style>
* {
margin: 0;

1
Tests/LibWeb/Screenshot/input/canvas-shadow.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/canvas-shadow-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-809">
<style>
* {
margin: 0;

1
Tests/LibWeb/Screenshot/input/canvas-text.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/canvas-text-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-563">
<html>
<head>
<style>

1
Tests/LibWeb/Screenshot/input/clip-path-transformed.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/clip-path-transformed-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-15">
<style>
body {
font-size: 0;

1
Tests/LibWeb/Screenshot/input/color-scheme.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/color-scheme-ref.html" />
<meta name="fuzzy" content="maxDifference=0-3;totalPixels=0-4638">
<style>
body>div>div {
padding: 1em;

1
Tests/LibWeb/Screenshot/input/css-background-clip-text.html
View file

@ -4,6 +4,7 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="match" href="../expected/css-background-clip-text-ref.html" />
<meta name="fuzzy" content="maxDifference=0-3;totalPixels=0-25921">
<title>Document</title>
<style>

1
Tests/LibWeb/Screenshot/input/css-background-repeat.html
View file

@ -2,6 +2,7 @@
<html>
<head>
<link rel="match" href="../expected/css-background-repeat-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-8">
<style>
body {
background: cyan;

1
Tests/LibWeb/Screenshot/input/css-backgrounds.html
View file

@ -4,6 +4,7 @@
<meta charset="UTF-8" />
<title>CSS Background Tests</title>
<link rel="match" href="../expected/css-backgrounds-ref.html" />
<meta name="fuzzy" content="maxDifference=0-21;totalPixels=0-36803">
<style>
.box {
width: 180px;

1
Tests/LibWeb/Screenshot/input/css-color-functions.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/css-color-functions-ref.html" />
<meta name="fuzzy" content="maxDifference=0-2;totalPixels=0-145632">
<div style="background-color: rgb(10, 20, 30);">
<p>legacy rgb with numbers</p>
</div>

1
Tests/LibWeb/Screenshot/input/css-compositing.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/css-compositing-ref.html" />
<meta name="fuzzy" content="maxDifference=0-3;totalPixels=0-2915">
<style>
div {
position: absolute;

1
Tests/LibWeb/Screenshot/input/css-filter-drop-shadow.html
View file

@ -1,6 +1,7 @@
<!DOCTYPE html>
<html>
<link rel="match" href="../expected/css-filter-drop-shadow-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-414">
<style>
div {
height: 20px;

1
Tests/LibWeb/Screenshot/input/css-filter.html
View file

@ -1,6 +1,7 @@
<!DOCTYPE html>
<html>
<link rel="match" href="../expected/css-filter-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-1063">
<style>
body {
font-size: 0;

1
Tests/LibWeb/Screenshot/input/image-unpremultiplied-data.html
View file

@ -1,3 +1,4 @@
<!DOCTYPE html>
<link rel="match" href="../expected/image-unpremultiplied-data-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-1332">
<img style="width: 200px; height: 200px" src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADIAAAAyCAYAAAAeP4ixAAAAxUlEQVRoge2Z0Q6EIAwEwf//Z+5pE0P0BCxlLTvvKOOSqm1KQghhRSmpzL5HtrzYyIZzttmDyUUsnvhboVeLZxyZUaGhRS5nvlOoW8RDAvTIdIl4SoBWmWaRFRKgRaZJZKUEeJI5vDYym8dEGNIA/1LZIxGmNMBdKvETYUwDXKUSJhGJsCERNi6rFnPFAnXlUvllQyJsSIQNfcazoV9dNtQOYmOvTiMI0fsFIbrxIMR85MznJ1Y1n58h1qyc6rrA8EIVYnd+9SZLMlMCtbAAAAAASUVORK5CYII=">

1
Tests/LibWeb/Screenshot/input/inline-node.html
View file

@ -2,6 +2,7 @@
<html lang="en">
<head>
<link rel="match" href="../expected/inline-node-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-930">
<meta charset="UTF-8">
<title>Inline Node styling test</title>
<style>

1
Tests/LibWeb/Screenshot/input/nested-boxes-with-hidden-overflow-and-border-radius.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/nested-boxes-with-hidden-overflow-and-border-radius-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-153">
<style>
.outer {
overflow: hidden;

1
Tests/LibWeb/Screenshot/input/object-fit-position.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/object-fit-position-ref.html"/>
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-316">
<style>
.images img {
border: 1px solid black;

1
Tests/LibWeb/Screenshot/input/opacity-stacking.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/opacity-stacking-ref.html" />
<meta name="fuzzy" content="maxDifference=0-2;totalPixels=0-59533">
<style>
body {
background-color: white;

1
Tests/LibWeb/Screenshot/input/outer-box-shadow.html
View file

@ -1,6 +1,7 @@
<!DOCTYPE html>
<html>
<link rel="match" href="../expected/outer-box-shadow-ref.html" />
<meta name="fuzzy" content="maxDifference=0-2;totalPixels=0-19189">
<head>
<style>
.box {

1
Tests/LibWeb/Screenshot/input/selection-start-in-end-node-2.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/selection-start-in-end-node-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-230">
<!-- The space after the start node, as well as "NOT SELECTED" must not be selected. -->
<span id="end">End Node <span id="start">Start Node</span> </span>NOT SELECTED
<script>

1
Tests/LibWeb/Screenshot/input/selection-start-in-end-node-3.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/selection-start-in-end-node-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-230">
<!-- here the end node contains the start node, as well as some further selected content and some non-selected content. -->
<span id="end">End Node <span id="start">Start</span> <span>Node</span> </span>NOT SELECTED
<script>

1
Tests/LibWeb/Screenshot/input/selection-start-in-end-node.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/selection-start-in-end-node-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-230">
<!-- The text after the end node must not be selected. -->
<span id="end">End Node <span id="start">Start Node</span></span> NOT SELECTED
<script>

1
Tests/LibWeb/Screenshot/input/svg-background-no-natural-size.html
View file

@ -1,5 +1,6 @@
<!doctype html>
<link rel="match" href="../expected/svg-background-no-natural-size-ref.html" />
<meta name="fuzzy" content="maxDifference=0-2;totalPixels=0-455">
<style>
body {
background-color: white;

1
Tests/LibWeb/Screenshot/input/svg-clip-path-and-mask.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/svg-clip-path-and-mask-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-12">
<style>
body {
background-color: white;

1
Tests/LibWeb/Screenshot/input/svg-clip-path-transform.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/svg-clip-path-transform-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-5">
<svg>
<g id="root" transform="scale(.5 .5)">
<g id="maskContainer" transform="scale(.5 2)">

1
Tests/LibWeb/Screenshot/input/svg-filters-lb-website.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/svg-filters-lb-website-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-212768">
<svg width="714" height="749" viewBox="0 0 714 749" fill="none" xmlns="http://www.w3.org/2000/svg">
<g filter="url(#filter0_f_27_183)">
<circle cx="88.5" cy="100.5" r="278.5" fill="#8A64E5" fill-opacity="0.21"/>

1
Tests/LibWeb/Screenshot/input/svg-gradient-paint-transformation.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/svg-gradient-paint-transformation-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-231">
<svg height="150" width="150" xmlns="http://www.w3.org/2000/svg">
<style>
.hi {

1
Tests/LibWeb/Screenshot/input/svg-gradient-spreadMethod.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/svg-gradient-spreadMethod-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-68">
<style>
* {
margin: 0;

1
Tests/LibWeb/Screenshot/input/svg-maskContentUnits.html
View file

@ -1,4 +1,5 @@
<!DOCTYPE html>
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-33">
<svg width="200" viewBox="0 0 100 100" xmlns="http://www.w3.org/2000/svg">
<link rel="match" href="../expected/svg-maskContentUnits-ref.html" />
<mask id="myMask1" maskContentUnits="userSpaceOnUse">

1
Tests/LibWeb/Screenshot/input/svg-path-offset-rounding.html
View file

@ -1,5 +1,6 @@
<!doctype html>
<link rel="match" href="../expected/svg-path-offset-rounding-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-1">
<style>
body {
padding-left: 10.6px;

1
Tests/LibWeb/Screenshot/input/svg-paths-cardinal-directions-less-than-1px-wide.html
View file

@ -2,6 +2,7 @@
<html>
<head>
<link rel="match" href="../expected/svg-paths-cardinal-directions-less-than-1px-wide-ref.html"/>
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-160">
<style>
img, svg {
width: 24px;

1
Tests/LibWeb/Screenshot/input/svg-radialGradient.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/svg-radialGradient-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-1509">
<style>
* {
margin: 0;

1
Tests/LibWeb/Screenshot/input/svg-stroke-paintstyle-with-opacity.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/svg-stroke-paintstyle-with-opacity-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-2540">
<style>
body {
background-color: white;

1
Tests/LibWeb/Screenshot/input/svg-stroke-styles.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/svg-stroke-styles-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-450">
<style>
path {
stroke-width: 20;

1
Tests/LibWeb/Screenshot/input/svg-text-effects.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/svg-text-effects-ref.html" />
<meta name="fuzzy" content="maxDifference=0-1;totalPixels=0-717">
<svg height="200" width="350" xmlns="http://www.w3.org/2000/svg">
<style>
.text {

1
Tests/LibWeb/Screenshot/input/svg-textPath.html
View file

@ -1,5 +1,6 @@
<!DOCTYPE html>
<link rel="match" href="../expected/svg-textPath-ref.html" />
<meta name="fuzzy" content="maxDifference=0-2;totalPixels=0-355">
<svg width="400" viewBox="0 0 100 100" xmlns="http://www.w3.org/2000/svg">
<path
id="path"

1
Tests/LibWeb/Screenshot/input/text-decorations.html
View file

@ -2,6 +2,7 @@
<html>
<head>
<link rel="match" href="../expected/text-decorations-ref.html" />
<meta name="fuzzy" content="maxDifference=0-2;totalPixels=0-1342">
<style>
.overline { text-decoration: wavy blue overline 2px; }
.underline { text-decoration: red underline double; }

2
Tests/LibWeb/Screenshot/input/text-shadow.html
View file

@ -1,9 +1,9 @@
<!DOCTYPE html>
<html>
<head>
<title>Text-Shadow</title>
<link rel="match" href="../expected/text-shadow-ref.html" />
<meta name="fuzzy" content="maxDifference=0-2;totalPixels=0-4066">
<style>
* {
margin: 0;

45
Tests/LibWeb/TestConfig.ini
View file

@ -320,48 +320,3 @@ Text/input/wpt-import/webaudio/the-audio-api/the-periodicwave-interface/periodic
; https://github.com/LadybirdBrowser/ladybird/issues/5333
Text/input/wpt-import/webaudio/the-audio-api/the-audionode-interface/audionode-disconnect.html
Text/input/wpt-import/webaudio/the-audio-api/the-audionode-interface/audionode-disconnect-audioparam.html
[Skipped:arm64]
; These Screenshot tests fail because of tiny rendering differences by Skia between Linux @ x86_64 and Linux/macOS @ arm64.
Screenshot/input/border-radius.html
Screenshot/input/canvas-arcs-and-ellipses.html
Screenshot/input/canvas-fillstyle-gradients.html
Screenshot/input/canvas-filters.html
Screenshot/input/canvas-implict-moves-and-lines.html
Screenshot/input/canvas-path-rect.html
Screenshot/input/canvas-shadow.html
Screenshot/input/canvas-text.html
Screenshot/input/clip-path-transformed.html
Screenshot/input/color-scheme.html
Screenshot/input/css-background-clip-text.html
Screenshot/input/css-background-repeat.html
Screenshot/input/css-backgrounds.html
Screenshot/input/css-color-functions.html
Screenshot/input/css-compositing.html
Screenshot/input/css-filter-drop-shadow.html
Screenshot/input/css-filter.html
Screenshot/input/image-unpremultiplied-data.html
Screenshot/input/inline-node.html
Screenshot/input/nested-boxes-with-hidden-overflow-and-border-radius.html
Screenshot/input/object-fit-position.html
Screenshot/input/opacity-stacking.html
Screenshot/input/outer-box-shadow.html
Screenshot/input/selection-start-in-end-node-2.html
Screenshot/input/selection-start-in-end-node-3.html
Screenshot/input/selection-start-in-end-node.html
Screenshot/input/svg-background-no-natural-size.html
Screenshot/input/svg-clip-path-and-mask.html
Screenshot/input/svg-clip-path-transform.html
Screenshot/input/svg-filters-lb-website.html
Screenshot/input/svg-gradient-paint-transformation.html
Screenshot/input/svg-gradient-spreadMethod.html
Screenshot/input/svg-maskContentUnits.html
Screenshot/input/svg-path-offset-rounding.html
Screenshot/input/svg-paths-cardinal-directions-less-than-1px-wide.html
Screenshot/input/svg-radialGradient.html
Screenshot/input/svg-stroke-paintstyle-with-opacity.html
Screenshot/input/svg-stroke-styles.html
Screenshot/input/svg-text-effects.html
Screenshot/input/svg-textPath.html
Screenshot/input/text-decorations.html
Screenshot/input/text-shadow.html

13
Tests/LibWeb/test-web/main.cpp
View file

@ -67,19 +67,10 @@ static ErrorOr<void> load_test_config(StringView test_root_path)
}
auto config = config_or_error.release_value();
auto add_to_skipped_tests = [&](auto const& group) -> ErrorOr<void> {
for (auto& key : config->keys(group))
s_skipped_tests.append(TRY(FileSystem::real_path(LexicalPath::join(test_root_path, key).string())));
return {};
};
for (auto const& group : config->groups()) {
if (group == "Skipped"sv) {
TRY(add_to_skipped_tests(group));
} else if (group == "Skipped:arm64"sv) {
#if ARCH(AARCH64)
TRY(add_to_skipped_tests(group));
#endif
for (auto& key : config->keys(group))
s_skipped_tests.append(TRY(FileSystem::real_path(LexicalPath::join(test_root_path, key).string())));
} else {
warnln("Unknown group '{}' in config {}", group, config_path);
}