LibWeb: Make HTML tokenizer stop at insertion point after state switch

If we reach the insertion point at the same time as we switch to another tokenizer state, we have to bail immediately without proceeding with the next code point. Otherwise we'd fetch the next token, get an EOF marker, and then proceed as if we're at the end of the input stream, even though more data may be coming (with more calls to document.write()..)
Author: https://github.com/awesomekling Commit: 69367194a6 Pull-request: https://github.com/LadybirdBrowser/ladybird/pull/2531
2025-08-07 00:29:15 +00:00 · 2024-11-23 18:30:01 +01:00 · 2024-11-23 18:30:01 +01:00 · 69367194a6 · 2024-11-23 18:20:21 +00:00
commit 69367194a6
parent 1d554f97de
18 changed files with 44 additions and 42 deletions
--- a/Libraries/LibWeb/HTML/Parser/HTMLTokenizer.cpp
+++ b/Libraries/LibWeb/HTML/Parser/HTMLTokenizer.cpp
@ -34,6 +34,8 @@ namespace Web::HTML {
    do {                                                                                          \
        will_switch_to(State::new_state);                                                         \
        m_state = State::new_state;                                                               \
        if (stop_at_insertion_point == StopAtInsertionPoint::Yes && is_insertion_point_reached()) \
            return {};                                                                            \
        CONSUME_NEXT_INPUT_CHARACTER;                                                             \
        goto new_state;                                                                           \
    } while (0)
--- a/Tests/LibWeb/Text/expected/wpt-import/html/syntax/parsing/ambiguous-ampersand.txt
+++ b/Tests/LibWeb/Text/expected/wpt-import/html/syntax/parsing/ambiguous-ampersand.txt
@ -6,9 +6,9 @@ Rerun
 Found 3 tests
-1 Pass
+2 Pass
-2 Fail
+1 Fail
 Details
-Result	Test Name	MessageFail	Check number of divs	
+Result	Test Name	MessagePass	Check number of divs	
 Pass	Check div structure: network	
-Fail	Check div structure: document.write	Cannot access property "childNodes" on undefined object "div"
+Fail	Check div structure: document.write	
--- a/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/003.txt
+++ b/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/003.txt
@ -6,6 +6,6 @@ Rerun
 Found 1 tests
-1 Fail
+1 Pass
 Details
-Result	Test Name	MessageFail	document.write	
+Result	Test Name	MessagePass	document.write	
--- a/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/007.txt
+++ b/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/007.txt
@ -6,6 +6,6 @@ Rerun
 Found 1 tests
-1 Fail
+1 Pass
 Details
-Result	Test Name	MessageFail	document.write	
+Result	Test Name	MessagePass	document.write	
--- a/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/010.txt
+++ b/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/010.txt
@ -6,6 +6,6 @@ Rerun
 Found 1 tests
-1 Fail
+1 Pass
 Details
-Result	Test Name	MessageFail	document.write	
+Result	Test Name	MessagePass	document.write	
--- a/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/011.txt
+++ b/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/011.txt
@ -6,6 +6,6 @@ Rerun
 Found 1 tests
-1 Fail
+1 Pass
 Details
-Result	Test Name	MessageFail	document.write	
+Result	Test Name	MessagePass	document.write	
--- a/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/014.txt
+++ b/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/014.txt
@ -6,6 +6,6 @@ Rerun
 Found 1 tests
-1 Fail
+1 Pass
 Details
-Result	Test Name	MessageFail	document.write	Cannot access property "firstChild" on null object "document.body"
+Result	Test Name	MessagePass	document.write	
--- a/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/015.txt
+++ b/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/015.txt
@ -6,6 +6,6 @@ Rerun
 Found 1 tests
-1 Fail
+1 Pass
 Details
-Result	Test Name	MessageFail	document.write	
+Result	Test Name	MessagePass	document.write	
--- a/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/017.txt
+++ b/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/017.txt
@ -6,6 +6,6 @@ Rerun
 Found 1 tests
-1 Fail
+1 Pass
 Details
-Result	Test Name	MessageFail	document.write	
+Result	Test Name	MessagePass	document.write	
--- a/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/033.txt
+++ b/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/033.txt
@ -6,6 +6,6 @@ Rerun
 Found 1 tests
-1 Fail
+1 Pass
 Details
-Result	Test Name	MessageFail	document.write	
+Result	Test Name	MessagePass	document.write	
--- a/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/035.txt
+++ b/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/035.txt
@ -6,6 +6,6 @@ Rerun
 Found 1 tests
-1 Fail
+1 Pass
 Details
-Result	Test Name	MessageFail	document.write	
+Result	Test Name	MessagePass	document.write	
--- a/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/038.txt
+++ b/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/038.txt
@ -6,6 +6,6 @@ Rerun
 Found 1 tests
-1 Fail
+1 Pass
 Details
-Result	Test Name	MessageFail	document.write	Cannot access property "childNodes" on null object "document.body"
+Result	Test Name	MessagePass	document.write	
--- a/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/039.txt
+++ b/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/039.txt
@ -6,6 +6,6 @@ Rerun
 Found 1 tests
-1 Fail
+1 Pass
 Details
-Result	Test Name	MessageFail	document.write	
+Result	Test Name	MessagePass	document.write	
--- a/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/046.txt
+++ b/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/046.txt
@ -6,6 +6,6 @@ Rerun
 Found 1 tests
-1 Fail
+1 Pass
 Details
-Result	Test Name	MessageFail	document.write	
+Result	Test Name	MessagePass	document.write	
--- a/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/049.txt
+++ b/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/049.txt
@ -6,6 +6,6 @@ Rerun
 Found 1 tests
-1 Fail
+1 Pass
 Details
-Result	Test Name	MessageFail	document.write plaintext	
+Result	Test Name	MessagePass	document.write plaintext	
--- a/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/iframe_002.txt
+++ b/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/iframe_002.txt
@ -6,6 +6,6 @@ Rerun
 Found 1 tests
-1 Fail
+1 Pass
 Details
-Result	Test Name	MessageFail	document.write into iframe	
+Result	Test Name	MessagePass	document.write into iframe	
--- a/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/iframe_008.txt
+++ b/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/iframe_008.txt
@ -6,6 +6,6 @@ Rerun
 Found 1 tests
-1 Fail
+1 Pass
 Details
-Result	Test Name	MessageFail	document.write plaintext into iframe	
+Result	Test Name	MessagePass	document.write plaintext into iframe	
--- a/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/iframe_009.txt
+++ b/Tests/LibWeb/Text/expected/wpt-import/html/webappapis/dynamic-markup-insertion/document-write/iframe_009.txt
@ -6,6 +6,6 @@ Rerun
 Found 1 tests
-1 Fail
+1 Pass
 Details
-Result	Test Name	MessageFail	document.write plaintext into iframe	
+Result	Test Name	MessagePass	document.write plaintext into iframe