Commit graph

118 commits

Author SHA1 Message Date
Shannon Booth
501f92b54e LibWeb+LibURL: Consolidate Origin parsing and serialization into LibURL
Because of the previous awkward factoring of Origin we had two
implementations of Origin serializing and creation. Move the
implementation of DOMURL::url_origin into URL::origin, and
instead use the implemenation of URL::Origin::serialize for
serialization (replacing URL::serialize_origin).

This happens to fix 8 URL subtests as the two implemenations had
diverged, and URL::serialize_origin was previously missing the spec
changes of: whatwg/url@eee49fd and whatwg/url@fff33c3
2024-10-05 10:46:30 +02:00
Shannon Booth
dc401f49ea LibWeb+LibURL: Move HTML::Origin to URL::Origin
While Origin is defined in the HTML spec - this leaves us with quite an
awkward relationship as the URL spec makes use of AO's from what is
defined in the HTML spec.

To simplify this factoring, relocate Origin into LibURL.
2024-10-05 10:46:30 +02:00
sideshowbarker
23da1752b5 LibWeb/Fetch: Set HTTP status code on cached responses
This change causes HTTP status codes to be set on cached HTTP responses.

Otherwise, without this change, no status codes at all are set on cached
HTTP responses — which causes all cached responses to default to being
loaded/served with a 200 status code. And as a result of that, if the
cached response is from a 30x redirect, then without this change, when
that cached 30x response is loaded, we don’t follow the redirect —
because we see a 200 status, rather than the expected/original 30x.

Fixes https://github.com/LadybirdBrowser/ladybird/issues/863

Note that this change also reverts the temporary workaround added in
f735c464d3
(https://github.com/LadybirdBrowser/ladybird/pull/899).
2024-08-14 14:49:04 +02:00
Jelle Raaijmakers
e7984a7711 LibWeb: Check presence of WWW-Authenticate header in fetch response
If a HTTP 401 response we get does not contain a `WWW-Authenticate`
header, we should not trigger the logic to ask the user for credentials
and retry the request.

This part is hinted at in a TODO / 'Needs testing' remark in the spec
but needs to be fleshed out. Raised an upstream issue to do so:

  https://github.com/whatwg/fetch/issues/1766

This fixes login forms triggering an infinite fetch loop when providing
incorrect credentials.

Co-Authored-By: Victor Tran <vicr12345@gmail.com>
2024-08-13 16:01:48 +01:00
Jamie Mansfield
35047de1d8 LibWeb/Fetch: Add a constant for the keepalive maximum size 2024-08-11 20:07:10 +01:00
Shannon Booth
84a7fead0e LibURL: Make percent_encode return a String
This simplifies a bunch of places which were needing to error check and
convert from a ByteString to String.
2024-08-10 10:39:43 +02:00
Shannon Booth
ffe070d7f9 LibWeb+LibURL: Use URL paths directly for comparison
This matches the text of the spec a little more closely in many cases
and is also more efficient than serializing the URL path.
2024-08-05 09:58:13 +02:00
Shannon Booth
f511c0b441 LibURL+LibWeb: Do not percent decode in password/username getters
Doing it is not part of the spec. Whenever needed, the spec will
explicitly percent decode the username and password.

This fixes some URL WPT tests.
2024-08-04 12:59:02 +01:00
sideshowbarker
f735c464d3 LibWeb/Fetch: Don’t cache redirects
This change disables caching for 301, 302, 303, 307, and 308 responses.
This is just for now, ad-hoc — not adhering to any particular spec.
Fixes https://github.com/LadybirdBrowser/ladybird/issues/863
2024-08-01 11:39:27 +02:00
Kenneth Myhra
a744a9ebe7 LibWeb: Append fetch record to client's fetch group when request is a
...subresource.
2024-07-26 10:22:17 +02:00
Kemal Zebari
c5f1e47883 LibWeb: Implement Blob::bytes()
Implements https://w3c.github.io/FileAPI/#dom-blob-bytes.
2024-07-26 10:21:32 +02:00
Jamie Mansfield
2ca8fd1832 LibWeb: Make preferred languages configurable
This also changes fetch to use the preferred languages for the
Accept-Language header.
2024-07-25 11:38:59 +01:00
mobounya
9c93630d02 LibWeb: Conform HTTP-network-or-cache to specs
Implement step 8.10 in '4.5. HTTP-network-or-cache fetch' from the
fetch specs.
2024-07-20 14:09:51 -06:00
Jamie Mansfield
9ce727d315 LibWeb/Fetch: Use correct header for prefetch requests
See:
 - a5560d2
2024-07-14 12:08:20 +02:00
Jamie Mansfield
aee77b975c LibWeb/Fetch: Add missing fetch step for preloaded resources
This is currently no-op and a FIXME exists to implement the "consume a
preloaded resource" AO.
2024-07-14 12:08:20 +02:00
circl
4e6eb35520 LibWeb/Fetch: Pass error from ResourceLoader into network_error 2024-07-05 15:08:13 -06:00
Jamie Mansfield
fb20326979 LibWeb: Support sending DNT header with requests 2024-07-04 16:42:34 +02:00
circl
1f3285eb04 LibWeb: Restrict fetching file: and resource: URLs to internal pages
They are now blocked on pages which:
- Don't have an opaque origin (should be only user-initiated or about:)
- Aren't other file: pages
- Aren't other resource: pages
2024-06-26 12:15:33 -06:00
Andreas Kling
a3c8e60710 LibWeb: Implement very basic in-memory HTTP caching
This patch adds a simple in-memory HTTP cache to each WebContent
process.

It's currently off by default (turn it on with --enable-http-cache)
since the validation logic is lacking and incomplete.
2024-06-23 09:37:23 +02:00
Kenneth Myhra
c51d01bea7 Fetch: Make sure we iterate over HeaderMap's headers()
This fixes a build failure when built with CMake option
'-DENABLE_ALL_THE_DEBUG_MACROS=ON'.
2024-06-09 21:13:25 -04:00
Andreas Kling
e636851481 LibHTTP+RequestServer: Add HTTP::HeaderMap and use for response headers
Instead of using a HashMap<ByteString, ByteString, CaseInsensitive...>
everywhere, we now encapsulate this in a class.

Even better, the new class also allows keeping track of multiple headers
with the same name! This will make it possible for HTTP responses to
actually retain all their headers on the perilous journey from
RequestServer to LibWeb.
2024-06-09 15:34:02 +02:00
Jamie Mansfield
ab6b687d4c LibWeb: Implement "Set request’s referrer policy on redirect" AO 2024-06-07 09:54:19 +02:00
Jamie Mansfield
8542a8b458 LibWeb/Fetch: Update "HTTP-redirect fetch" algorithm to latest spec
The spec and implementation's comments had diverged a little, this
brings them in line :)
2024-06-07 09:54:19 +02:00
Jamie Mansfield
7ce35b75aa LibWeb: Implement should block mixed content response to request 2024-06-07 09:50:30 +02:00
Jamie Mansfield
2159377296 LibWeb: Upgrade mixed requests to potentially trustworthy URLs
(if appropriate)
2024-06-07 09:50:30 +02:00
Jamie Mansfield
8f0d035145 LibWeb: Implement should block mixed content request 2024-06-07 09:50:30 +02:00
Jamie Mansfield
8e062a52a3 LibWeb: Move NetworkPartitionKey into a header
This is used elsewhere, such as the HTMLLinkElement.
2024-06-05 05:42:47 +01:00
Jamie Mansfield
a419dcbe8b LibWeb/Fetch: Update "HTTP fetch" algorithm to latest spec comments
The spec and implementation's comments had diverged a little, this
brings them in line :)
2024-06-04 07:27:52 +02:00
Jamie Mansfield
295c4ef51a LibWeb/Fetch: Use MimeType in DataURL 2024-06-02 19:55:53 +02:00
Timothy Flynn
6056428cb5 LibWeb: Support unbuffered fetch requests
Supporting unbuffered fetches is actually part of the fetch spec in its
HTTP-network-fetch algorithm. We had previously implemented this method
in a very ad-hoc manner as a simple wrapper around ResourceLoader. This
is still the case, but we now implement a good amount of these steps
according to spec, using ResourceLoader's unbuffered API. The response
data is forwarded through to the fetch response using streams.

This will eventually let us remove the use of ResourceLoader's buffered
API, as all responses should just be streamed this way. The streams spec
then supplies ways to wait for completion, thus allowing fully buffered
responses. However, we have more work to do to make the other parts of
our fetch implementation (namely, Body::fully_read) use streams before
we can do this.
2024-05-26 18:29:24 +02:00
Lucas CHOLLET
b30c361b08 LibWeb/Fetch: Implement logic to process a response from HTTP's cache
Co-Authored-By: Andrew Kaster <akaster@serenityos.org>
2024-05-23 13:25:29 -04:00
Andrew Kaster
c05f296014 LibWeb: Start adding infrastructure for an HTTP Cache 2024-05-23 13:25:29 -04:00
Andrew Kaster
a40b331f39 LibWeb: Update steps of HTTP-network-or-cache-fetch per the latest spec
One step was added since we last visited this AO.
2024-05-23 13:25:29 -04:00
Kenneth Myhra
f119ac1a9d LibWeb: Integrate TransformStream into fetch_response_handover() 2024-05-20 16:57:52 -04:00
Timothy Flynn
b5ba60f1d1 LibWeb: Change Fetch's ProcessBodyError to accept a plain JS value
This callback is meant to be triggered by streams, which does not always
provide a WebIDL::DOMException. Pass a plain value instead. Of all the
users of this callback, only one actually uses the value, and already
converts the DOMException to a plain value.
2024-05-20 16:57:52 -04:00
Jamie Mansfield
3438293e7b LibWeb/Fetch: Share a conditional in fetch response handover
See:
- aaada1f
2024-05-19 16:25:50 +02:00
Jamie Mansfield
951fbb1837 LibWeb/Fetch: Expose a minimised Content-Type to Resource Timing
See:
- 931cd06
2024-05-19 16:25:50 +02:00
Shannon Booth
53eb9af42f LibWeb: Use URL's 'blob URL entry' for blob fetches
Performing a lookup in the blob URL registry does not work in the case
of a web worker - as the registry is not shared between processes.
However - the URL itself passed to a worker has the blob attached to it,
which we can pull out of the URL on a fetch.
2024-05-12 15:46:29 -06:00
Jamie Mansfield
987198782c LibWeb/Fetch: Use "json" destination
See:
- da8d0d8
- 49bff76
- 37659e9
2024-05-07 17:27:37 -06:00
Jamie Mansfield
f5799f7d2c LibWeb/Fetch: Append the Fetch metadata headers 2024-05-01 12:57:35 +02:00
Jamie Mansfield
e52f444329 LibWeb/Fetch: Implement the "set the Sec-Fetch-User header" AO 2024-05-01 12:57:35 +02:00
Jamie Mansfield
1ff90aa3e0 LibWeb/Fetch: Implement the "set the Sec-Fetch-Site header" AO 2024-05-01 12:57:35 +02:00
Jamie Mansfield
5eb46a5f01 LibWeb/Fetch: Implement the "set the Sec-Fetch-Mode header" AO 2024-05-01 12:57:35 +02:00
Jamie Mansfield
f4af1833c1 LibWeb/Fetch: Implement the "set the Sec-Fetch-Dest header" AO 2024-05-01 12:57:35 +02:00
Timothy Flynn
1ffda6a805 LibWeb: Propagate OOM in Body::fully_read() through its error callback
Fetched bodies can be on the order of gigabytes, so rather than crashing
when we hit OOM here, we can simply invoke the error callback with a DOM
exception. We use "UnknownError" here as the spec directly supports this
for OOM errors:

    UnknownError: The operation failed for an unknown transient reason
                  (e.g. out of memory).

This is still an ad-hoc implementation. We should be using streams, and
we do have the AOs available to do so. But they need to be massaged to
be compatible with callers of Body::fully_read. And once we do use
streams, this function will become infallible - so making it infallible
here is at least a step in the right direction.
2024-04-27 07:08:14 +02:00
Timothy Flynn
5f51a11618 LibWeb: Remove OOM propagation from Fetch::Infrastructure::Responses 2024-04-27 07:08:14 +02:00
Timothy Flynn
5a4f13dcd4 LibWeb: Remove OOM propagation from Fetch::Infrastructure::Requests 2024-04-27 07:08:14 +02:00
Timothy Flynn
c79f46fe6f LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers 2024-04-27 07:08:14 +02:00
Andreas Kling
184368285c LibWeb: Fix GC leaks in Fetch::Infrastructure::Body::fully_read()
By making this function accept the success and error steps as
HeapFunction rather than SafeFunction, we break a bunch of strong
GC cycles.
2024-04-23 12:50:40 +02:00
Kenneth Myhra
291d0e5de8 LibWeb: Let queue_fetch_task() take a JS::HeapFunction
Changes the signature of queue_fetch_task() from AK:Function to
JS::HeapFunction to be more clear to the user of the function that this
is what it uses internally.
2024-04-20 18:11:01 +02:00