There's a bit of a UTF-8 assumption with this change. But nearly every
caller of these methods were immediately creating a String from the
resulting ByteString anyways.
We were not properly handling the case that prefix code point was the
empty string (which we represent as an OptionalNone). While this
still resulted in the correct pattern string being generated, an
incorrect regular expression was being generated causing matching
to fail.
This has no functional difference as far as I can tell, but for
clarity explicitly do not attempt to do this, which has the nice
side effect of not checking for whitespace known to not exist.
It turns out that the problem here was simply that we were trimming
trailing whitespace when we did not need to, which was meaning that
the port number of '80 ' was being converted to the empty string
per URLPattern elision as the port matches the http scheme.
Corresponds to: 46c30fda8f
Along with a follow up bug fix that I made of:
5e1c93e2
This for example, fixes canonicalization of URL hosts containing
special characters that should have the unicode ToAscii algorithm
performed on them as the URLs were not being treated as special.
Corresponds to URL spec change:
cc8b776b
Note that the new test failure being introduced here is an unrelated
WPT test change bundled in the resources test file update that I am
not convinced is correct.
The URL spec represents its path as a:
Variant<String, Vector<String>>
A URL is defined has having an opaque path if it has a single String,
the URL path otherwise containing a list of path components.
We (like in an older version of the spec) track this through a boolean
and only use a Vector with a single component for opaque paths.
This means it was incorrect to simple assign the path to a list with
a single empty string without setting that URL as opaque, which
meant that the path serialization was producing incorrect results.
It may make sense changing the API so this situation is a little more
clear. But for now, we simply need to set the opaque path boolean
to true here.
This provides the infrastructure for taking a part list from the
pattern parser and generating the actual regexp object which is
used for matching against URLs from the pattern.
Compiling a URLPattern component will generate a 'parts list' which
is used for generating the regular expression that is used for
matching against URLs.
This parts list is also used to generate (through this function) a
pattern string. The pattern string of a URL component is what is
exposed on the USVString getters of the URLPattern class itself.
As an example, the following:
```
let pattern = new URLPattern({ "pathname": "/foo/(.*)*" });
console.log(pattern.pathname);
```
Will log the pattern string of: '/foo/**'.
These control how a pattern string is generated, which can vary for
different components and is also impacted by the 'ignoreCase' option
that can be provided in the URLPattern constructor.
This is to save a future name conflict that will appear between
the options IDL dictionary and the options struct that are both
present in the spec.
It is also a nicer interface for now given there is only a single
option at the moment.
We were using the public suffix of the URL's host as its registrable
domain. But the registrable domain is actually the public suffix plus
one additional label.
As the comment in this file explains the caller of LibURL APIs are
meant to assume if they see any error, that it is a TypeError since
that is all the spec throws at the moment.
A custom error type exists here so that we can include more
information in TypeError's which are thrown.
All URLs are now either constucted through the URL Parser or by
default constructing a URL, and setting each of the fields of that
URL manually. This makes it much more difficult to create invalid
URLs.
This is the core object behind a URL pattern which when constructed
can be used for matching the pattern against URLs.
However, the implementation here is missing key functions such as
the constructor and the 'test'/'exec' functions as that relies on
a significant amount of supporting URLPattern infrastructure such
as two different parsers and a tokenizer.
However, this is enough for us to implement some more of the IDL
wrapper layer of this specification.
A URL pattern consists of components such as the 'port', 'password'
'hostname', etc. A component is compiled from the input to the
URLPattern constructor and is what is used for matching against
URLs to produce a match result.
This is also where the regex dependency is introduced into LibURL
to support the URLPattern implementation.
Currently we create URLs such as 'about:blank' through the StringView
or ByteString constructor of URL. However, in order to elimate the
use of URL::is_valid, we need to get rid of these constructors as it
makes it way too easy to create an invalid URL.
It is very cumbersome to construct an 'about:blank' URL when using
URL::Parser::basic_parse. So instead of doing that, create some
helper functions which will create the 'about:XXX' URLs with the
correct properties set.
Conveniently, this is also a much faster way of creating these URLs
as it means we do not need to parse the URL and can set all of the
members up front.
This is the return value of a URLPattern after `exec` is called on it.
It conveys information about the named (or unammed) regex groups
matched for each component of the URL. For example,
```
let p = new URLPattern({ hostname: "{:subdomain.}*example.com" });
const result = pattern.exec({ hostname: "foo.bar.example.com" });
console.log(result.hostname.groups.subdomain);
```
Will log 'foo.bar'.
The URLPattern spec is intended to be implemented inside of LibURL, with
LibWeb only responsible for the IDL conversion layer, in a similar
manner to how URL is implemented.
Instead of just putting in members directly, wrap them up in structs
which represent what a URL blob entry is meant to hold per the spec.
This makes more obvious what this is meant to represent, such as the
ByteBuffer being used to represent the bytes behind a Blob.
This also allows us to use a stronger type for a function that needs
to return a Blob URL entry's object.
Specifically, after implementing some recent spec changes to navigables,
we end up calling `get_public_suffix("localhost")` here, which returns
OptionalNone. This would previously crash.
Our get_public_suffix() seems a little incorrect. From the spec:
> If no rules match, the prevailing rule is "*".
> https://github.com/publicsuffix/list/wiki/Format#algorithm
However, ours returns an empty Optional in that case. To avoid breaking
other users of it, this patch modifies Host's uses of it, rather than
the function itself.
URL::basic_parse has a subtle bug where the resulting URL is not set
to valid when StateOveride is provided and the URL parser early returns
a valid URL.
This has not surfaced as a problem so far, as the only users of the
state override API provide an already valid URL buffer and also ignore
the result of basic parsing with a state override.
However, this bug surfaces implementing the URL pattern spec, which as
part of URL canonicalization:
* Provides a dummy URL record
* Basic URL parses that URL with state override
* Checks the result of the URL parser to validate the URL
While we could set URL validity on every early return of the URL parser
during state override, it has been a long standing FIXME around the code
to try and remove the awkward validity state of the URL class. So this
commit makes the first stage of this change by migrating the basic
parser API to return Optional, which also happens to make this subtle
issue not a problem any more.
