Memory stream is a more suitable container for the socket send queue,
as using it results in fewer allocations than trying to emulate a stream
using a Vector.
Bring back d6080d1fdc with a missing check
whether underlying socket is closed, before accessing `fd()` that is
optional and empty in case of closed socket.
With this change TransportSocket becomes capable of sending large
messages without relying on workarounds, such as sending the message as
a shared memory file descriptor when it can't fully fit into the socket
buffer.
It's implemented by combining all enqueued messages into two buffers:
one for bytes and another for fds, and repeatedly attempts to write them
in smaller chunks, waiting for the socket to become writable again if
the receiver needs time to consume the data.
Another significant improvement brought by this change is that we no
longer drop messages queued for sending if the socket doesn't become
writable after a 100ms timeout. Instead, we return the message to the
send buffer and continue waiting for the socket to become writable.
Instead of wrapping all non-movable members of TransportSocket in OwnPtr
to keep it movable, make TransportSocket itself non-movable and wrap it
in OwnPtr.
By doing this we also make MessagePort, that relies on IPC transport,
to send messages from separate thread, which solves the problem when
WebWorker and WebContent could deadlock if both were trying to post
messages at the same time.
Fixes https://github.com/LadybirdBrowser/ladybird/issues/4254
Reimplements c3121c9d at the transport layer, allowing us to solve the
same problem once, in a single place, for both the LibIPC connection and
MessagePort. This avoids exposing a workaround for a macOS specific Unix
domain socket issue to higher abstraction layers.
With this change, the responsibility for prepending messages with their
size and ensuring the entire message is received before returning it to
the caller is moved to TransportSocket. This removes the need to
duplicate this logic in both LibIPC and MessagePort.
Another advantage of reducing message granularity at IPC::Transport
layer is that it will make it easier to support alternative transport
implementations (like Mach ports, which unlike Unix domain sockets are
not stream oriented).
This change ensures that instead of immediately deallocating the message
buffer after sending, we retain it in an acknowledgement wait queue
until an acknowledgement is received from the peer. This is necessary
to handle a behavior of the macOS kernel, which may prematurely
garbage-collect file descriptors contained within the message buffer
before the peer receives them.
The acknowledgement mechanism assumes messages are received in the same
order they were sent so, each acknowledgement message simply indicates
the count of successfully received messages, specifying how many entries
can safely be removed from the acknowledgement wait queue.
This commit removes the -Wno-unusued-private-field flag, thus
reenabling the warning. Unused field were either removed or marked
[[maybe_unused]] when unsure.
It turned out that some web applications want to send fairly large
messages to WebWorker through IPC (for example, MapLibre GL sends
~1200KiB), which led to failures (at least on macOS) because buffer size
of TransportSocket is limited to 128KiB. This change solves the problem
by wrapping messages that exceed socket buffer size into another message
that holds wrapped message content in shared memory.
Co-Authored-By: Luke Wilde <luke@ladybird.org>
This has been a longstanding ergonomic issue with our IPC compiler. Non-
trivial types were previously passed by const&. So if we wanted to avoid
expensive copies, we would have to const_cast and move the data.
We now pass ownership of all transferred data to the client subclasses.
This allows us to remove const_cast from these methods, and allows us to
avoid some trivial expensive copies that we didn't bother to const_cast.
For example, consider the following IPC message:
do_something(u64 page_id, String string, Vector<Data> data) =|
We would previously generate the following C++ method to encode/transfer
this message:
void do_something(u64 page_id, String string, Vector<Data> data);
This required the caller to either have to copy the non-trivial types or
`move` them in. In some places, this meant we had to construct temporary
vectors just to send an IPC.
This isn't necessary because we weren't holding onto these parameters
anyways. We would construct an IPC::Message subclass with them (which
does require owning types), but then immediate encode the message to
an IPC::MessageBuffer and send it.
We now generate code such that we don't need to construct a Message. We
can simply encode the parameters directly without needing ownership.
This allows us to take view-types to IPC parameters.
So the above example now becomes:
void do_something(u64, StringView, ReadonlySpan<Data>);
This removes the use of StringBuilder::OutputType (which was ByteString,
and only used by the JSON classes). And it removes the StringBuilder
template parameter from the serialization methods; this was only ever
used with StringBuilder, so a template is pretty overkill here.
Invalid URLs should be signified by a wrapper class, such as an
Optional<URL::URL> in the IPC file. I do not believe that we have
anything which currently relies on passing through an invalid URL.
At some point, we stopped ever constructing invalid messages. This makes
that clearer, and will allow us to stop requiring that IPC arguments be
default-constructible.
The Linux IPC uses SCM_RIGHTS to transfer fds to another process
(see TransportSocket::transfer, which calls LocalSocket::send_message).
File descriptors are handled separately from regular data.
On Windows handles are embedded in regular data. They are duplicated
in the sender process.
Socket handles need special code both on sender side (because they
require using WSADuplicateSocket instead of DuplicateHandle, see
TransportSocketWindows::duplicate_handles) and on receiver side
(because they require WSASocket, see FileWindows.cpp).
TransportSocketWindows::ReadResult::fds vector is always empty, it is
kept the same as Linux version to avoid OS #ifdefs in Connection.h/.cpp
and Web::HTML::MessagePort::read_from_transport. Separate handling of
fds permeates all IPC code, it doesn't make sense to #ifdef out all this
code on Windows. In other words, the Linux code is more generic -
it handles both regular data and fds. On Windows, we need only the
regular data portion of it, and we just use that.
Duplicating handles on Windows requires pid of target (receiver)
process (see TransportSocketWindows::m_peer_pid). This pid is received
during special TransportSocketWindows initialization, which is performed
only on Windows. It is handled in a separate PR #3179.
Note: ChatGPT and [stackoverflow](https://stackoverflow.com/questions/25429887/getting-pid-of-peer-socket-on-windows) suggest using GetExtendedTcpTable/GetTcpTable2
to get peer pid, but this doesn't work because [MIB_TCPROW2::dwOwningPid](https://learn.microsoft.com/en-us/windows/win32/api/tcpmib/ns-tcpmib-mib_tcprow2)
is "The PID of the process that issued a context bind for this TCP
connection.", so for both ends it will return the pid of the process
that called socketpair.
Co-Authored-By: Andrew Kaster <andrew@ladybird.org>
Also:
* Use SpecializationOf for Optional and Variant concepts
* Remove inline because it is implied by constexpr
* Reorder declarations from less to more specialized (in common sense)
Instead of just putting in members directly, wrap them up in structs
which represent what a URL blob entry is meant to hold per the spec.
This makes more obvious what this is meant to represent, such as the
ByteBuffer being used to represent the bytes behind a Blob.
This also allows us to use a stronger type for a function that needs
to return a Blob URL entry's object.
This lets us move a few Host-related functions (like serialization and
checks for what the Host is) into Host instead of having them dotted
around the codebase.
For now, the interface is still very Variant-like, to avoid having to
change quite so much in one go.
Problem:
- Many constructors are defined as `{}` rather than using the ` =
default` compiler-provided constructor.
- Some types provide an implicit conversion operator from `nullptr_t`
instead of requiring the caller to default construct. This violates
the C++ Core Guidelines suggestion to declare single-argument
constructors explicit
(https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#c46-by-default-declare-single-argument-constructors-explicit).
Solution:
- Change default constructors to use the compiler-provided default
constructor.
- Remove implicit conversion operators from `nullptr_t` and change
usage to enforce type consistency without conversion.
Problem:
- `(void)` simply casts the expression to void. This is understood to
indicate that it is ignored, but this is really a compiler trick to
get the compiler to not generate a warning.
Solution:
- Use the `[[maybe_unused]]` attribute to indicate the value is unused.
Note:
- Functions taking a `(void)` argument list have also been changed to
`()` because this is not needed and shows up in the same grep
command.
It is now possible to use the special IPC::File type in message arguments. In
C++, the type is nothing more than a wrapper over a file descriptor. But when
serializing/deserializing IPC::File arguments, LibIPC will use the sendfd/recvfd
kernel APIs instead of sending the integer inline.
This makes it quite convenient to pass files over IPC, and will allow us to
significantly tighten sandboxes in the future :^)
Closes https://github.com/SerenityOS/serenity/issues/3643
Since we're using byte streamed Unix sockets for the IPC protocols,
it's possible for the kernel to run out of socket buffer space with
a partial message near the end of the buffer.
Handle this situation in IPC::Connection by buffering the bytes of
what may be a partial message, and prepending them to the incoming
data next time we receive from the peer.
This fixes WindowServer asserting when a peer is spamming it hard.
This would previously fail at runtime, and it would have zero indication
of what exactly went wrong.
Also adds `AK::DependentFalse<Ts...>', which is a...dependent false.
Writing to the socket may trigger a close of the socket descriptor
if a disconnect was detected. We need to check if it is still valid
when waiting for a response after attempting to send a synchronous
message.
Fixes#3515
Because we're closing a file descriptor, we need to disable any
Notifier that is using it so that the EventLoop does not use invalid
file descriptors.
Fixes#3508
