- dev - passt

[PATCH 0/7] Draft, incomplete series introducing state migration
by Stefano Brivio 03 Feb '25

03 Feb '25

This is obviously incomplete. I have code on top of this, not really working yet, with a loop on transferred flows, and an implementation matching passt-repair, requesting to enable/disable the TCP_REPAIR option as needed, as well as setting/receiving sequences. I'm sending this for early review/rework/rewrite/whatever. What's here should all be tested and working. Adding: { &flow_first_free, sizeof(flow_first_free) }, { flowtab, sizeof(flowtab) }, to data version 1 in 6/7 will properly transfer those sections. Declaring functions and assigning pointers such as: { flow_migrate_source_pre, NULL }, { flow_migrate_source_post, NULL }, { flow_migrate_target_post_v1, NULL }, also executes them. The passt-repair helper in 7/7 is (lightly) tested against a stand-alone source/target implementation which I'll share in a bit. Stefano Brivio (7): icmp, udp: Pad time_t timestamp to 64-bit to ease state migration flow, flow_table: Pad flow table entries to 128 bytes, hash entries to 32 bits tcp_conn: Avoid 7-bit hole in struct tcp_splice_conn flow_table: Use size in extern declaration for flowtab util: Add read_remainder() and read_all_buf() Introduce facilities for guest migration on top of vhost-user infrastructure Introduce passt-repair Makefile | 22 +++-- flow.h | 18 ++-- flow_table.h | 15 ++- icmp_flow.h | 6 +- migrate.c | 259 +++++++++++++++++++++++++++++++++++++++++++++++++ migrate.h | 90 +++++++++++++++++ passt-repair.c | 111 +++++++++++++++++++++ passt.c | 2 +- tcp_conn.h | 2 +- udp_flow.h | 6 +- util.c | 70 +++++++++++++ util.h | 2 + vu_common.c | 122 +++++++++++++++-------- vu_common.h | 2 +- 14 files changed, 662 insertions(+), 65 deletions(-) create mode 100644 migrate.c create mode 100644 migrate.h create mode 100644 passt-repair.c -- 2.43.0

2 52

[PATCH v3 00/20] Draft, incomplete series introducing state migration
by Stefano Brivio 03 Feb '25

03 Feb '25

...and finally connections survive migration from source to target, at least the ones originating from the (source) guest. I didn't try the other way around, small tweaks might be needed. Tested as follows, roughly as instructed by Laurent: Source: $ ./passt --vhost-user $ qemu-system-x86_64 -machine accel=kvm -cpu host -kernel ... \ -initrd mbuto.img -nographic -serial mon:stdio -nodefaults \ -append "console=ttyS0" \ -chardev socket,id=chr0,path=/tmp/passt_1.socket \ -netdev vhost-user,id=netdev0,chardev=chr0 \ -device virtio-net,netdev=netdev0 \ -object memory-backend-memfd,id=memfd0,share=on,size=$((2 * 1024 * 1024 * 1024)) \ -numa node,memdev=memfd0 -m 2G # ./passt-repair /tmp/passt_1.socket.repair Target (same host): $ ./passt --vhost-user $ qemu-system-x86_64 -machine accel=kvm -cpu host -kernel ... \ -initrd mbuto.img -nographic -serial mon:stdio -nodefaults \ -append "console=ttyS0" \ -chardev socket,id=chr0,path=/tmp/passt_2.socket \ -netdev vhost-user,id=netdev0,chardev=chr0 \ -device virtio-net,netdev=netdev0 \ -object memory-backend-memfd,id=memfd0,share=on,size=$((2 * 1024 * 1024 * 1024)) \ -numa node,memdev=memfd0 -m 2G \ -incoming tcp:0:4444 # ./passt-repair /tmp/passt_2.socket.repair Test server: $ nc -l 9091 Once the guest boots: # ip link set dev eth0 up # dhclient eth0 # socat STDIN TCP:$DEFAULT_GW:9091 abcd ^a-c migrate tcp:0:4444 Then continue typing in the target guest: efgh The purpose of this is mostly to show the complete flow, but it needs a number of reworks. What's missing (letting aside pending packet queues for a moment, those are not strictly needed): 1. tests based on the two_guests layout/setup. Even with reverse-search in the shell, this is getting quite hard on wrists. I guess we can start QEMU with -monitor unix:mon.sock,server,nowait and send the 'migrate' command via socat STDIN UNIX-CONNECT:mon.sock 2. dump and transfer of *socket-side* MSS and window scale (I used hardcoded values): this needs more storage, so it needs to be transferred outside the flow table 3. dump, transfer and restore of TCP_REPAIR_WINDOW parameters (not strictly needed, but easy to add once we have appropriate storage) 4. perhaps some small bits of implementation for socket-originated connections (I tested only guest-originated ones so far) 5. UDP and ICMP flows (ping already happens to "survive" nicely, by the way) 6. man page for passt-repair, and man page changes for everything 7. packaging and Linux Security Module changes for passt-repair 8. error handling here and there, and repair rollback/migration abort 9. setting original receive/send buffer sizes and socket options (TCP_NODELAY) What clearly needs changes: a. we can't dump more stuff to the flow table, because we would exceed 128 bytes. We need to copy everything from tcp_tap_conn except for: - state in flow_common - in_epoll - sock - timer and on top of this we need: - values for TCPOPT_WINDOW and TCPOPT_MAXSEG - struct tcp_repair_window somewhat unexpectedly, this is actually bigger than a flow table entry. In any case, we need to implement a stream/per-entry migration right away. b. at this point, I guess we can throw the header away, and just keep a magic (0xB1BB1D1B0BB1D1B0 has a missing 0 at the end but, well, https://en.wikipedia.org/wiki/Bibbidi-Bobbidi-Boo is the Magic Song: can we keep it?) and a version number. The rest, let's go with big/network endianness I'd say, and 64-bit time_t c. the declarative data thing is very convenient but we need to fetch stuff from struct ctx, as shown by the hash_secret example. What's very convenient of this approach is the iovec / writev() / readv() idea. I'm not sure if we can maintain that convenience, though Patches that could be applied regardless of this series to make it more manageable: 1/20 tcp: Always pass NULL event with EPOLL_CTL_DEL 2/20 util: Rename and make global vu_remove_watch() 6/20 util: Add read_remainder() and read_all_buf() 8/20 Introduce passt-repair 16/20 vhost_user: Turn vhost-user message reports to trace() 17/20 vhost_user: Make source quit after reporting migration state 18/20 tcp: Get our socket port using getsockname() when connecting from guest 19/20 tcp: Add HOSTSIDE(x), HOSTFLOW(x) macros Patches that we can throw away with the changes outlined above: 3/20 icmp, udp: Pad time_t timestamp to 64-bit to ease state migration 4/20 flow, flow_table: Pad flow table entries to 128 bytes, hash entries to 32 bits 15/20 flow, flow_table: Export declaration of hash table David Gibson (6): tcp: Always pass NULL event with EPOLL_CTL_DEL util: Rename and make global vu_remove_watch() migrate: vu_migrate_{source,target}() aren't actually vu speciic migrate: Move repair_sock_init() to vu_init() migrate: Make more handling common rather than vhost-user specific migrate: Don't handle the migration channel through epoll Stefano Brivio (14): icmp, udp: Pad time_t timestamp to 64-bit to ease state migration flow, flow_table: Pad flow table entries to 128 bytes, hash entries to 32 bits flow_table: Use size in extern declaration for flowtab util: Add read_remainder() and read_all_buf() Introduce facilities for guest migration on top of vhost-user infrastructure Introduce passt-repair Add interfaces and configuration bits for passt-repair flow, tcp: Basic pre-migration source handler to dump sequence numbers flow, flow_table: Export declaration of hash table vhost_user: Turn vhost-user message reports to trace() vhost_user: Make source quit after reporting migration state tcp: Get our socket port using getsockname() when connecting from guest tcp: Add HOSTSIDE(x), HOSTFLOW(x) macros Implement target side of migration .gitignore | 1 + Makefile | 24 +-- conf.c | 44 +++++- epoll_type.h | 6 +- flow.c | 97 +++++++++++- flow.h | 20 ++- flow_table.h | 22 ++- icmp.c | 2 +- icmp_flow.h | 6 +- migrate.c | 408 +++++++++++++++++++++++++++++++++++++++++++++++++ migrate.h | 84 ++++++++++ passt-repair.c | 117 ++++++++++++++ passt.1 | 11 ++ passt.c | 17 ++- passt.h | 17 +++ repair.c | 193 +++++++++++++++++++++++ repair.h | 16 ++ tap.c | 64 +------- tcp.c | 198 +++++++++++++++++++++++- tcp_conn.h | 7 + tcp_internal.h | 10 +- tcp_splice.c | 4 +- udp_flow.c | 2 +- udp_flow.h | 6 +- util.c | 155 +++++++++++++++++++ util.h | 4 + vhost_user.c | 94 +++--------- virtio.h | 4 - vu_common.c | 62 +++----- vu_common.h | 2 +- 30 files changed, 1469 insertions(+), 228 deletions(-) create mode 100644 migrate.c create mode 100644 migrate.h create mode 100644 passt-repair.c create mode 100644 repair.c create mode 100644 repair.h -- 2.43.0

2 40

[PATCH] tcp: Don't reset outbound connection on SYN retries
by Stefano Brivio 03 Feb '25

03 Feb '25

Reported by somebody on IRC: if the server has considerable latency, it might happen that the client retries sending SYN segments for the same flow while we're still in a TAP_SYN_RCVD, non-ESTABLISHED state. In that case, we should go with the blanket assumption that we need to reset the connection on any unexpected segment: RFC 9293 explicitly mentions this case in Figure 8: Recovery from Old Duplicate SYN, section 3.5. It doesn't make sense for us to set a specific sequence number, socket-side, but we should definitely wait and see. Ignoring the duplicate SYN segment should also be compatible with section 3.10.7.3. SYN-SENT STATE, which mentions updating sequences socket-side (which we can't do anyway), but certainly not reset the connection. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- tcp.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tcp.c b/tcp.c index 7787381..51ad692 100644 --- a/tcp.c +++ b/tcp.c @@ -1920,6 +1920,9 @@ int tcp_tap_handler(const struct ctx *c, uint8_t pif, sa_family_t af, /* Establishing connection from tap */ if (conn->events & TAP_SYN_RCVD) { + if (th->syn && !th->ack && !th->fin) + return 1; /* SYN retry: ignore and keep waiting */ + if (!(conn->events & TAP_SYN_ACK_SENT)) goto reset; -- 2.43.0

2 1

[PATCH] tcp_splice, udp_flow: fcntl64() support on PPC64 depends on glibc version
by Stefano Brivio 03 Feb '25

03 Feb '25

I explicitly added fcntl64() to the list of allowed system calls for PPC64 a while ago, and now it turns out it's not available in recent Debian builds. The warning from seccomp.sh is harmless because we unconditionally try to enable fcntl() anyway, but take care of it anyway. Link: https://buildd.debian.org/status/fetch.php?pkg=passt&arch=ppc64&ver=0.0%7Eg… Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- tcp_splice.c | 2 +- udp_flow.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tcp_splice.c b/tcp_splice.c index 5db1d62..f048a82 100644 --- a/tcp_splice.c +++ b/tcp_splice.c @@ -28,7 +28,7 @@ * - FIN_SENT_0: FIN (write shutdown) sent to accepted socket * - FIN_SENT_1: FIN (write shutdown) sent to target socket * - * #syscalls:pasta pipe2|pipe fcntl arm:fcntl64 ppc64:fcntl64 i686:fcntl64 + * #syscalls:pasta pipe2|pipe fcntl arm:fcntl64 ppc64:fcntl64|fcntl i686:fcntl64 */ #include <sched.h> diff --git a/udp_flow.c b/udp_flow.c index 7fae81d..83c2568 100644 --- a/udp_flow.c +++ b/udp_flow.c @@ -174,7 +174,7 @@ cancel: * @s_in: Source socket address, filled in by recvmmsg() * @now: Timestamp * - * #syscalls fcntl arm:fcntl64 ppc64:fcntl64 i686:fcntl64 + * #syscalls fcntl arm:fcntl64 ppc64:fcntl64|fcntl i686:fcntl64 * * Return: sidx for the destination side of the flow for this packet, or * FLOW_SIDX_NONE if we couldn't find or create a flow. -- 2.43.0

2 1

[PATCH] vhost_user: On 32-bit ARM, mmap() is not available, mmap2() is used instead
by Stefano Brivio 03 Feb '25

03 Feb '25

Link: https://buildd.debian.org/status/fetch.php?pkg=passt&arch=armel&ver=0.0%7Eg… Link: https://buildd.debian.org/status/fetch.php?pkg=passt&arch=armhf&ver=0.0%7Eg… Fixes: 31117b27c6c9 ("vhost-user: introduce vhost-user API") Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- vhost_user.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/vhost_user.c b/vhost_user.c index 19ede8a..f9a9eb3 100644 --- a/vhost_user.c +++ b/vhost_user.c @@ -419,7 +419,7 @@ static bool map_ring(struct vu_dev *vdev, struct vu_virtq *vq) * * Return: False as no reply is requested * - * #syscalls:vu mmap munmap + * #syscalls:vu mmap|mmap2 munmap */ static bool vu_set_mem_table_exec(struct vu_dev *vdev, struct vhost_user_msg *msg) -- 2.43.0

2 1

[PATCH 0/2] Trivial cleanups to EPOLL_CTL_DEL handling
by David Gibson 03 Feb '25

03 Feb '25

2 3

[PATCH 0/2] Fancier version handling for migration
by David Gibson 31 Jan '25

31 Jan '25

This series introduces more flexible version handling for our device state migration stream. This is based on my previous assorted migration improvements series. David Gibson (2): migrate: Merge protocol version information into one table migrate: More sophisticated versioning migrate.c | 213 +++++++++++++++++++++++++++++++++++------------------- migrate.h | 73 +++++++++++-------- 2 files changed, 179 insertions(+), 107 deletions(-) -- 2.48.1

2 6

[net,v3] tcp: correct handling of extreme memory squeeze
by jmaloy＠redhat.com 30 Jan '25

30 Jan '25

From: Jon Maloy <jmaloy(a)redhat.com> Testing with iperf3 using the "pasta" protocol splicer has revealed a bug in the way tcp handles window advertising in extreme memory squeeze situations. Under memory pressure, a socket endpoint may temporarily advertise a zero-sized window, but this is not stored as part of the socket data. The reasoning behind this is that it is considered a temporary setting which shouldn't influence any further calculations. However, if we happen to stall at an unfortunate value of the current window size, the algorithm selecting a new value will consistently fail to advertise a non-zero window once we have freed up enough memory. This means that this side's notion of the current window size is different from the one last advertised to the peer, causing the latter to not send any data to resolve the sitution. The problem occurs on the iperf3 server side, and the socket in question is a completely regular socket with the default settings for the fedora40 kernel. We do not use SO_PEEK or SO_RCVBUF on the socket. The following excerpt of a logging session, with own comments added, shows more in detail what is happening: // tcp_v4_rcv(->) // tcp_rcv_established(->) [5201<->39222]: ==== Activating log @ net/ipv4/tcp_input.c/tcp_data_queue()/5257 ==== [5201<->39222]: tcp_data_queue(->) [5201<->39222]: DROPPING skb [265600160..265665640], reason: SKB_DROP_REASON_PROTO_MEM [rcv_nxt 265600160, rcv_wnd 262144, snt_ack 265469200, win_now 131184] [copied_seq 259909392->260034360 (124968), unread 5565800, qlen 85, ofoq 0] [OFO queue: gap: 65480, len: 0] [5201<->39222]: tcp_data_queue(<-) [5201<->39222]: __tcp_transmit_skb(->) [tp->rcv_wup: 265469200, tp->rcv_wnd: 262144, tp->rcv_nxt 265600160] [5201<->39222]: tcp_select_window(->) [5201<->39222]: (inet_csk(sk)->icsk_ack.pending & ICSK_ACK_NOMEM) ? --> TRUE [tp->rcv_wup: 265469200, tp->rcv_wnd: 262144, tp->rcv_nxt 265600160] returning 0 [5201<->39222]: tcp_select_window(<-) [5201<->39222]: ADVERTISING WIN 0, ACK_SEQ: 265600160 [5201<->39222]: [__tcp_transmit_skb(<-) [5201<->39222]: tcp_rcv_established(<-) [5201<->39222]: tcp_v4_rcv(<-) // Receive queue is at 85 buffers and we are out of memory. // We drop the incoming buffer, although it is in sequence, and decide // to send an advertisement with a window of zero. // We don't update tp->rcv_wnd and tp->rcv_wup accordingly, which means // we unconditionally shrink the window. [5201<->39222]: tcp_recvmsg_locked(->) [5201<->39222]: __tcp_cleanup_rbuf(->) tp->rcv_wup: 265469200, tp->rcv_wnd: 262144, tp->rcv_nxt 265600160 [5201<->39222]: [new_win = 0, win_now = 131184, 2 * win_now = 262368] [5201<->39222]: [new_win >= (2 * win_now) ? --> time_to_ack = 0] [5201<->39222]: NOT calling tcp_send_ack() [tp->rcv_wup: 265469200, tp->rcv_wnd: 262144, tp->rcv_nxt 265600160] [5201<->39222]: __tcp_cleanup_rbuf(<-) [rcv_nxt 265600160, rcv_wnd 262144, snt_ack 265469200, win_now 131184] [copied_seq 260040464->260040464 (0), unread 5559696, qlen 85, ofoq 0] returning 6104 bytes [5201<->39222]: tcp_recvmsg_locked(<-) // After each read, the algorithm for calculating the new receive // window in __tcp_cleanup_rbuf() finds it is too small to advertise // or to update tp->rcv_wnd. // Meanwhile, the peer thinks the window is zero, and will not send // any more data to trigger an update from the interrupt mode side. [5201<->39222]: tcp_recvmsg_locked(->) [5201<->39222]: __tcp_cleanup_rbuf(->) tp->rcv_wup: 265469200, tp->rcv_wnd: 262144, tp->rcv_nxt 265600160 [5201<->39222]: [new_win = 262144, win_now = 131184, 2 * win_now = 262368] [5201<->39222]: [new_win >= (2 * win_now) ? --> time_to_ack = 0] [5201<->39222]: NOT calling tcp_send_ack() [tp->rcv_wup: 265469200, tp->rcv_wnd: 262144, tp->rcv_nxt 265600160] [5201<->39222]: __tcp_cleanup_rbuf(<-) [rcv_nxt 265600160, rcv_wnd 262144, snt_ack 265469200, win_now 131184] [copied_seq 260099840->260171536 (71696), unread 5428624, qlen 83, ofoq 0] returning 131072 bytes [5201<->39222]: tcp_recvmsg_locked(<-) // The above pattern repeats again and again, since nothing changes // between the reads. [...] [5201<->39222]: tcp_recvmsg_locked(->) [5201<->39222]: __tcp_cleanup_rbuf(->) tp->rcv_wup: 265469200, tp->rcv_wnd: 262144, tp->rcv_nxt 265600160 [5201<->39222]: [new_win = 262144, win_now = 131184, 2 * win_now = 262368] [5201<->39222]: [new_win >= (2 * win_now) ? --> time_to_ack = 0] [5201<->39222]: NOT calling tcp_send_ack() [tp->rcv_wup: 265469200, tp->rcv_wnd: 262144, tp->rcv_nxt 265600160] [5201<->39222]: __tcp_cleanup_rbuf(<-) [rcv_nxt 265600160, rcv_wnd 262144, snt_ack 265469200, win_now 131184] [copied_seq 265600160->265600160 (0), unread 0, qlen 0, ofoq 0] returning 54672 bytes [5201<->39222]: tcp_recvmsg_locked(<-) // The receive queue is empty, but no new advertisement has been sent. // The peer still thinks the receive window is zero, and sends nothing. // We have ended up in a deadlock situation. Furthermore, we have observed that in these situations this side may send out an updated 'th->ack_seq´ which is not stored in tp->rcv_wup as it should be. Backing ack_seq seems to be harmless, but is of course still wrong from a protocol viewpoint. We fix this by updating the socket state correctly when a packet has been dropped because of memory exhaustion and we have to advertize a zero window. Further testing shows that the connection recovers neatly from the squeeze situation, and traffic can continue indefinitely. Fixes: e2142825c120 ("net: tcp: send zero-window ACK when no memory") Cc: Menglong Dong <menglong8.dong(a)gmail.com> Reviewed-by: Stefano Brivio <sbrivio(a)redhat.com> Signed-off-by: Jon Maloy <jmaloy(a)redhat.com> --- v1: -Posted on Apr 6, 2024: https://lore.kernel.org/all/20240406182107.261472-1-jmaloy@redhat.com/#r v2: -Improved commit log to clarify how we end up in this situation. -After feedback from Eric Dumazet, removed references to use of SO_PEEK and SO_PEEK_OFF which may lead to a misunderstanding about how this situation occurs. Those flags are used at the peer side's incoming connection, and not on this one. v3: -Clearing tp->pred_flags when we do a zero advertisement. -Edited (manually) and shortened the log documenting the problem. --- net/ipv4/tcp_output.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index 0e5b9a654254..bc95d2a5924f 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -265,11 +265,14 @@ static u16 tcp_select_window(struct sock *sk) u32 cur_win, new_win; /* Make the window 0 if we failed to queue the data because we - * are out of memory. The window is temporary, so we don't store - * it on the socket. + * are out of memory. */ - if (unlikely(inet_csk(sk)->icsk_ack.pending & ICSK_ACK_NOMEM)) + if (unlikely(inet_csk(sk)->icsk_ack.pending & ICSK_ACK_NOMEM)) { + tp->pred_flags = 0; + tp->rcv_wnd = 0; + tp->rcv_wup = tp->rcv_nxt; return 0; + } cur_win = tcp_receive_window(tp); new_win = __tcp_select_window(sk); -- 2.48.0

7 8

[PATCH v2 0/8] Draft, incomplete series introducing state migration
by Stefano Brivio 30 Jan '25

30 Jan '25

A bit more complete than the previous iteration: this adds fixes, interfaces for the TCP_REPAIR helper, and the first (working) basic source pre-migration handler, dumping (socket-side) sequence numbers into the flow table. v2: - drop 3/7, (7-bit hole in tcp_splice_conn), not really relevant - handle EOF (ENODATA) in read_remainder() and read_all_buf() - fix definition of reverse-endianness magic, and version match in migrate_target_read_header() - pass context to the handlers, not an arbitrary data pointer, as that's kind of useless - change passt-repair protocol to single-byte command, exit on EOF instead of special command - add interface and infrastructure for passt-repair - add basic pre-migration source handler dumping sequence numbers once sockets are switched to repair mode by helper Stefano Brivio (8): icmp, udp: Pad time_t timestamp to 64-bit to ease state migration flow, flow_table: Pad flow table entries to 128 bytes, hash entries to 32 bits flow_table: Use size in extern declaration for flowtab util: Add read_remainder() and read_all_buf() Introduce facilities for guest migration on top of vhost-user infrastructure Introduce passt-repair Add interfaces and configuration bits for passt-repair flow, tcp: Basic pre-migration source handler to dump sequence numbers Makefile | 24 +++-- conf.c | 46 ++++++++- epoll_type.h | 4 + flow.c | 43 ++++++++ flow.h | 19 ++-- flow_table.h | 15 ++- icmp_flow.h | 6 +- migrate.c | 265 +++++++++++++++++++++++++++++++++++++++++++++++++ migrate.h | 88 ++++++++++++++++ passt-repair.c | 117 ++++++++++++++++++++++ passt.1 | 11 ++ passt.c | 11 +- passt.h | 7 ++ repair.c | 192 +++++++++++++++++++++++++++++++++++ repair.h | 16 +++ tap.c | 65 +----------- tcp.c | 56 +++++++++++ tcp_conn.h | 5 + udp_flow.h | 6 +- util.c | 142 ++++++++++++++++++++++++++ util.h | 3 + vu_common.c | 124 +++++++++++++++-------- vu_common.h | 2 +- 23 files changed, 1136 insertions(+), 131 deletions(-) create mode 100644 migrate.c create mode 100644 migrate.h create mode 100644 passt-repair.c create mode 100644 repair.c create mode 100644 repair.h -- 2.43.0

2 19

Handling discontinuous packet buffers
by David Gibson 29 Jan '25

29 Jan '25

Hi Laurent, I had a closer look at extending IOV_REMOVE_HEADER() and the rest to handle discontinuous buffers, but I realised it's a bit more complex than I thought. What we need to do is a bit different depending on whether we're sending or receiving. The single current user of IOV_REMOVE_HEADER() is sending (guest Rx), as are the places I had been thinking about using it. For the send path, obviously, it's not helpful to linearise the section from the IOV, instead we need to write things into the IOV, maybe discontiguously. The IOV_REMOVE_HEADER() interface doesn't really work for this. I was wondering about a sort of "reserve/set" interface. You "reserve" the header region, which gives you either a pointer directly into the iov, or a pointer to a temporary buffer. Then you "set", which is a no-op if you had a direct pointer, but if you had the temp buffer, then it copies back with iov_from_buf(). I don't love this interface; it's pretty easy to forget the "set" stage, and not notice (if the kernel is usually giving contiguous buffers, the tests won't catch it). Not sure what else to do at this stage, though. IIUC, the problems you're grappling with right now are on this path. However, I suspect we could hit this problem on the other path (guest Tx) as well: that would be if the guest gives us an outbound frame in multiple discontiguous buffers. For that case we do want to linearise the sections we need to parse, but extending IOV_REMOVE_HEADER() isn't enough. To handle this case we need to extend the packet pool stuff to cope with discontiguous frames. It would be nice, of course, if we can share at least some of the logic between the two paths, but how to make an interface that works for that isn't really clear to me yet. -- David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson

1 0