- dev - passt

[PATCH v2 0/4] Fix race condition while closing spliced connections
by David Gibson 26 May '26

26 May '26

Fix bug 202, where a race condition could cause connections to be incorrectly reset in certain circumstances. Patch 2/4 is the bug fix proper. 1/4 improves error reporting and debugging messages in the vicinity. Patches 3..4/4 are some small cleanups I noticed in the area while working on the fix. Link: https://bugs.passt.top/show_bug.cgi?id=202 v2: * Formatting and comment fixes, per Stefano's review * Dropped patches 5 & 6 for now. I still think they're worthwhile, but are closely related to other oddities that need some work. I didn't want to delay the bugfix itself. David Gibson (4): tcp_splice: Improve error reporting tcp_splice: Avoid missing EOF recognition while forwarding tcp_splice: Clean up flow control path for splice forwarding tcp_splice: Simplify tracking of read/written bytes flow.h | 7 ++ log.h | 19 +++--- passt.c | 2 +- tcp_conn.h | 6 +- tcp_splice.c | 185 ++++++++++++++++++++++++++++++--------------------- tcp_splice.h | 2 +- 6 files changed, 129 insertions(+), 92 deletions(-) -- 2.54.0

2 6

[PATCH v7 0/4] vhost-user,tcp: Handle multiple iovec entries per virtqueue element
by Laurent Vivier 26 May '26

26 May '26

This is the TCP counterpart to the UDP multi-iov series. It converts the TCP vhost-user receive path from direct pointer arithmetic (via vu_eth(), vu_ip(), etc.) to the iov_tail abstraction, removing the assumption that all headers reside in a single contiguous buffer. With this series applied, the TCP path correctly handles virtio-net drivers that provide multiple buffers per virtqueue element (e.g. iPXE provides the vnet header in the first buffer and the frame payload in a second one), matching the support already present in the UDP path. Based-on: 20260520095526.21519-1-lvivier(a)redhat.com v7: - Use initializers for struct tcphdr - Rename reused 'payload' variable to 'l2frame' - Fix vu_pad() double-offset bug in tcp_vu_data_from_sock() - Remove redundant *elem_used = 0 before truncation loop - Add missing @vnlen to tcp_vu_send_dup() doc comment v6: - Rebase on v8 of UDP series (tcp_update_csum() takes dlen rather than l4len) v5: - Use l2len variable for pcap_iov() length in tcp_vu_send_flag() - Add braces - Move pcap_iov() before vu_flush() - Remove vu_flush() from tcp_vu_send_dup(), let the caller handle it v4: - fix error during rebase, s/vu_pad_len/vu_pad/ v3: - Rebased on top of [PATCH 00/10] vhost-user: Preparatory series for multiple iovec entries per virtqueue element v2: - add "tcp: Encode checksum computation flags in a single parameter" - remove IOV_PUT_HEADER()/with_header() and use IOV_PUSH_HEADER() - don't use the iov_tail to provide the headers to the functions Laurent Vivier (4): tcp: Encode checksum computation flags in a single parameter tcp_vu: Build headers on the stack and write them into the iovec tcp_vu: Support multibuffer frames in tcp_vu_sock_recv() tcp_vu: Support multibuffer frames in tcp_vu_send_flag() iov.c | 1 - tcp.c | 25 +-- tcp_buf.c | 23 +-- tcp_internal.h | 7 +- tcp_vu.c | 409 +++++++++++++++++++++++++++++-------------------- vu_common.h | 20 --- 6 files changed, 272 insertions(+), 213 deletions(-) -- 2.54.0

2 4

[PATCH v11 0/3] vhost-user,udp: Handle multiple iovec entries per virtqueue element
by Laurent Vivier 26 May '26

26 May '26

Some virtio-net drivers (notably iPXE) provide descriptors where the vnet header and the frame payload are in separate buffers, resulting in two iovec entries per virtqueue element. Currently, the RX (host to guest) path assumes a single iovec per element, which triggers: ASSERTION FAILED in virtqueue_map_desc (virtio.c:403): num_sg < max_num_sg This series reworks the UDP vhost-user receive path to support multiple iovec entries per element, fixing the iPXE crash. This series only addresses the UDP path. TCP vhost-user will be updated to use multi-iov elements in a subsequent series. v11: - Rebase on master - use sizeof(*uh) instead of sizeof(struct udphdr) in udp_tap_prepare() - improve @payload doc in udp_vu_sock_recv() - add R-b from David v10: - udp_update_hdr4()/udp_update_hdr6(): take separate @uh pointer and @payload - udp_tap_prepare(): extract uh from UDP_IOV_PAYLOAD, @payload is UDP payload only - udp_vu_sock_recv(): extract UDP payload from the function - udp_vu_prepare(): take @payload as parameter, compute checksum conditionally via !*c->pcap instead of hardcoded true - udp_vu_csum(): removed v9: - iov_push_header_(): skip memcpy() when the source pointer already aliases the iov buffer - udp_update_hdr4()//udp_update_hdr6()/usp_csum(): update payload doc - udp_tap_prepare(): use existing (*tap_iov)[UDP_IOV_PAYLOAD] v8: - Add Reviewed-by from David for 1/3 and 2/3 - Remove VLA to use fixed size (VIRTQUEUE_MAX_SIZE) - Rename udp_frame to datagram - Use sizeof(uh) instead of sizeof(struct udphdr) - Push back uh in memory in udp_vu_csum() v7: - In udp_vu_sock_to_tap(), introduce iov_still_needed variable - Fix iov_tail @data doc comments in udp_vu_prepare()/udp_vu_csum() - Don't rewrap the virtio checksum comment in udp_update_hdr6() - Add NULL check for uh in udp_vu_csum() v6: - Rebased on top of [PATCH 00/10] vhost-user: Preparatory series for multiple iovec entries per virtqueue element v5: - This version doesn't change the padding system regarding v4, it's a complex task that will be addressed in another version - reorder patches and add new patches - remove IOV_PUT_HEADER()/with_header() and introduce IOV_PUSH_HEADER() - don't use the iov_tail to provide the headers to the functions - move vu_set_vnethdr() to vu_flush(), extract vu_queue_notify() - move vu_flush() inside loop in tcp_vu_data_from_sock() to flush data by frame and not by full data length v4: - rebase - replace ASSERT() by assert() v3: - include the series "Decouple iovec management from virtqueues elements" - because of this series, drop: "vu_common: Accept explicit iovec counts in vu_set_element()" "vu_common: Accept explicit iovec count per element in vu_init_elem()" "vu_common: Prepare to use multibuffer with guest RX" "vhost-user,udp: Use 2 iovec entries per element" - drop "vu_common: Pass iov_tail to vu_set_vnethdr()" as the specs insures a buffer is big enough to contain vnet header - introduce "with_header()" and merge "udp: Pass iov_tail to udp_update_hdr4()/udp_update_hdr6()" and "udp_vu: Use iov_tail in udp_vu_prepare()" to use it v2: - add iov_truncate(), iov_memset() - remove iov_tail_truncate() and iov_tail_zero_end() - manage 802.3 minimum frame size Laurent Vivier (3): udp_vu: Allow virtqueue elements with multiple iovec entries iov: Introduce IOV_PUSH_HEADER() macro udp: Pass iov_tail to udp_update_hdr4()/udp_update_hdr6() iov.c | 30 ++++++++++++ iov.h | 11 +++++ udp.c | 74 ++++++++++++++-------------- udp_internal.h | 6 ++- udp_vu.c | 129 +++++++++++++++++++------------------------------ 5 files changed, 131 insertions(+), 119 deletions(-) -- 2.54.0

2 1

[PATCH] Makefile: Remove misleading comments on BASE_*FLAGS
by David Gibson 26 May '26

26 May '26

I added these comments attempting to describe the difference between BASE_CFLAGS and CFLAGS. However, the description given isn't really accurate, and I'm not even sure the concept I'm trying to describe is coherent. Just remove it. Suggested-by: Stefano Brivio <sbrivio(a)redhat.com> Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au> --- Makefile | 4 ---- 1 file changed, 4 deletions(-) diff --git a/Makefile b/Makefile index c4af57b4..0a0a60b0 100644 --- a/Makefile +++ b/Makefile @@ -30,15 +30,11 @@ ifeq ($(shell $(CC) -O2 -dM -E - < /dev/null 2>&1 | grep ' _FORTIFY_SOURCE ' > / FORTIFY_FLAG := -D_FORTIFY_SOURCE=2 endif -# Mandatory preprocessor flags that won't be overridden with $(CPPFLAGS) -# FIXME: Could some of these be default, rather than required? BASE_CPPFLAGS := -D_XOPEN_SOURCE=700 -D_GNU_SOURCE $(FORTIFY_FLAG) BASE_CPPFLAGS += -DPAGE_SIZE=$(shell getconf PAGE_SIZE) BASE_CPPFLAGS += -DVERSION=\"$(VERSION)\" BASE_CPPFLAGS += -DDUAL_STACK_SOCKETS=$(DUAL_STACK_SOCKETS) -# Mandatory compiler flags that won't be overridden with $(CFLAGS) -# FIXME: Could some of these be default, rather than required? BASE_CFLAGS := -std=c11 -pie -fPIE -O2 BASE_CFLAGS += -pedantic -Wall -Wextra -Wno-format-zero-length -Wformat-security -- 2.54.0

2 1

[PATCH 0/2] Fix equence collision with neighbour notification, and one nit
by Stefano Brivio 26 May '26

26 May '26

Patch 1/2 fixes https://bugs.passt.top/show_bug.cgi?id=203, while patch 2/2 is a cosmetic follow-up. Stefano Brivio (2): netlink: Use regular request/response netlink socket for initial neighbour sync netlink: Fix comments to variables for netlink sockets and sequence netlink.c | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) -- 2.43.0

2 5

[PATCH v4 00/10] vhost-user: Preparatory series for multiple iovec entries per virtqueue element
by Laurent Vivier 26 May '26

26 May '26

Currently, the vhost-user path assumes each virtqueue element contains exactly one iovec entry covering the entire frame. This assumption breaks as some virtio-net drivers (notably iPXE) provide descriptors where the vnet header and the frame payload are in separate buffers, resulting in two iovec entries per virtqueue element. This series refactors the vhost-user data path so that frame lengths, header sizes, and padding are tracked and passed explicitly rather than being derived from iovec sizes. This decoupling is a prerequisite for correctly handling padding of multi-buffer frames. The changes in this series can be split in 3 groups: - New iov helpers (patches 1-2): iov_memset() and iov_memcpy() operate across iovec boundaries. These are needed by the final patch to pad and copy frame data when a frame spans multiple iovec entries. - Structural refactoring (patches 3-5): Move vnethdr setup into vu_flush(), separate virtqueue management from socket I/O in the UDP path, and pass iov arrays explicitly instead of using file-scoped state. These changes make it possible to pass explicit frame lengths through the stack, which is required to pad frames independently of iovec layout. - Explicit length passing throughout the stack (patches 6-10): Thread explicit L4, L2, frame, and data lengths through checksum, pcap, vu_flush(), and tcp_fill_headers(), replacing lengths that were previously derived from iovec sizes. With lengths tracked explicitly, the final patch can centralise Ethernet frame padding into vu_collect() and a new vu_pad() helper that correctly pads frames spanning multiple iovec entries. v4: - rebase - iov_memcpy: use size_t for loop indices i and j - udp_vu: reorder elem[] declaration for inverted christmas tree style - pcap: wrap pcap_iov() declaration and definition to respect line length - write_remainder(): update length parameter description - Add Reviewed-by tags from Jon and David v3: - csum_udp4()/csum_udp6()/udp_vu_csum receive payload length (dlen) rather than l4len - Add a length parameter to write_remainder() and use it in pcap_frame() v2: - Rename iov_memcopy() to iov_memcpy() and use clearer parameter names - Use clearer code in pcap_frame() - Add braces around bodies in pcap.c and tcp_vu.c for style consistency - Extract l2len variable in tap_add_packet() and tcp_vu_send_flag() to avoid repeating the same expression - Fix indentation alignment of iov_skip_bytes() arguments in tcp_vu_c - Introduce fill_size variable in vu_flush() - Reposition comment for ETH_ZLEN in vu_collect() Laurent Vivier (10): iov: Introduce iov_memset() iov: Add iov_memcpy() to copy data between iovec arrays vu_common: Move vnethdr setup into vu_flush() udp_vu: Move virtqueue management from udp_vu_sock_recv() to its caller udp_vu: Pass iov explicitly to helpers instead of using file-scoped array checksum: Pass explicit L4 length to checksum functions pcap: Pass explicit L2 length to pcap_iov() vu_common: Pass explicit frame length to vu_flush() tcp: Pass explicit data length to tcp_fill_headers() vhost-user: Centralise Ethernet frame padding in vu_collect() and vu_pad() checksum.c | 43 +++++++----- checksum.h | 6 +- iov.c | 77 ++++++++++++++++++++++ iov.h | 5 ++ pcap.c | 29 ++++++--- pcap.h | 3 +- tap.c | 10 +-- tcp.c | 14 ++-- tcp_buf.c | 3 +- tcp_internal.h | 2 +- tcp_vu.c | 66 ++++++++++--------- udp.c | 5 +- udp_vu.c | 173 +++++++++++++++++++++++++------------------------ util.c | 31 +++++++-- util.h | 3 +- vu_common.c | 58 ++++++++++------- vu_common.h | 5 +- 17 files changed, 339 insertions(+), 194 deletions(-) -- 2.54.0

4 29

[PATCH v7 2/4] tcp_vu: Build headers on the stack and write them into the iovec
by Laurent Vivier 22 May '26

22 May '26

tcp_vu_prepare() currently assumes the first iovec element provided by the guest is large enough to hold all L2-L4 headers, and builds them in place via pointer casts into iov[0].iov_base. This assumption is enforced by an assert(). Since the headers in the buffer are uninitialized anyway, we can just as well build the Ethernet, IP, and TCP headers on the stack instead, and write them into the iovec with IOV_PUSH_HEADER(). This mirrors the approach already used in udp_vu_prepare(), and prepares for support of elements with multiple iovecs. Signed-off-by: Laurent Vivier <lvivier(a)redhat.com> Reviewed-by: Jon Maloy <jmaloy(a)redhat.com> Reviewed-by: David Gibson <david(a)gibson.dropbear.id.au> --- tcp_vu.c | 64 +++++++++++++++++++++++++------------------------------- 1 file changed, 28 insertions(+), 36 deletions(-) diff --git a/tcp_vu.c b/tcp_vu.c index 65e7dfe63437..c13c45f71f3b 100644 --- a/tcp_vu.c +++ b/tcp_vu.c @@ -298,49 +298,41 @@ static void tcp_vu_prepare(const struct ctx *c, struct tcp_tap_conn *conn, bool v6 = !(inany_v4(&toside->eaddr) && inany_v4(&toside->oaddr)); size_t hdrlen = tcp_vu_hdrlen(v6); struct iov_tail payload = IOV_TAIL(iov, iov_cnt, hdrlen); - char *base = iov[0].iov_base; - struct ipv6hdr *ip6h = NULL; - struct iphdr *ip4h = NULL; - struct tcphdr *th; - struct ethhdr *eh; - - /* we guess the first iovec provided by the guest can embed - * all the headers needed by L2 frame, including any padding - */ - assert(iov[0].iov_len >= hdrlen); - - eh = vu_eth(base); - - memcpy(eh->h_dest, c->guest_mac, sizeof(eh->h_dest)); + struct tcphdr th = { + .doff = sizeof(th) / 4, + .ack = 1, + .psh = push, + }; + struct iov_tail l2frame; + struct ipv6hdr ip6h; + struct iphdr ip4h; + struct ethhdr eh; + + memcpy(eh.h_dest, c->guest_mac, sizeof(eh.h_dest)); /* initialize header */ - if (!v6) { - eh->h_proto = htons(ETH_P_IP); - - ip4h = vu_ip(base); - *ip4h = (struct iphdr)L2_BUF_IP4_INIT(IPPROTO_TCP); - th = vu_payloadv4(base); - } else { - eh->h_proto = htons(ETH_P_IPV6); - - ip6h = vu_ip(base); - *ip6h = (struct ipv6hdr)L2_BUF_IP6_INIT(IPPROTO_TCP); + if (!v6) + ip4h = (struct iphdr)L2_BUF_IP4_INIT(IPPROTO_TCP); + else + ip6h = (struct ipv6hdr)L2_BUF_IP6_INIT(IPPROTO_TCP); - th = vu_payloadv6(base); - } + tcp_fill_headers(c, conn, &eh, v6 ? NULL : &ip4h, v6 ? &ip6h : NULL, &th, + &payload, dlen, *csum_flags, conn->seq_to_tap); - memset(th, 0, sizeof(*th)); - th->doff = sizeof(*th) / 4; - th->ack = 1; - th->psh = push; + /* Preserve TCP_CSUM, overwrite IP4_CSUM as we set the checksum */ + if (!v6) + *csum_flags = (*csum_flags & TCP_CSUM) | ip4h.check; - tcp_fill_headers(c, conn, eh, ip4h, ip6h, th, &payload, dlen, - *csum_flags, conn->seq_to_tap); + /* write headers */ + l2frame = IOV_TAIL(iov, iov_cnt, VNET_HLEN); - /* Preserve TCP_CSUM, overwrite IP4_CSUM as we set the checksum */ - if (ip4h) - *csum_flags = (*csum_flags & TCP_CSUM) | ip4h->check; + IOV_PUSH_HEADER(&l2frame, eh); + if (!v6) + IOV_PUSH_HEADER(&l2frame, ip4h); + else + IOV_PUSH_HEADER(&l2frame, ip6h); + IOV_PUSH_HEADER(&l2frame, th); } /** -- 2.54.0

2 5

[PATCH 0/6] Fix race condition while closing spliced connections
by David Gibson 22 May '26

22 May '26

Fix bug 202, where a race condition could cause connections to be incorrectly reset in certain circumstances. Patch 2/6 is the bug fix proper. 1/6 improves error reporting and debugging messages in the vicinity. Patches 3..6/6 are some cleanups I noticed in the area while working on the fix. Link: https://bugs.passt.top/show_bug.cgi?id=202 David Gibson (6): tcp_splice: Improve error reporting tcp_splice: Avoid missing EOF recognition while forwarding tcp_splice: Clean up flow control path for splice forwarding tcp_splice: Simplify tracking of read/written bytes tcp_splice: Simplify EPOLLRDHUP / eof / FIN handling tcp_splice: Simplify shutdown(2) handling tcp_conn.h | 6 +- tcp_splice.c | 180 +++++++++++++++++++++++++++------------------------ 2 files changed, 97 insertions(+), 89 deletions(-) -- 2.54.0

2 19

[PATCH v6 0/4] vhost-user,tcp: Handle multiple iovec entries per virtqueue element
by Laurent Vivier 21 May '26

21 May '26

This is the TCP counterpart to the UDP multi-iov series. It converts the TCP vhost-user receive path from direct pointer arithmetic (via vu_eth(), vu_ip(), etc.) to the iov_tail abstraction, removing the assumption that all headers reside in a single contiguous buffer. With this series applied, the TCP path correctly handles virtio-net drivers that provide multiple buffers per virtqueue element (e.g. iPXE provides the vnet header in the first buffer and the frame payload in a second one), matching the support already present in the UDP path. Based-on: 20260416160926.3822963-1-lvivier(a)redhat.com v6: - Rebase on v8 of UDP series (tcp_update_csum() takes dlen rather than l4len) v5: - Use l2len variable for pcap_iov() length in tcp_vu_send_flag() - Add braces - Move pcap_iov() before vu_flush() - Remove vu_flush() from tcp_vu_send_dup(), let the caller handle it v4: - fix error during rebase, s/vu_pad_len/vu_pad/ v3: - Rebased on top of [PATCH 00/10] vhost-user: Preparatory series for multiple iovec entries per virtqueue element v2: - add "tcp: Encode checksum computation flags in a single parameter" - remove IOV_PUT_HEADER()/with_header() and use IOV_PUSH_HEADER() - don't use the iov_tail to provide the headers to the functions Laurent Vivier (4): tcp: Encode checksum computation flags in a single parameter tcp_vu: Build headers on the stack and write them into the iovec tcp_vu: Support multibuffer frames in tcp_vu_sock_recv() tcp_vu: Support multibuffer frames in tcp_vu_send_flag() iov.c | 1 - tcp.c | 25 +-- tcp_buf.c | 23 +-- tcp_internal.h | 7 +- tcp_vu.c | 403 +++++++++++++++++++++++++++++-------------------- vu_common.h | 20 --- 6 files changed, 270 insertions(+), 209 deletions(-) -- 2.53.0

3 16

[PATCH 1/6] tcp_splice: Improve error reporting
by David Gibson 21 May '26

21 May '26

A number of things can, at least theoretically, go wrong when forwarding data across a spliced connection. We generally handle this by resetting the connection on both sides. However, in many cases we don't log any message about why the connection was reset, which can make it hard to debug why this is happening. Add a bunch of debug and error logging to make this easier to figure out. Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au> --- tcp_splice.c | 31 +++++++++++++++++++++++-------- 1 file changed, 23 insertions(+), 8 deletions(-) diff --git a/tcp_splice.c b/tcp_splice.c index 42ee8abc..1359d6b8 100644 --- a/tcp_splice.c +++ b/tcp_splice.c @@ -502,15 +502,18 @@ void tcp_splice_sock_handler(struct ctx *c, union epoll_ref ref, if (rc) flow_perror(conn, "Error retrieving SO_ERROR"); else - flow_trace(conn, "Error event on socket: %s", - strerror_(err)); - + flow_dbg(conn, "Error event on %s socket: %s", + pif_name(conn->f.pif[evsidei]), + strerror_(err)); goto reset; } if (conn->events == SPLICE_CONNECT) { - if (!(events & EPOLLOUT)) + if (!(events & EPOLLOUT)) { + flow_err(conn, "Unexpected events 0x%x during connect", + events); goto reset; + } if (tcp_splice_connect_finish(c, conn)) goto reset; } @@ -545,8 +548,11 @@ retry: SPLICE_F_MOVE | SPLICE_F_NONBLOCK); while (readlen < 0 && errno == EINTR); - if (readlen < 0 && errno != EAGAIN) + if (readlen < 0 && errno != EAGAIN) { + flow_perror(conn, "Splicing from %s socket", + pif_name(conn->f.pif[fromsidei])); goto reset; + } flow_trace(conn, "%zi from read-side call", readlen); @@ -569,8 +575,11 @@ retry: SPLICE_F_MOVE | more | SPLICE_F_NONBLOCK); while (written < 0 && errno == EINTR); - if (written < 0 && errno != EAGAIN) + if (written < 0 && errno != EAGAIN) { + flow_perror(conn, "Splicing to %s socket", + pif_name(conn->f.pif[!fromsidei])); goto reset; + } flow_trace(conn, "%zi from write-side call (passed %zi)", written, c->tcp.pipe_size); @@ -627,8 +636,11 @@ retry: flow_foreach_sidei(sidei) { if ((conn->events & FIN_RCVD(sidei)) && !(conn->events & FIN_SENT(!sidei))) { - if (shutdown(conn->s[!sidei], SHUT_WR) < 0) + if (shutdown(conn->s[!sidei], SHUT_WR) < 0) { + flow_perror(conn, "shutdown() on %s", + pif_name(conn->f.pif[!sidei])); goto reset; + } conn_event(conn, FIN_SENT(!sidei)); } } @@ -647,8 +659,11 @@ retry: goto swap; } - if (events & EPOLLHUP) + if (events & EPOLLHUP) { + flow_dbg(conn, "Hangup from %s socket", + pif_name(conn->f.pif[evsidei])); goto reset; + } return; -- 2.54.0

2 3