- dev - passt

[PATCH v2 0/9] tcp: Fix throughput issues with non-local peers
by Stefano Brivio 08 Dec '25

08 Dec '25

Patch 2/9 is the most relevant fix here, as we currently advertise a window that might be too big for what we can write to the socket, causing retransmissions right away and occasional high latency on short transfers to non-local peers. Mostly as a consequence of fixing that, we now need several improvements and small fixes, including, most notably, an adaptive approach to pick the interval between checks for socket-side ACKs (patch 3/9), and several tricks to reliably trigger TCP buffer size auto-tuning as implemented by the Linux kernel (patches 5/9 and 7/9). These changes make some existing issues more relevant, fixed by the other patches. With this series, I'm getting the expected (wirespeed) throughput for transfers between peers with varying non-local RTTs: I checked different guests bridged on the same machine (~500 us) and hosts with increasing distance using iperf3, as well as HTTP transfers only for some hosts I have control over (500 us and 5 ms case). With increasing RTTs, I can finally see the throughput converging to the available bandwidth reasonably fast: * 500 us RTT, ~10 Gbps available: [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 785 MBytes 6.57 Gbits/sec 13 1.84 MBytes [ 5] 1.00-2.00 sec 790 MBytes 6.64 Gbits/sec 0 1.92 MBytes * 5 ms, 1 Gbps: [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 4.88 MBytes 40.9 Mbits/sec 22 240 KBytes [ 5] 1.00-2.00 sec 46.2 MBytes 388 Mbits/sec 34 900 KBytes [ 5] 2.00-3.00 sec 110 MBytes 923 Mbits/sec 0 1.11 MBytes * 10 ms, 1 Gbps: [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 67.9 MBytes 569 Mbits/sec 2 960 KBytes [ 5] 1.00-2.00 sec 110 MBytes 926 Mbits/sec 0 1.05 MBytes [ 5] 2.00-3.00 sec 111 MBytes 934 Mbits/sec 0 1.17 MBytes * 24 ms, 1 Gbps: [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 2.50 MBytes 20.9 Mbits/sec 16 240 KBytes [ 5] 1.00-2.00 sec 1.50 MBytes 12.6 Mbits/sec 9 120 KBytes [ 5] 2.00-3.00 sec 99.2 MBytes 832 Mbits/sec 4 2.40 MBytes [ 5] 3.00-4.00 sec 122 MBytes 1.03 Gbits/sec 1 3.16 MBytes [ 5] 4.00-5.00 sec 119 MBytes 1.00 Gbits/sec 0 4.16 MBytes * 40 ms, ~600 Mbps: [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 2.12 MBytes 17.8 Mbits/sec 0 600 KBytes [ 5] 1.00-2.00 sec 3.25 MBytes 27.3 Mbits/sec 14 420 KBytes [ 5] 2.00-3.00 sec 31.5 MBytes 264 Mbits/sec 11 1.29 MBytes [ 5] 3.00-4.00 sec 72.5 MBytes 608 Mbits/sec 0 1.46 MBytes * 100 ms, 1 Gbps: [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 4.88 MBytes 40.9 Mbits/sec 1 840 KBytes [ 5] 1.00-2.00 sec 1.62 MBytes 13.6 Mbits/sec 9 240 KBytes [ 5] 2.00-3.00 sec 5.25 MBytes 44.0 Mbits/sec 5 780 KBytes [ 5] 3.00-4.00 sec 9.75 MBytes 81.8 Mbits/sec 0 1.29 MBytes [ 5] 4.00-5.00 sec 15.8 MBytes 132 Mbits/sec 0 1.99 MBytes [ 5] 5.00-6.00 sec 22.9 MBytes 192 Mbits/sec 0 3.05 MBytes [ 5] 6.00-7.00 sec 132 MBytes 1.11 Gbits/sec 0 7.62 MBytes * 114 ms, 1 Gbps: [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.62 MBytes 13.6 Mbits/sec 8 420 KBytes [ 5] 1.00-2.00 sec 2.12 MBytes 17.8 Mbits/sec 15 120 KBytes [ 5] 2.00-3.00 sec 26.0 MBytes 218 Mbits/sec 50 1.82 MBytes [ 5] 3.00-4.00 sec 103 MBytes 865 Mbits/sec 31 5.10 MBytes [ 5] 4.00-5.00 sec 111 MBytes 930 Mbits/sec 0 5.92 MBytes * 153 ms, 1 Gbps: [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.12 MBytes 9.43 Mbits/sec 2 180 KBytes [ 5] 1.00-2.00 sec 2.12 MBytes 17.8 Mbits/sec 11 180 KBytes [ 5] 2.00-3.00 sec 12.6 MBytes 106 Mbits/sec 40 1.29 MBytes [ 5] 3.00-4.00 sec 44.5 MBytes 373 Mbits/sec 22 2.75 MBytes [ 5] 4.00-5.00 sec 86.0 MBytes 721 Mbits/sec 0 6.68 MBytes [ 5] 5.00-6.00 sec 120 MBytes 1.01 Gbits/sec 119 6.97 MBytes [ 5] 6.00-7.00 sec 110 MBytes 927 Mbits/sec 0 6.97 MBytes * 186 ms, 1 Gbps: [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.12 MBytes 9.43 Mbits/sec 3 180 KBytes [ 5] 1.00-2.00 sec 512 KBytes 4.19 Mbits/sec 4 120 KBytes [ 5] 2.00-3.00 sec 2.12 MBytes 17.8 Mbits/sec 6 360 KBytes [ 5] 3.00-4.00 sec 27.1 MBytes 228 Mbits/sec 6 1.11 MBytes [ 5] 4.00-5.00 sec 38.2 MBytes 321 Mbits/sec 0 1.99 MBytes [ 5] 5.00-6.00 sec 38.2 MBytes 321 Mbits/sec 0 2.46 MBytes [ 5] 6.00-7.00 sec 69.2 MBytes 581 Mbits/sec 71 3.63 MBytes [ 5] 7.00-8.00 sec 110 MBytes 919 Mbits/sec 0 5.92 MBytes * 271 ms, 1 Gbps: [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.12 MBytes 9.43 Mbits/sec 0 320 KBytes [ 5] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec 0 600 KBytes [ 5] 2.00-3.00 sec 512 KBytes 4.19 Mbits/sec 7 420 KBytes [ 5] 3.00-4.00 sec 896 KBytes 7.34 Mbits/sec 8 60.0 KBytes [ 5] 4.00-5.00 sec 2.62 MBytes 22.0 Mbits/sec 13 420 KBytes [ 5] 5.00-6.00 sec 12.1 MBytes 102 Mbits/sec 7 1020 KBytes [ 5] 6.00-7.00 sec 19.9 MBytes 167 Mbits/sec 0 1.82 MBytes [ 5] 7.00-8.00 sec 17.9 MBytes 150 Mbits/sec 44 1.76 MBytes [ 5] 8.00-9.00 sec 57.4 MBytes 481 Mbits/sec 30 2.70 MBytes [ 5] 9.00-10.00 sec 88.0 MBytes 738 Mbits/sec 0 6.45 MBytes * 292 ms, ~ 600 Mbps: [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.12 MBytes 9.43 Mbits/sec 0 450 KBytes [ 5] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec 3 180 KBytes [ 5] 2.00-3.00 sec 640 KBytes 5.24 Mbits/sec 4 120 KBytes [ 5] 3.00-4.00 sec 384 KBytes 3.15 Mbits/sec 2 120 KBytes [ 5] 4.00-5.00 sec 4.50 MBytes 37.7 Mbits/sec 1 660 KBytes [ 5] 5.00-6.00 sec 13.0 MBytes 109 Mbits/sec 3 960 KBytes [ 5] 6.00-7.00 sec 64.5 MBytes 541 Mbits/sec 0 2.17 MBytes * 327 ms, 1 Gbps: [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.12 MBytes 9.43 Mbits/sec 0 600 KBytes [ 5] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec 4 240 KBytes [ 5] 2.00-3.00 sec 768 KBytes 6.29 Mbits/sec 4 120 KBytes [ 5] 3.00-4.00 sec 1.62 MBytes 13.6 Mbits/sec 5 120 KBytes [ 5] 4.00-5.00 sec 1.88 MBytes 15.7 Mbits/sec 0 480 KBytes [ 5] 5.00-6.00 sec 17.6 MBytes 148 Mbits/sec 14 1.05 MBytes [ 5] 6.00-7.00 sec 35.1 MBytes 295 Mbits/sec 0 2.58 MBytes [ 5] 7.00-8.00 sec 45.2 MBytes 380 Mbits/sec 0 4.63 MBytes [ 5] 8.00-9.00 sec 27.0 MBytes 226 Mbits/sec 96 3.93 MBytes [ 5] 9.00-10.00 sec 85.9 MBytes 720 Mbits/sec 67 4.22 MBytes [ 5] 10.00-11.00 sec 118 MBytes 986 Mbits/sec 0 9.67 MBytes [ 5] 11.00-12.00 sec 124 MBytes 1.04 Gbits/sec 0 15.9 MBytes For short transfers, we strictly stick to the available sending buffer size to (almost) make sure we avoid local retransmissions, and significantly decrease transfer time as a result: from 1.2 s to 60 ms for a 5 MB HTTP transfer from a container hosted in a virtual machine to another guest. v2: - Add 1/9, factoring out a generic version of the scaling function we already use for tcp_get_sndbuf(), as I'm now using it in 7/9 as well - in 3/9, use 4 bits instead of 3 to represent the RTT, which is important as we now use RTT values for more than just the ACK checks - in 5/9, instead of just comparing the sending buffer to SNDBUF_BIG to decide when to acknowledge data, use an adaptive approach based on the bandwidth-delay product - in 6/9, clarify the relationship between SWS avoidance and Nagle's algorithm, and introduce a reference to RFC 813, Section 4 - in 7/9, use an adaptive approach based on the product of bytes sent (and acknowledged) so far and RTT, instead of the previous approach based on bytes sent only, as it allows us to converge to the expected throughput much quicker with high RTT destinations Stefano Brivio (9): tcp, util: Add function for scaling to linearly interpolated factor, use it tcp: Limit advertised window to available, not total sending buffer size tcp: Adaptive interval based on RTT for socket-side acknowledgement checks tcp: Don't clear ACK_TO_TAP_DUE if we're advertising a zero-sized window tcp: Acknowledge everything if it looks like bulk traffic, not interactive tcp: Don't limit window to less-than-MSS values, use zero instead tcp: Allow exceeding the available sending buffer size in window advertisements tcp: Send a duplicate ACK also on complete sendmsg() failure tcp: Skip redundant ACK on partial sendmsg() failure README.md | 2 +- tcp.c | 168 ++++++++++++++++++++++++++++++++++++++++++----------- tcp_conn.h | 9 +++ util.c | 52 +++++++++++++++++ util.h | 2 + 5 files changed, 197 insertions(+), 36 deletions(-) -- 2.43.0

2 22

[PATCH v2] iov: Fix coding style of basic (non-IOV_TAIL) parts
by Stefano Brivio 08 Dec '25

08 Dec '25

I happened to touch these functions for a change that turned out to be unnecessary, but coding style and documentation could still benefit from some rephrasing and fixes. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- v2: Make description of @offset consistent between iov_from_buf() and iov_to_buf() iov.c | 68 +++++++++++++++++++++++++---------------------------------- 1 file changed, 29 insertions(+), 39 deletions(-) diff --git a/iov.c b/iov.c index 8c74a59..ad726da 100644 --- a/iov.c +++ b/iov.c @@ -20,24 +20,21 @@ * Contributions after 2012-01-13 are licensed under the terms of the * GNU GPL, version 2 or (at your option) any later version. */ + #include <sys/socket.h> #include "util.h" #include "iov.h" - /** - * iov_skip_bytes() - Skip leading bytes of an IO vector - * @iov: IO vector + * iov_skip_bytes() - Find index and offset in iovec array given byte offset + * @iov: iovec array * @n: Number of entries in @iov - * @skip: Number of leading bytes of @iov to skip - * @offset: Offset of first unskipped byte in its @iov entry + * @skip: Byte offset: leading bytes of @iov to skip + * @offset: Offset within matching @iov entry, set on return, can be NULL * - * Return: index I of individual struct iovec which contains the byte at @skip - * bytes into the vector (as though all its buffers were contiguous). - * If @offset is non-NULL, update it to the offset of that byte within - * @iov[I] (guaranteed to be less than @iov[I].iov_len) If the whole - * vector has <= @skip bytes, return @n. + * Return: index of iovec array containing the @skip byte counted as if buffers + * were contiguous. If iovec has less than @skip bytes, return @n. */ size_t iov_skip_bytes(const struct iovec *iov, size_t n, size_t skip, size_t *offset) @@ -57,17 +54,14 @@ size_t iov_skip_bytes(const struct iovec *iov, size_t n, } /** - * iov_from_buf() - Copy data from a buffer to an I/O vector (struct iovec) - * efficiently. - * - * @iov: Pointer to the array of struct iovec describing the - * scatter/gather I/O vector. - * @iov_cnt: Number of elements in the iov array. - * @offset: Byte offset in the iov array where copying should start. - * @buf: Pointer to the source buffer containing the data to copy. - * @bytes: Total number of bytes to copy from buf to iov. + * iov_from_buf() - Copy from flat buffer to iovec array + * @iov: Destination iovec array + * @iov_cnt: Number of elements in the iovec array + * @offset: Destination offset in @iov counted as if buffers were contiguous + * @buf: Source buffer + * @bytes: Bytes to copy * - * Return: the number of bytes successfully copied. + * Return: number of bytes copied */ size_t iov_from_buf(const struct iovec *iov, size_t iov_cnt, size_t offset, const void *buf, size_t bytes) @@ -78,12 +72,12 @@ size_t iov_from_buf(const struct iovec *iov, size_t iov_cnt, if (__builtin_constant_p(bytes) && iov_cnt && offset <= iov[0].iov_len && bytes <= iov[0].iov_len - offset) { memcpy((char *)iov[0].iov_base + offset, buf, bytes); + return bytes; } i = iov_skip_bytes(iov, iov_cnt, offset, &offset); - /* copying data */ for (copied = 0; copied < bytes && i < iov_cnt; i++) { size_t len = MIN(iov[i].iov_len - offset, bytes - copied); @@ -97,17 +91,14 @@ size_t iov_from_buf(const struct iovec *iov, size_t iov_cnt, } /** - * iov_to_buf() - Copy data from a scatter/gather I/O vector (struct iovec) to - * a buffer efficiently. - * - * @iov: Pointer to the array of struct iovec describing the scatter/gather - * I/O vector. - * @iov_cnt: Number of elements in the iov array. - * @offset: Offset within the first element of iov from where copying should start. - * @buf: Pointer to the destination buffer where data will be copied. - * @bytes: Total number of bytes to copy from iov to buf. + * iov_to_buf() - Copy from iovec to flat buffer + * @iov: Source iovec array + * @iov_cnt: Number of elements in iovec array + * @offset: Source offset in @iov counted as if buffers were contiguous + * @buf: Destination buffer + * @bytes: Bytes to copy * - * Return: the number of bytes successfully copied. + * Return: number of bytes copied */ size_t iov_to_buf(const struct iovec *iov, size_t iov_cnt, size_t offset, void *buf, size_t bytes) @@ -118,15 +109,17 @@ size_t iov_to_buf(const struct iovec *iov, size_t iov_cnt, if (__builtin_constant_p(bytes) && iov_cnt && offset <= iov[0].iov_len && bytes <= iov[0].iov_len - offset) { memcpy(buf, (char *)iov[0].iov_base + offset, bytes); + return bytes; } i = iov_skip_bytes(iov, iov_cnt, offset, &offset); - /* copying data */ for (copied = 0; copied < bytes && i < iov_cnt; i++) { size_t len = MIN(iov[i].iov_len - offset, bytes - copied); + ASSERT(iov[i].iov_base); + memcpy((char *)buf + copied, (char *)iov[i].iov_base + offset, len); copied += len; @@ -137,14 +130,11 @@ size_t iov_to_buf(const struct iovec *iov, size_t iov_cnt, } /** - * iov_size() - Calculate the total size of a scatter/gather I/O vector - * (struct iovec). + * iov_size() - Calculate total data size of iovec + * @iov: Source iovec array + * @iov_cnt: Number of elements in iovec array * - * @iov: Pointer to the array of struct iovec describing the - * scatter/gather I/O vector. - * @iov_cnt: Number of elements in the iov array. - * - * Return: the total size in bytes. + * Return: total size in bytes */ size_t iov_size(const struct iovec *iov, size_t iov_cnt) { -- 2.43.0

2 1

[PATCH v3 0/5] Pad all inbound frames to 802.3 minimum size if needed
by Stefano Brivio 08 Dec '25

08 Dec '25

Patch 1/5 handles the easy non-batched case with a copy to a padded buffer (only if needed). Patches 2/5 and 3/5 clean coding style up before further changes. Patch 4/5 deals with TCP and UDP batched frames in non-vhost-user modes. Patch 5/5 is for batched frames in vhost-user mode instead. v3: * in 4/5, add calls to tap_hdr_update() from UDP and TCP code to actually update the frame length descriptors after padding * refactor 5/5 entirely based on David's feedback: assume that the first buffer supplied by the guest is at least 60 bytes long, and introduce a vu_pad() common function for both TCP and UDP. Don't recalculate l2len in it, as all the callers already do. v2: * in 1/5, calculate vnet_len *after* padding * in 5/5: - pass the right pointers to memset() - explicitly request at least ETH_ZLEN bytes from vu_collect() Stefano Brivio (5): tap: Pad non-batched frames to 802.3 minimum (60 bytes) if needed tcp: Fix coding style for comment to enum tcp_iov_parts udp: Fix coding style for comment to enum udp_iov_idx tcp, udp: Pad batched frames to 60 bytes (802.3 minimum) in non-vhost-user modes tcp, udp: Pad batched frames for vhost-user modes to 60 bytes (802.3 minimum) tap.c | 11 ++++++++++- tcp.c | 2 +- tcp_buf.c | 23 +++++++++++++++++++++++ tcp_internal.h | 4 +++- tcp_vu.c | 17 ++++++++++++----- udp.c | 43 +++++++++++++++++++++++++++++++++++-------- udp_vu.c | 8 ++++++-- util.c | 3 +++ util.h | 3 +++ vu_common.c | 14 ++++++++++++++ vu_common.h | 1 + 11 files changed, 111 insertions(+), 18 deletions(-) -- 2.43.0

2 7

[PATCH 0/8] tcp: Fix throughput issues with non-local peers
by Stefano Brivio 08 Dec '25

08 Dec '25

Patch 1/8 is the most relevant fix here, as we currently advertise a window that might be too big for what we can write to the socket, causing retransmissions right away and occasional high latency on short transfers to non-local peers. Mostly as a consequence of fixing that, we now need several improvements and small fixes, including, most notably, an adaptive approach to pick the interval between checks for socket-side ACKs (patch 2/8), and several tricks to reliably trigger TCP buffer size auto-tuning as implemented by the Linux kernel (patches 4/8 and 6/8). These changes make some existing issues more relevant, fixed by the other patches. With this series, I'm getting the expected (wirespeed) throughput for transfers between peers with varying non-local RTTs: I checked different guests bridged on the same machine (~600 us) and hosts with increasing distance (approximately 100 to 600 km, ~4 to ~35 ms), using iperf3 as well as HTTP transfers. For short transfers, we strictly stick to the available sending buffer size to (almost) make sure we avoid local retransmissions, and significantly decrease transfer time as a result: from 1.2 s to 60 ms for a 5 MB HTTP transfer from a container hosted in a virtual machine to another guest. Stefano Brivio (8): tcp: Limit advertised window to available, not total sending buffer size tcp: Adaptive interval based on RTT for socket-side acknowledgement checks tcp: Don't clear ACK_TO_TAP_DUE if we're advertising a zero-sized window tcp: Acknowledge everything if sending buffer is less than SNDBUF_BIG tcp: Don't limit window to less-than-MSS values, use zero instead tcp: Allow exceeding the available sending buffer size in window advertisements tcp: Send a duplicate ACK also on complete sendmsg() failure tcp: Skip redundant ACK on partial sendmsg() failure README.md | 2 +- tcp.c | 85 ++++++++++++++++++++++++++++++++++++++++++------------ tcp_conn.h | 9 ++++++ util.c | 14 +++++++++ util.h | 1 + 5 files changed, 92 insertions(+), 19 deletions(-) -- 2.43.0

2 24

[PATCH] iov: Fix coding style of basic (non-IOV_TAIL) parts
by Stefano Brivio 08 Dec '25

08 Dec '25

I happened to touch these functions for a change that turned out to be unnecessary, but coding style and documentation could still benefit from some rephrasing and fixes. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- iov.c | 68 +++++++++++++++++++++++++---------------------------------- 1 file changed, 29 insertions(+), 39 deletions(-) diff --git a/iov.c b/iov.c index 8c74a59..d31464d 100644 --- a/iov.c +++ b/iov.c @@ -20,24 +20,21 @@ * Contributions after 2012-01-13 are licensed under the terms of the * GNU GPL, version 2 or (at your option) any later version. */ + #include <sys/socket.h> #include "util.h" #include "iov.h" - /** - * iov_skip_bytes() - Skip leading bytes of an IO vector - * @iov: IO vector + * iov_skip_bytes() - Find index and offset in iovec array given byte offset + * @iov: iovec array * @n: Number of entries in @iov - * @skip: Number of leading bytes of @iov to skip - * @offset: Offset of first unskipped byte in its @iov entry + * @skip: Byte offset: leading bytes of @iov to skip + * @offset: Offset within matching @iov entry, set on return, can be NULL * - * Return: index I of individual struct iovec which contains the byte at @skip - * bytes into the vector (as though all its buffers were contiguous). - * If @offset is non-NULL, update it to the offset of that byte within - * @iov[I] (guaranteed to be less than @iov[I].iov_len) If the whole - * vector has <= @skip bytes, return @n. + * Return: index of iovec array containing the @skip byte counted as if buffers + * were contiguous. If iovec has less than @skip bytes, return @n. */ size_t iov_skip_bytes(const struct iovec *iov, size_t n, size_t skip, size_t *offset) @@ -57,17 +54,14 @@ size_t iov_skip_bytes(const struct iovec *iov, size_t n, } /** - * iov_from_buf() - Copy data from a buffer to an I/O vector (struct iovec) - * efficiently. - * - * @iov: Pointer to the array of struct iovec describing the - * scatter/gather I/O vector. - * @iov_cnt: Number of elements in the iov array. - * @offset: Byte offset in the iov array where copying should start. - * @buf: Pointer to the source buffer containing the data to copy. - * @bytes: Total number of bytes to copy from buf to iov. + * iov_from_buf() - Copy from flat buffer to iovec array + * @iov: Destination iovec array + * @iov_cnt: Number of elements in the iovec array + * @offset: Destination offset in iovec array + * @buf: Source buffer + * @bytes: Bytes to copy * - * Return: the number of bytes successfully copied. + * Return: number of bytes copied */ size_t iov_from_buf(const struct iovec *iov, size_t iov_cnt, size_t offset, const void *buf, size_t bytes) @@ -78,12 +72,12 @@ size_t iov_from_buf(const struct iovec *iov, size_t iov_cnt, if (__builtin_constant_p(bytes) && iov_cnt && offset <= iov[0].iov_len && bytes <= iov[0].iov_len - offset) { memcpy((char *)iov[0].iov_base + offset, buf, bytes); + return bytes; } i = iov_skip_bytes(iov, iov_cnt, offset, &offset); - /* copying data */ for (copied = 0; copied < bytes && i < iov_cnt; i++) { size_t len = MIN(iov[i].iov_len - offset, bytes - copied); @@ -97,17 +91,14 @@ size_t iov_from_buf(const struct iovec *iov, size_t iov_cnt, } /** - * iov_to_buf() - Copy data from a scatter/gather I/O vector (struct iovec) to - * a buffer efficiently. - * - * @iov: Pointer to the array of struct iovec describing the scatter/gather - * I/O vector. - * @iov_cnt: Number of elements in the iov array. - * @offset: Offset within the first element of iov from where copying should start. - * @buf: Pointer to the destination buffer where data will be copied. - * @bytes: Total number of bytes to copy from iov to buf. + * iov_to_buf() - Copy from iovec to flat buffer + * @iov: Source iovec array + * @iov_cnt: Number of elements in iovec array + * @offset: Source offset in @iov, counted as if buffers were contiguous + * @buf: Destination buffer + * @bytes: Bytes to copy * - * Return: the number of bytes successfully copied. + * Return: number of bytes copied */ size_t iov_to_buf(const struct iovec *iov, size_t iov_cnt, size_t offset, void *buf, size_t bytes) @@ -118,15 +109,17 @@ size_t iov_to_buf(const struct iovec *iov, size_t iov_cnt, if (__builtin_constant_p(bytes) && iov_cnt && offset <= iov[0].iov_len && bytes <= iov[0].iov_len - offset) { memcpy(buf, (char *)iov[0].iov_base + offset, bytes); + return bytes; } i = iov_skip_bytes(iov, iov_cnt, offset, &offset); - /* copying data */ for (copied = 0; copied < bytes && i < iov_cnt; i++) { size_t len = MIN(iov[i].iov_len - offset, bytes - copied); + ASSERT(iov[i].iov_base); + memcpy((char *)buf + copied, (char *)iov[i].iov_base + offset, len); copied += len; @@ -137,14 +130,11 @@ size_t iov_to_buf(const struct iovec *iov, size_t iov_cnt, } /** - * iov_size() - Calculate the total size of a scatter/gather I/O vector - * (struct iovec). + * iov_size() - Calculate total data size of iovec + * @iov: Source iovec array + * @iov_cnt: Number of elements in iovec array * - * @iov: Pointer to the array of struct iovec describing the - * scatter/gather I/O vector. - * @iov_cnt: Number of elements in the iov array. - * - * Return: the total size in bytes. + * Return: total size in bytes */ size_t iov_size(const struct iovec *iov, size_t iov_cnt) { -- 2.43.0

2 2

[PATCH v2] tcp: Enable SO_KEEPALIVE if we see keep-alive segments from container / guest
by Stefano Brivio 06 Dec '25

06 Dec '25

This is an approximation, as enabling SO_KEEPALIVE doesn't mean that a keep-alive segment will be sent right away, rather that keep-alive segments will start being sent if the connection is idle. On the other hand, we don't have direct control over the host-side TCP, so this is probably the best approximation we can get. By default, namespaces inherit keep-alive parameters from their parent namespace, so we can assume that, in case of a container, we'll wait for the same interval it took for the container to start sending us keep-alives, effectively doubling that interval. To keep this simple, set SO_KEEPALIVE whenever we see a keep-alive segment, instead of tracking its state. Keep-alive segments are relatively infrequent, so we don't expect any substantial cost from doing that. Reported-by: Dominic Kohls <Dominic.Wenzler(a)rewe-group.com> Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- v2: Coverity Scan just reminded me that we shouldn't ignore setsockopt() return codes. It doesn't really matter if this fails as we can't do much else in that case, but a trace-level message is a good idea anyway (not that I've ever seen this failing). tcp.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tcp.c b/tcp.c index 8357c0e..d6a5337 100644 --- a/tcp.c +++ b/tcp.c @@ -1838,6 +1838,10 @@ static int tcp_data_from_tap(const struct ctx *c, struct tcp_tap_conn *conn, tcp_send_flag(c, conn, ACK); tcp_timer_ctl(c, conn); + if (setsockopt(conn->sock, SOL_SOCKET, SO_KEEPALIVE, + &((int){ 1 }), sizeof(int))) + flow_trace(conn, "failed to set SO_KEEPALIVE"); + if (p->count == 1) { tcp_tap_window_update(c, conn, ntohs(th->window)); -- 2.43.0

2 1

[PATCH v2 0/5] Pad all inbound frames to 802.3 minimum size if needed
by Stefano Brivio 06 Dec '25

06 Dec '25

Patch 1/5 handles the easy non-batched case with a copy to a padded buffer (only if needed). Patches 2/5 and 3/5 clean coding style up before further changes. Patch 4/5 deals with TCP and UDP batched frames in non-vhost-user modes. Patch 5/5 is for batched frames in vhost-user mode instead. v2: * in 1/5, calculate vnet_len *after* padding * in 5/5: - pass the right pointers to memset() - explicitly request at least ETH_ZLEN bytes from vu_collect() Stefano Brivio (5): tap: Pad non-batched frames to 802.3 minimum (60 bytes) if needed tcp: Fix coding style for comment to enum tcp_iov_parts udp: Fix coding style for comment to enum udp_iov_idx tcp, udp: Pad batched frames to 60 bytes (802.3 minimum) in non-vhost-user modes tcp, udp: Pad batched frames for vhost-user modes to 60 bytes (802.3 minimum) tap.c | 11 ++++++++++- tcp.c | 2 -- tcp_buf.c | 23 +++++++++++++++++++++++ tcp_internal.h | 5 ++++- tcp_vu.c | 32 ++++++++++++++++++++++++++++---- udp.c | 31 ++++++++++++++++++++++++++----- udp_vu.c | 13 ++++++++++++- util.c | 3 +++ util.h | 3 +++ 9 files changed, 109 insertions(+), 14 deletions(-) -- 2.43.0

2 10

PASST/PASST Dynamic Networking Features
by Jon Maloy 05 Dec '25

05 Dec '25

Hi all, Before going further with the implementation, I spent some time to understand the contents and implications of the various bugs/texts we have in this area. I then tried, to the best of my ability, to compile them into user stories, which I think might be a good format for our users to evaluate this. https://pad.passt.top/p/NetlinkMonitor Please have a look and give feedback! ///jon

1 0

[PATCH] conf: Separate local mode for each IP version, don't enable disabled IP version
by Stefano Brivio 05 Dec '25

05 Dec '25

This fixes an issue and introduces a feature: - in local mode, we accidentally ignored the -4 / --ipv4-only and -6 / --ipv6-only command line options - if no template interface is available for a given IP version, instead of disabling that IP version, use local mode, separately, for it Link: https://bugs.passt.top/show_bug.cgi?id=129 Link: https://bugs.passt.top/show_bug.cgi?id=128 Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- conf.c | 19 +++++++++++-------- passt.1 | 15 ++++++++------- 2 files changed, 19 insertions(+), 15 deletions(-) diff --git a/conf.c b/conf.c index 02a4b65..fdc19e8 100644 --- a/conf.c +++ b/conf.c @@ -1109,7 +1109,7 @@ static void conf_print(const struct ctx *c) info("Template interface: %s%s%s%s%s", c->ifi4 > 0 ? if_indextoname(c->ifi4, ifn) : "", c->ifi4 > 0 ? " (IPv4)" : "", - (c->ifi4 && c->ifi6) ? ", " : "", + (c->ifi4 > 0 && c->ifi6 > 0) ? ", " : "", c->ifi6 > 0 ? if_indextoname(c->ifi6, ifn) : "", c->ifi6 > 0 ? " (IPv6)" : ""); } @@ -2001,20 +2001,23 @@ void conf(struct ctx *c, int argc, char **argv) (*c->ip6.ifname_out && !c->ifi6)) die("External interface not usable"); + if (!c->ifi4 && !c->ifi6 && !*c->pasta_ifn) { + strncpy(c->pasta_ifn, pasta_default_ifn, + sizeof(c->pasta_ifn) - 1); + } - if (!c->ifi4 && !c->ifi6) { - info("No external interface as template, switch to local mode"); + if (!c->ifi4 && !v6_only) { + info("IPv4: no external interface as template, use local mode"); conf_ip4_local(&c->ip4); c->ifi4 = -1; + } + + if (!c->ifi6 && !v4_only) { + info("IPv6: no external interface as template, use local mode"); conf_ip6_local(&c->ip6); c->ifi6 = -1; - - if (!*c->pasta_ifn) { - strncpy(c->pasta_ifn, pasta_default_ifn, - sizeof(c->pasta_ifn) - 1); - } } if (c->ifi4 && !no_map_gw && diff --git a/passt.1 b/passt.1 index 0c17454..db0d662 100644 --- a/passt.1 +++ b/passt.1 @@ -166,8 +166,8 @@ By default, assigned IPv4 and IPv6 addresses are taken from the host interfaces with the first default route, if any, for the corresponding IP version. If no default routes are available and there is any interface with any route for a given IP version, the first of these interfaces will be chosen instead. If no -such interface exists, the link-local address 169.254.2.1 is assigned for IPv4, -and no additional address will be assigned for IPv6. +such interface exists for a given IP version, the link-local address 169.254.2.1 +is assigned for IPv4, and no additional address will be assigned for IPv6. .TP .BR \-n ", " \-\-netmask " " \fImask @@ -194,9 +194,9 @@ first default route, if any, for the corresponding IP version. If the default route is a multipath one, the gateway is the first nexthop router returned by the kernel which has the highest weight in the set of paths. If no default routes are available and there is just one interface with any route, that -interface will be chosen instead. If no such interface exists, the link-local -address 169.254.2.2 is used for IPv4, and the link-local address fe80::1 is used -for IPv6. +interface will be chosen instead. If no such interface exists for a given IP +version, the link-local address 169.254.2.2 is used for IPv4, and the link-local +address fe80::1 is used for IPv6. Note: these addresses are also used as source address for packets directed to the guest or to the target namespace having a loopback or local source address, @@ -1117,8 +1117,9 @@ throughput of TCP connections. .SS Local mode for disconnected setups If \fBpasst\fR and \fBpasta\fR fail to find a host interface with a configured -address, other than loopback addresses, they will, obviously, not attempt to -source addresses or routes from the host. +address for a given IP version, other than loopback addresses, they will, +obviously, not attempt to source addresses or routes from the host, for that +IP version. In this case, unless configured otherwise, they will assign the IPv4 link-local address 169.254.2.1 to the guest or target namespace, and no IPv6 address. The -- 2.43.0

3 5

[RFC PATCH 0/5] Pad all inbound frames to 802.3 minimum size if needed
by Stefano Brivio 05 Dec '25

05 Dec '25

Patch 1/5 handles the easy non-batched case with a copy to a padded buffer (only if needed). Patches 2/5 and 3/5 clean coding style up before further changes. Patch 4/5 deals with TCP and UDP batched frames in non-vhost-user modes. Patch 5/5 is for batched frames in vhost-user mode instead, and this is the part I'm the least confident about, in particular with regard to the size assumptions on vhost-user provided buffers. Stefano Brivio (5): tap: Pad non-batched frames to 802.3 minimum (60 bytes) if needed tcp: Fix coding style for comment to enum tcp_iov_parts udp: Fix coding style for comment to enum udp_iov_idx tcp, udp: Pad batched frames to 60 bytes (802.3 minimum) in non-vhost-user modes tcp, udp: Pad batched frames for vhost-user modes to 60 bytes (802.3 minimum) tap.c | 7 +++++++ tcp.c | 2 -- tcp_buf.c | 23 +++++++++++++++++++++++ tcp_internal.h | 5 ++++- tcp_vu.c | 27 +++++++++++++++++++++++++++ udp.c | 31 ++++++++++++++++++++++++++----- udp_vu.c | 11 ++++++++++- util.c | 3 +++ util.h | 3 +++ 9 files changed, 103 insertions(+), 9 deletions(-) -- 2.43.0

3 22