- dev - passt

[PATCH 4/4] tcp: Send TCP keepalive segments after a period of tap-side inactivity
by David Gibson 06 Feb '26

06 Feb '26

There are several circumstances in which a live, but idle TCP connection can be forgotten by a guest, with no "on the wire" indication that this has happened. The most obvious is if the guest abruptly reboots. A more subtle case can happen with a half-closed connection, specifically one in FIN_WAIT_2 state on the guest. A connection can, legitimately, remain in this state indefinitely. If however, a socket in this state is closed by userspace, Linux at least will remove the kernel socket after 60s (or as configured in the net.ipv4.tcp_fin_timeout sysctl). Because there's no on the wire indication in these cases, passt will pointlessly retain the connection in its flow table, at least until it is removed by the inactivity timeout after several hours. To avoid keeping connections around for so long in this state, add functionality to periodically send TCP keepalive segments to the guest if we've seen no activity on the tap interface. If the guest is no longer aware of the connection, it should respond with an RST which will let passt remove the stale entry. To do this we use a method similar to the inactivity timeout - a 1-bit page replacement / clock algorithm, but with a shorter interval, and only checking for tap side activity. Currently we use a 300s interval, meaning we'll send a keepalive after 5-10 minutes of (tap side) inactivity. Link: https://bugs.passt.top/show_bug.cgi?id=179 Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au> --- tcp.c | 39 +++++++++++++++++++++++++++++++++++++++ tcp.h | 2 ++ tcp_conn.h | 2 ++ 3 files changed, 43 insertions(+) diff --git a/tcp.c b/tcp.c index acdac7df..bf57be23 100644 --- a/tcp.c +++ b/tcp.c @@ -206,6 +206,12 @@ * keepalives) will be removed between INACTIVITY_INTERVAL s and * 2*INACTIVITY_INTERVAL s after the last activity. * + * - KEEPALIVE_INTERVAL: if a connection has had no tap-side activity for an + * entire interval, send a tap-side keepalive. If the endpoint is no longer + * aware of the connection (due to a reboot, or a kernel timeout in FIN_WAIT_2 + * state) that should trigger an RST, so we won't keep track of connections + * that the guest endpoint no longer cares about. + * * Summary of data flows (with ESTABLISHED event) * ---------------------------------------------- * @@ -342,6 +348,7 @@ enum { #define RTO_INIT_AFTER_SYN_RETRIES 3 /* s, RFC 6298 */ #define INACTIVITY_INTERVAL 7200 /* s */ +#define KEEPALIVE_INTERVAL 30 /* s */ #define LOW_RTT_TABLE_SIZE 8 #define LOW_RTT_THRESHOLD 10 /* us */ @@ -2263,6 +2270,7 @@ int tcp_tap_handler(const struct ctx *c, uint8_t pif, sa_family_t af, } conn->inactive = false; + conn->tap_inactive = false; if (th->ack && !(conn->events & ESTABLISHED)) tcp_update_seqack_from_tap(c, conn, ntohl(th->ack_seq)); @@ -2884,6 +2892,36 @@ int tcp_init(struct ctx *c) return 0; } +/** + * tcp_keepalive() - Send keepalives for connections which need it + * @: Execution context + */ +static void tcp_keepalive(struct ctx *c, const struct timespec *now) +{ + union flow *flow; + + if (now->tv_sec - c->tcp.keepalive_run < KEEPALIVE_INTERVAL) + return; + + c->tcp.keepalive_run = now->tv_sec; + + flow_foreach(flow) { + struct tcp_tap_conn *conn = &flow->tcp; + + if (flow->f.type != FLOW_TCP) + continue; + + if (conn->tap_inactive) { + flow_dbg(conn, "No tap activity for least %us, send keepalive", + KEEPALIVE_INTERVAL); + tcp_send_flag(c, conn, KEEPALIVE); + } + + /* Ready to check fot next interval */ + conn->tap_inactive = true; + } +} + /** * tcp_inactivity() - Scan for and close long-inactive connections * @: Execution context @@ -2927,6 +2965,7 @@ void tcp_timer(struct ctx *c, const struct timespec *now) if (c->mode == MODE_PASTA) tcp_splice_refill(c); + tcp_keepalive(c, now); tcp_inactivity(c, now); } diff --git a/tcp.h b/tcp.h index e104d453..2739f309 100644 --- a/tcp.h +++ b/tcp.h @@ -38,6 +38,7 @@ extern bool peek_offset_cap; * @rto_max: Maximum retry timeout (in s) * @syn_retries: SYN retries using exponential backoff timeout * @syn_linear_timeouts: SYN retries before using exponential backoff timeout + * @keepalive_run: Time we last issued tap-side keepalives * @inactivity_run: Time we last scanned for inactive connections */ struct tcp_ctx { @@ -48,6 +49,7 @@ struct tcp_ctx { int rto_max; uint8_t syn_retries; uint8_t syn_linear_timeouts; + time_t keepalive_run; time_t inactivity_run; }; diff --git a/tcp_conn.h b/tcp_conn.h index 7197ff63..c82e1441 100644 --- a/tcp_conn.h +++ b/tcp_conn.h @@ -16,6 +16,7 @@ * @ws_from_tap: Window scaling factor advertised from tap/guest * @ws_to_tap: Window scaling factor advertised to tap/guest * @tap_mss: MSS advertised by tap/guest, rounded to 2 ^ TCP_MSS_BITS + * @tapinactive: No tao activity within the current KEEPALIVE_INTERVAL * @inactive: No activity within the current INACTIVITY_INTERVAL * @sock: Socket descriptor number * @events: Connection events, implying connection states @@ -58,6 +59,7 @@ struct tcp_tap_conn { (conn->rtt_exp = MIN(RTT_EXP_MAX, ilog2(MAX(1, rtt / RTT_STORE_MIN)))) #define RTT_GET(conn) (RTT_STORE_MIN << conn->rtt_exp) + bool tap_inactive :1; bool inactive :1; int sock :FD_REF_BITS; -- 2.52.0

2 2

[PATCH 0/4] RFC: Reworks and improvements to TCP activity timers
by David Gibson 06 Feb '26

06 Feb '26

Here's a bunch of patches aimed at fixing bug 179, and reworking the currently broken inactivity timer along the way. I believe patches 1..2/4 are ready to go - I've tested them, and I'm happy with how they're behaving. Patches 3..4/4 I think are correct, but I've been getting bogged down in details trying to test them in the specific FIN_WAIT_2 situation that occurs in bug 179. I'm sending this out for comment, while I look at some other things to clear my head. Caveats: * Currently the inactivity timer uses an interval of 2h to match the intended behaviour of the existig non-working activity timeout. Arguably it should be much longer (several days), like the kernel NAT timeout for idle connection tracking. * The keepalive interval is currently 30s. That's too short - I have it there just for testing. I was intending to change it to 300s (5m) for "real", but that's certainly open to discussion. * This introduces two new fields in the connection structure, as the clock values for the two new timers. These are new 1-bit bool fields slotted into a 3-bit gap. Arguably these would be cleaner as new bits in the 'flags' field. However, since we only have one spare bit there at the moment, that would require some more complex reorganization which I didn't want to tackle right now. David Gibson (4): tcp: Remove non-working activity timeout mechanism tcp: Re-introduce inactivity timeouts based on a clock algorithm tcp: Extend tcp_send_flag() to send TCP keepalive segments tcp: Send TCP keepalive segments after a period of tap-side inactivity tcp.c | 115 ++++++++++++++++++++++++++++++++++++++----------- tcp.h | 6 ++- tcp_buf.c | 4 ++ tcp_conn.h | 5 +++ tcp_internal.h | 2 + tcp_vu.c | 3 ++ 6 files changed, 109 insertions(+), 26 deletions(-) -- 2.52.0

2 4

[PATCH] virtio: Remove duplicate VIRTQUEUE_MAX_SIZE definition
by Laurent Vivier 04 Feb '26

04 Feb '26

VIRTQUEUE_MAX_SIZE is already defined in virtio.h, which virtio.c includes. Remove the redundant local definition. Signed-off-by: Laurent Vivier <lvivier(a)redhat.com> --- virtio.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/virtio.c b/virtio.c index 447137ee83dd..7e31e799f881 100644 --- a/virtio.c +++ b/virtio.c @@ -83,8 +83,6 @@ #include "virtio.h" #include "vhost_user.h" -#define VIRTQUEUE_MAX_SIZE 1024 - /** * vu_gpa_to_va() - Translate guest physical address to our virtual address. * @dev: Vhost-user device -- 2.52.0

2 1

[PATCH 3/4] tcp: Extend tcp_send_flag() to send TCP keepalive segments
by David Gibson 04 Feb '26

04 Feb '26

TCP keepalives aren't technically a flag, but they are a zero-data segment so they can be generated with only a small modification to tcp_{buf,vu}_send_flag(). Implement this, using a new "pseudo-flag" value (similar to DUP_ACK), KEEPALIVE. Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au> --- tcp_buf.c | 4 ++++ tcp_internal.h | 2 ++ tcp_vu.c | 3 +++ 3 files changed, 9 insertions(+) diff --git a/tcp_buf.c b/tcp_buf.c index d2925410..18bdce81 100644 --- a/tcp_buf.c +++ b/tcp_buf.c @@ -227,6 +227,10 @@ int tcp_buf_send_flag(const struct ctx *c, struct tcp_tap_conn *conn, int flags) tcp_frame_conns[tcp_payload_used++] = conn; l4len = optlen + sizeof(struct tcphdr); iov[TCP_IOV_PAYLOAD].iov_len = l4len; + + if (flags & KEEPALIVE) + seq--; + tcp_l2_buf_fill_headers(c, conn, iov, NULL, seq, false); tcp_l2_buf_pad(iov); diff --git a/tcp_internal.h b/tcp_internal.h index 5f8fb358..36f443b4 100644 --- a/tcp_internal.h +++ b/tcp_internal.h @@ -38,6 +38,8 @@ /* Flags for internal usage */ #define DUP_ACK (1 << 5) +#define KEEPALIVE (1 << 6) + #define OPT_EOL 0 #define OPT_NOP 1 #define OPT_MSS 2 diff --git a/tcp_vu.c b/tcp_vu.c index b9e9b55e..f2d38aa5 100644 --- a/tcp_vu.c +++ b/tcp_vu.c @@ -135,6 +135,9 @@ int tcp_vu_send_flag(const struct ctx *c, struct tcp_tap_conn *conn, int flags) flags_elem[0].in_sg[0].iov_len = hdrlen + optlen; payload = IOV_TAIL(flags_elem[0].in_sg, 1, hdrlen); + if (flags & KEEPALIVE) + seq--; + tcp_fill_headers(c, conn, NULL, eh, ip4h, ip6h, th, &payload, NULL, seq, !*c->pcap); -- 2.52.0

1 0

[PATCH v2 0/3] Fix errors in FIN timeout logic
by David Gibson 31 Jan '26

31 Jan '26

While investigating bug 179, I found a number of things that confused me about the TCP timer handling. One of them, I think I figured out what's going on and what should be done about it. So, here are the changes. This is mostly about FIN handling and only tangentially about the timer, but it does at least slightly simplify the timer handling while I figure out the rest of it. Changes in v2: * Stefano pointed out some errors in my guesses at the history of things, evised commit message of 2/3 accordingly * Added 3/3 checking for shutdown(2) failures David Gibson (3): tcp: Retransmit FINs like data segments tcp: Eliminate FIN_TIMEOUT tcp, tcp_splice: Check for failures of shutdown(2) tcp.c | 49 ++++++++++++++++++++++++------------------------- tcp_buf.c | 1 + tcp_splice.c | 3 ++- tcp_vu.c | 1 + 4 files changed, 28 insertions(+), 26 deletions(-) -- 2.52.0

2 2

[PATCH v2 3/3] tcp, tcp_splice: Check for failures of shutdown(2)
by David Gibson 30 Jan '26

30 Jan '26

shutdown(2) should never fail, unless we give it bad parameters (e.g. passing it an fd which isn't a connected socket). However, if it ever did, we'd currently ignore the error and carry on which could lead to very confusing behaviour. In the interests of debugability, check for failure of shutdown(2), log an error and: - during runtime, reset the affected connection - during migration, fail the migration Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au> --- tcp.c | 31 ++++++++++++++++++++++++------- tcp_splice.c | 3 ++- 2 files changed, 26 insertions(+), 8 deletions(-) diff --git a/tcp.c b/tcp.c index 0be871a4..9dd02cd8 100644 --- a/tcp.c +++ b/tcp.c @@ -2284,7 +2284,11 @@ int tcp_tap_handler(const struct ctx *c, uint8_t pif, sa_family_t af, if (th->fin) { conn->seq_from_tap++; - shutdown(conn->sock, SHUT_WR); + if (shutdown(conn->sock, SHUT_WR) < 0) { + flow_dbg_perror(conn, "shutdown() failed"); + goto reset; + } + tcp_send_flag(c, conn, ACK); conn_event(c, conn, SOCK_FIN_SENT); @@ -2359,7 +2363,11 @@ int tcp_tap_handler(const struct ctx *c, uint8_t pif, sa_family_t af, socklen_t sl; struct tcp_info tinfo; - shutdown(conn->sock, SHUT_WR); + if (shutdown(conn->sock, SHUT_WR) < 0) { + flow_dbg_perror(conn, "shutdown() failed"); + goto reset; + } + conn_event(c, conn, SOCK_FIN_SENT); tcp_send_flag(c, conn, ACK); ack_due = 0; @@ -3831,10 +3839,15 @@ int tcp_flow_migrate_target_ext(struct ctx *c, struct tcp_tap_conn *conn, int fd int v; v = TCP_SEND_QUEUE; - if (setsockopt(s, SOL_TCP, TCP_REPAIR_QUEUE, &v, sizeof(v))) + if (setsockopt(s, SOL_TCP, TCP_REPAIR_QUEUE, &v, sizeof(v))) { flow_perror(conn, "Selecting repair queue"); - else - shutdown(s, SHUT_WR); + } else { + if (shutdown(s, SHUT_WR) < 0) { + flow_perror(conn, + "Repair mode shutdown() failed"); + goto fail; + } + } } if (tcp_flow_repair_wnd(conn, &t)) @@ -3861,8 +3874,12 @@ int tcp_flow_migrate_target_ext(struct ctx *c, struct tcp_tap_conn *conn, int fd * Call shutdown(x, SHUT_WR) *not* in repair mode, which moves us to * TCP_FIN_WAIT1. */ - if (t.tcpi_state == TCP_FIN_WAIT1) - shutdown(s, SHUT_WR); + if (t.tcpi_state == TCP_FIN_WAIT1) { + if (shutdown(s, SHUT_WR) < 0) { + flow_perror(conn, "Post-repair shutdown() failed"); + goto fail; + } + } if (tcp_set_peek_offset(conn, peek_offset)) goto fail; diff --git a/tcp_splice.c b/tcp_splice.c index 8806523a..d60981ca 100644 --- a/tcp_splice.c +++ b/tcp_splice.c @@ -627,7 +627,8 @@ retry: flow_foreach_sidei(sidei) { if ((conn->events & FIN_RCVD(sidei)) && !(conn->events & FIN_SENT(!sidei))) { - shutdown(conn->s[!sidei], SHUT_WR); + if (shutdown(conn->s[!sidei], SHUT_WR) < 0) + goto reset; conn_event(conn, FIN_SENT(!sidei)); } } -- 2.52.0

1 0

[PATCH v2 1/3] tcp: Retransmit FINs like data segments
by David Gibson 30 Jan '26

30 Jan '26

RFC 9293 doesn't distinguish between regular data segments and FIN segments for the purposes of retransmissions. Our existing retransmission logic will also work for FIN segments, except for one detail: we don't currently set the ACK_FROM_TAP_DUE flag when we send a FIN. Add the flag, so that we'll properly retransmit FIN segments like data segments. Remove the section from the theory of operation comment that describes a different way of handling FIN timeouts which (a) isn't correct behaviour and (b) doesn't appear to be implemented. I've tested this by adding logic to suppress sending the FIN if retries < some non-zero value. We correctly resend the FIN and close normally after the expected timeouts. Link: https://bugs.passt.top/show_bug.cgi?id=195 Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au> --- tcp.c | 4 ---- tcp_buf.c | 1 + tcp_vu.c | 1 + 3 files changed, 2 insertions(+), 4 deletions(-) diff --git a/tcp.c b/tcp.c index 17e5b006..dbfde2e0 100644 --- a/tcp.c +++ b/tcp.c @@ -190,10 +190,6 @@ * - RTO_INIT_AFTER_SYN_RETRIES: if SYN retries happened during handshake and * RTO is less than this, re-initialise RTO to this for data retransmissions * - * - FIN_TIMEOUT: if a FIN segment was sent to tap/guest (flag ACK_FROM_TAP_DUE - * with TAP_FIN_SENT event), and no ACK is received within this time, reset - * the connection - * * - FIN_TIMEOUT: if a FIN segment was acknowledged by tap/guest and a FIN * segment (write shutdown) was sent via socket (events SOCK_FIN_SENT and * TAP_FIN_ACKED), but no socket activity is detected from the socket within diff --git a/tcp_buf.c b/tcp_buf.c index 5d419d36..d2925410 100644 --- a/tcp_buf.c +++ b/tcp_buf.c @@ -407,6 +407,7 @@ int tcp_buf_data_from_sock(const struct ctx *c, struct tcp_tap_conn *conn) } conn_event(c, conn, TAP_FIN_SENT); + conn_flag(c, conn, ACK_FROM_TAP_DUE); } return 0; diff --git a/tcp_vu.c b/tcp_vu.c index db9db78a..b9e9b55e 100644 --- a/tcp_vu.c +++ b/tcp_vu.c @@ -425,6 +425,7 @@ int tcp_vu_data_from_sock(const struct ctx *c, struct tcp_tap_conn *conn) } conn_event(c, conn, TAP_FIN_SENT); + conn_flag(c, conn, ACK_FROM_TAP_DUE); } return 0; -- 2.52.0

1 0

[PATCH 0/2] Fix errors in FIN timeout logic
by David Gibson 30 Jan '26

30 Jan '26

While investigating bug 179, I found a number of things that confused me about the TCP timer handling. One of them, I think I figured out what's going on and what should be done about it. So, here are the changes. This is mostly about FIN handling and only tangentially about the timer, but it does at least slightly simplify the timer handling while I figure out the rest of it. David Gibson (2): tcp: Retransmit FINs like data segments tcp: Eliminate FIN_TIMEOUT tcp.c | 18 ------------------ tcp_buf.c | 1 + tcp_vu.c | 1 + 3 files changed, 2 insertions(+), 18 deletions(-) -- 2.52.0

2 4

Thoughts on interface modes / multiple guest addresses
by David Gibson 29 Jan '26

29 Jan '26

Hi Jon, As discussed on the call yesterday, I've written up my thoughts on what a bunch of the address semantics should be. Turns out I'd already done some of this at: https://pad.passt.top/p/InterfaceMode I've now updated to cover some more things, and considering the possibility of multiple guest addresses.. Turns out etherpad doesn't really do tables, so it's two sections for the two suggested modes, with matching subheadings. It's still very much a draft so: * It might be unclear - please ask me to clarify anything * It will have some mistakes - make your case if you think my proposed semantics don't make sense * It will be incomplete - feel free to add extra subheadings for anything you -- David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson

3 15

[PATCH 0/3] Fix bugs with RST propagation
by David Gibson 28 Jan '26

28 Jan '26

In various circumstances we may not properly report abrupt ends to a TCP connection to peers with an RST. Fix them. Link: https://bugs.passt.top/show_bug.cgi?id=191 Link: https://bugs.passt.top/show_bug.cgi?id=193 David Gibson (3): doc: Add test program verifying socket RST behaviour tcp: Properly propagate tap-side RST to socket side tcp_splice: Force TCP RST on abnormal close conditions doc/platform-requirements/.gitignore | 1 + doc/platform-requirements/Makefile | 8 +- doc/platform-requirements/tcp-close-rst.c | 204 ++++++++++++++++++++++ tcp.c | 37 +++- tcp_splice.c | 58 ++++-- 5 files changed, 290 insertions(+), 18 deletions(-) create mode 100644 doc/platform-requirements/tcp-close-rst.c -- 2.52.0

2 5