- dev - passt

[PATCH] doc: Add mock of migration source and target
by Stefano Brivio 04 Feb '25

04 Feb '25

These test programs show the migration of a TCP connection using the passt-repair helper. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- Note: I'm merging this right now, but posting anyway to ease review. doc/migration/.gitignore | 2 + doc/migration/Makefile | 20 ++++++++ doc/migration/README | 51 ++++++++++++++++++++ doc/migration/source.c | 92 +++++++++++++++++++++++++++++++++++ doc/migration/target.c | 102 +++++++++++++++++++++++++++++++++++++++ 5 files changed, 267 insertions(+) create mode 100644 doc/migration/.gitignore create mode 100644 doc/migration/Makefile create mode 100644 doc/migration/README create mode 100644 doc/migration/source.c create mode 100644 doc/migration/target.c diff --git a/doc/migration/.gitignore b/doc/migration/.gitignore new file mode 100644 index 0000000..59cb765 --- /dev/null +++ b/doc/migration/.gitignore @@ -0,0 +1,2 @@ +/source +/target diff --git a/doc/migration/Makefile b/doc/migration/Makefile new file mode 100644 index 0000000..04f6891 --- /dev/null +++ b/doc/migration/Makefile @@ -0,0 +1,20 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# +# PASST - Plug A Simple Socket Transport +# for qemu/UNIX domain socket mode +# +# PASTA - Pack A Subtle Tap Abstraction +# for network namespace/tap device mode +# +# Copyright (c) 2025 Red Hat GmbH +# Author: Stefano Brivio <sbrivio(a)redhat.com> + +TARGETS = source target +CFLAGS = -Wall -Wextra -pedantic + +all: $(TARGETS) + +$(TARGETS): %: %.c + +clean: + rm -f $(TARGETS) diff --git a/doc/migration/README b/doc/migration/README new file mode 100644 index 0000000..375603b --- /dev/null +++ b/doc/migration/README @@ -0,0 +1,51 @@ + + +Migration +========= + +These test programs show a migration of a TCP connection from one process to +another using the TCP_REPAIR socket option. + +The two processes are a mock of the matching implementation in passt(1), and run +unprivileged, so they rely on the passt-repair helper to connect to them and set +or clear TCP_REPAIR on the connection socket, transferred to the helper using +SCM_RIGHTS. + +The passt-repair helper needs to have the CAP_NET_ADMIN capability, or run as +root. + +Example of usage +---------------- + +* Start the test server + + $ nc -l 9999 + +* Start the source side of the TCP client (mock of the source instance of passt) + + $ ./source 127.0.0.1 9999 9998 /tmp/repair.sock + +* The client sends a test string, and waits for a connection from passt-repair + + # passt-repair /tmp/repair.sock + +* The socket is now in repair mode, and `source` dumps sequences, then exits + + sending sequence: 3244673313 + receiving sequence: 2250449386 + +* Continue the connection on the target side, restarting from those sequences + + $ ./target 127.0.0.1 9999 9998 /tmp/repair.sock 3244673313 2250449386 + +* The target side now waits for a connection from passt-repair + + # passt-repair /tmp/repair.sock + +* The target side asks passt-repair to switch the socket to repair mode, sets up + the TCP sequences, then asks passt-repair to clear repair mode, and sends a + test string to the server diff --git a/doc/migration/source.c b/doc/migration/source.c new file mode 100644 index 0000000..d44ebf1 --- /dev/null +++ b/doc/migration/source.c @@ -0,0 +1,92 @@ +// SPDX-License-Identifier: GPL-2.0-or-later + +/* PASST - Plug A Simple Socket Transport + * for qemu/UNIX domain socket mode + * + * PASTA - Pack A Subtle Tap Abstraction + * for network namespace/tap device mode + * + * doc/migration/source.c - Mock of TCP migration source, use with passt-repair + * + * Copyright (c) 2025 Red Hat GmbH + * Author: Stefano Brivio <sbrivio(a)redhat.com> + */ + +#include <arpa/inet.h> +#include <sys/types.h> +#include <sys/socket.h> +#include <sys/un.h> +#include <stdint.h> +#include <stdio.h> +#include <stdlib.h> +#include <limits.h> +#include <unistd.h> +#include <netdb.h> +#include <netinet/tcp.h> + +int main(int argc, char **argv) +{ + struct sockaddr_in a = { AF_INET, htons(atoi(argv[3])), { 0 }, { 0 } }; + struct addrinfo hints = { 0, AF_UNSPEC, SOCK_STREAM, 0, 0, + NULL, NULL, NULL }; + struct sockaddr_un a_helper = { AF_UNIX, { 0 } }; + int seq, s, s_helper; + int8_t cmd; + struct iovec iov = { &cmd, sizeof(cmd) }; + char buf[CMSG_SPACE(sizeof(int))]; + struct msghdr msg = { NULL, 0, &iov, 1, buf, sizeof(buf), 0 }; + struct cmsghdr *cmsg = CMSG_FIRSTHDR(&msg); + socklen_t seqlen = sizeof(int); + struct addrinfo *r; + + (void)argc; + + if (argc != 5) { + fprintf(stderr, "%s DST_ADDR DST_PORT SRC_PORT HELPER_PATH\n", + argv[0]); + return -1; + } + + strcpy(a_helper.sun_path, argv[4]); + getaddrinfo(argv[1], argv[2], &hints, &r); + + /* Connect socket to server and send some data */ + s = socket(r->ai_family, SOCK_STREAM, IPPROTO_TCP); + setsockopt(s, SOL_SOCKET, SO_REUSEADDR, &((int){ 1 }), sizeof(int)); + bind(s, (struct sockaddr *)&a, sizeof(a)); + connect(s, r->ai_addr, r->ai_addrlen); + send(s, "before migration\n", sizeof("before migration\n"), 0); + + /* Wait for helper */ + s_helper = socket(AF_UNIX, SOCK_STREAM, 0); + unlink(a_helper.sun_path); + bind(s_helper, (struct sockaddr *)&a_helper, sizeof(a_helper)); + listen(s_helper, 1); + s_helper = accept(s_helper, NULL, NULL); + + /* Set up message for helper, with socket */ + cmsg->cmsg_level = SOL_SOCKET; + cmsg->cmsg_type = SCM_RIGHTS; + cmsg->cmsg_len = CMSG_LEN(sizeof(int)); + memcpy(CMSG_DATA(cmsg), &s, sizeof(s)); + + /* Send command to helper: turn repair mode on, wait for reply */ + cmd = TCP_REPAIR_ON; + sendmsg(s_helper, &msg, 0); + recv(s_helper, &((int8_t){ 0 }), 1, 0); + + /* Terminate helper */ + close(s_helper); + + /* Get sending sequence */ + seq = TCP_SEND_QUEUE; + setsockopt(s, SOL_TCP, TCP_REPAIR_QUEUE, &seq, sizeof(seq)); + getsockopt(s, SOL_TCP, TCP_QUEUE_SEQ, &seq, &seqlen); + fprintf(stdout, "%u ", seq); + + /* Get receiving sequence */ + seq = TCP_RECV_QUEUE; + setsockopt(s, SOL_TCP, TCP_REPAIR_QUEUE, &seq, sizeof(seq)); + getsockopt(s, SOL_TCP, TCP_QUEUE_SEQ, &seq, &seqlen); + fprintf(stdout, "%u\n", seq); +} diff --git a/doc/migration/target.c b/doc/migration/target.c new file mode 100644 index 0000000..f7d3108 --- /dev/null +++ b/doc/migration/target.c @@ -0,0 +1,102 @@ +// SPDX-License-Identifier: GPL-2.0-or-later + +/* PASST - Plug A Simple Socket Transport + * for qemu/UNIX domain socket mode + * + * PASTA - Pack A Subtle Tap Abstraction + * for network namespace/tap device mode + * + * doc/migration/target.c - Mock of TCP migration target, use with passt-repair + * + * Copyright (c) 2025 Red Hat GmbH + * Author: Stefano Brivio <sbrivio(a)redhat.com> + */ + +#include <arpa/inet.h> +#include <sys/types.h> +#include <sys/socket.h> +#include <sys/un.h> +#include <stdint.h> +#include <stdio.h> +#include <stdlib.h> +#include <limits.h> +#include <unistd.h> +#include <netdb.h> +#include <netinet/tcp.h> + +int main(int argc, char **argv) +{ + struct sockaddr_in a = { AF_INET, htons(atoi(argv[3])), { 0 }, { 0 } }; + struct addrinfo hints = { 0, AF_UNSPEC, SOCK_STREAM, 0, 0, + NULL, NULL, NULL }; + struct sockaddr_un a_helper = { AF_UNIX, { 0 } }; + int s, s_helper, seq; + int8_t cmd; + struct iovec iov = { &cmd, sizeof(cmd) }; + char buf[CMSG_SPACE(sizeof(int))]; + struct msghdr msg = { NULL, 0, &iov, 1, buf, sizeof(buf), 0 }; + struct cmsghdr *cmsg = CMSG_FIRSTHDR(&msg); + struct addrinfo *r; + + (void)argc; + + strcpy(a_helper.sun_path, argv[4]); + getaddrinfo(argv[1], argv[2], &hints, &r); + + if (argc != 7) { + fprintf(stderr, + "%s DST_ADDR DST_PORT SRC_PORT HELPER_PATH SSEQ RSEQ\n", + argv[0]); + return -1; + } + + /* Prepare socket, bind to source port */ + s = socket(r->ai_family, SOCK_STREAM, IPPROTO_TCP); + setsockopt(s, SOL_SOCKET, SO_REUSEADDR, &((int){ 1 }), sizeof(int)); + bind(s, (struct sockaddr *)&a, sizeof(a)); + + /* Wait for helper */ + s_helper = socket(AF_UNIX, SOCK_STREAM, 0); + unlink(a_helper.sun_path); + bind(s_helper, (struct sockaddr *)&a_helper, sizeof(a_helper)); + listen(s_helper, 1); + s_helper = accept(s_helper, NULL, NULL); + + /* Set up message for helper, with socket */ + cmsg->cmsg_level = SOL_SOCKET; + cmsg->cmsg_type = SCM_RIGHTS; + cmsg->cmsg_len = CMSG_LEN(sizeof(int)); + memcpy(CMSG_DATA(cmsg), &s, sizeof(s)); + + /* Send command to helper: turn repair mode on, wait for reply */ + cmd = TCP_REPAIR_ON; + sendmsg(s_helper, &msg, 0); + recv(s_helper, &((int){ 0 }), 1, 0); + + /* Set sending sequence */ + seq = TCP_SEND_QUEUE; + setsockopt(s, SOL_TCP, TCP_REPAIR_QUEUE, &seq, sizeof(seq)); + seq = atoi(argv[5]); + setsockopt(s, SOL_TCP, TCP_QUEUE_SEQ, &seq, sizeof(seq)); + + /* Set receiving sequence */ + seq = TCP_RECV_QUEUE; + setsockopt(s, SOL_TCP, TCP_REPAIR_QUEUE, &seq, sizeof(seq)); + seq = atoi(argv[6]); + setsockopt(s, SOL_TCP, TCP_QUEUE_SEQ, &seq, sizeof(seq)); + + /* Connect setting kernel state only, without actual SYN / handshake */ + connect(s, r->ai_addr, r->ai_addrlen); + + /* Send command to helper: turn repair mode off, wait for reply */ + cmd = TCP_REPAIR_OFF; + sendmsg(s_helper, &msg, 0); + + recv(s_helper, &((int8_t){ 0 }), 1, 0); + + /* Terminate helper */ + close(s_helper); + + /* Send some more data */ + send(s, "after migration\n", sizeof("after migration\n"), 0); +} -- 2.43.0

1 0

[PATCH] tcp: Get socket port and address using getsockname() when connecting from guest
by Stefano Brivio 04 Feb '25

04 Feb '25

For migration only: we need to store 'oport', our socket-side port, as we establish a connection from the guest, so that we can bind the same oport as source port in the migration target. Similar for 'oaddr': this is needed in case the migration target has additional network interfaces, and we need to make sure our socket is bound to the equivalent interface as it was on the source. Use getsockname() to fetch them. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- Note: I'm merging this right now, but posting anyway to ease review. flow.c | 4 ++-- flow_table.h | 4 ++-- tcp.c | 26 +++++++++++++++++++++++++- 3 files changed, 29 insertions(+), 5 deletions(-) diff --git a/flow.c b/flow.c index ee1221b..a6fe6d1 100644 --- a/flow.c +++ b/flow.c @@ -414,8 +414,8 @@ const struct flowside *flow_initiate_sa(union flow *flow, uint8_t pif, * * Return: pointer to the target flowside information */ -const struct flowside *flow_target(const struct ctx *c, union flow *flow, - uint8_t proto) +struct flowside *flow_target(const struct ctx *c, union flow *flow, + uint8_t proto) { char estr[INANY_ADDRSTRLEN], fstr[INANY_ADDRSTRLEN]; struct flow_common *f = &flow->f; diff --git a/flow_table.h b/flow_table.h index f15db53..eeb6f41 100644 --- a/flow_table.h +++ b/flow_table.h @@ -168,8 +168,8 @@ const struct flowside *flow_target_af(union flow *flow, uint8_t pif, sa_family_t af, const void *saddr, in_port_t sport, const void *daddr, in_port_t dport); -const struct flowside *flow_target(const struct ctx *c, union flow *flow, - uint8_t proto); +struct flowside *flow_target(const struct ctx *c, union flow *flow, + uint8_t proto); union flow *flow_set_type(union flow *flow, enum flow_type type); #define FLOW_SET_TYPE(flow_, t_, var_) (&flow_set_type((flow_), (t_))->var_) diff --git a/tcp.c b/tcp.c index 51ad692..cf2943d 100644 --- a/tcp.c +++ b/tcp.c @@ -1415,6 +1415,8 @@ static void tcp_bind_outbound(const struct ctx *c, * @opts: Pointer to start of options * @optlen: Bytes in options: caller MUST ensure available length * @now: Current timestamp + * + * #syscalls:vu getsockname */ static void tcp_conn_from_tap(const struct ctx *c, sa_family_t af, const void *saddr, const void *daddr, @@ -1423,9 +1425,10 @@ static void tcp_conn_from_tap(const struct ctx *c, sa_family_t af, { in_port_t srcport = ntohs(th->source); in_port_t dstport = ntohs(th->dest); - const struct flowside *ini, *tgt; + const struct flowside *ini; struct tcp_tap_conn *conn; union sockaddr_inany sa; + struct flowside *tgt; union flow *flow; int s = -1, mss; uint64_t hash; @@ -1530,6 +1533,27 @@ static void tcp_conn_from_tap(const struct ctx *c, sa_family_t af, } tcp_epoll_ctl(c, conn); + + if (c->mode == MODE_VU) { /* To rebind to same oport after migration */ + if (af == AF_INET) { + struct sockaddr_in s_in; + socklen_t sl; + + sl = sizeof(s_in); + getsockname(s, (struct sockaddr *)&s_in, &sl); + tgt->oport = ntohs(s_in.sin_port); + tgt->oaddr = inany_from_v4(s_in.sin_addr); + } else { + struct sockaddr_in6 s_in6; + socklen_t sl; + + sl = sizeof(s_in6); + getsockname(s, (struct sockaddr *)&s_in6, &sl); + tgt->oport = ntohs(s_in6.sin6_port); + tgt->oaddr.a6 = s_in6.sin6_addr; + } + } + FLOW_ACTIVATE(conn); return; -- 2.43.0

1 0

[PATCH 0/7] Draft, incomplete series introducing state migration
by Stefano Brivio 03 Feb '25

03 Feb '25

This is obviously incomplete. I have code on top of this, not really working yet, with a loop on transferred flows, and an implementation matching passt-repair, requesting to enable/disable the TCP_REPAIR option as needed, as well as setting/receiving sequences. I'm sending this for early review/rework/rewrite/whatever. What's here should all be tested and working. Adding: { &flow_first_free, sizeof(flow_first_free) }, { flowtab, sizeof(flowtab) }, to data version 1 in 6/7 will properly transfer those sections. Declaring functions and assigning pointers such as: { flow_migrate_source_pre, NULL }, { flow_migrate_source_post, NULL }, { flow_migrate_target_post_v1, NULL }, also executes them. The passt-repair helper in 7/7 is (lightly) tested against a stand-alone source/target implementation which I'll share in a bit. Stefano Brivio (7): icmp, udp: Pad time_t timestamp to 64-bit to ease state migration flow, flow_table: Pad flow table entries to 128 bytes, hash entries to 32 bits tcp_conn: Avoid 7-bit hole in struct tcp_splice_conn flow_table: Use size in extern declaration for flowtab util: Add read_remainder() and read_all_buf() Introduce facilities for guest migration on top of vhost-user infrastructure Introduce passt-repair Makefile | 22 +++-- flow.h | 18 ++-- flow_table.h | 15 ++- icmp_flow.h | 6 +- migrate.c | 259 +++++++++++++++++++++++++++++++++++++++++++++++++ migrate.h | 90 +++++++++++++++++ passt-repair.c | 111 +++++++++++++++++++++ passt.c | 2 +- tcp_conn.h | 2 +- udp_flow.h | 6 +- util.c | 70 +++++++++++++ util.h | 2 + vu_common.c | 122 +++++++++++++++-------- vu_common.h | 2 +- 14 files changed, 662 insertions(+), 65 deletions(-) create mode 100644 migrate.c create mode 100644 migrate.h create mode 100644 passt-repair.c -- 2.43.0

2 52

[PATCH v3 00/20] Draft, incomplete series introducing state migration
by Stefano Brivio 03 Feb '25

03 Feb '25

...and finally connections survive migration from source to target, at least the ones originating from the (source) guest. I didn't try the other way around, small tweaks might be needed. Tested as follows, roughly as instructed by Laurent: Source: $ ./passt --vhost-user $ qemu-system-x86_64 -machine accel=kvm -cpu host -kernel ... \ -initrd mbuto.img -nographic -serial mon:stdio -nodefaults \ -append "console=ttyS0" \ -chardev socket,id=chr0,path=/tmp/passt_1.socket \ -netdev vhost-user,id=netdev0,chardev=chr0 \ -device virtio-net,netdev=netdev0 \ -object memory-backend-memfd,id=memfd0,share=on,size=$((2 * 1024 * 1024 * 1024)) \ -numa node,memdev=memfd0 -m 2G # ./passt-repair /tmp/passt_1.socket.repair Target (same host): $ ./passt --vhost-user $ qemu-system-x86_64 -machine accel=kvm -cpu host -kernel ... \ -initrd mbuto.img -nographic -serial mon:stdio -nodefaults \ -append "console=ttyS0" \ -chardev socket,id=chr0,path=/tmp/passt_2.socket \ -netdev vhost-user,id=netdev0,chardev=chr0 \ -device virtio-net,netdev=netdev0 \ -object memory-backend-memfd,id=memfd0,share=on,size=$((2 * 1024 * 1024 * 1024)) \ -numa node,memdev=memfd0 -m 2G \ -incoming tcp:0:4444 # ./passt-repair /tmp/passt_2.socket.repair Test server: $ nc -l 9091 Once the guest boots: # ip link set dev eth0 up # dhclient eth0 # socat STDIN TCP:$DEFAULT_GW:9091 abcd ^a-c migrate tcp:0:4444 Then continue typing in the target guest: efgh The purpose of this is mostly to show the complete flow, but it needs a number of reworks. What's missing (letting aside pending packet queues for a moment, those are not strictly needed): 1. tests based on the two_guests layout/setup. Even with reverse-search in the shell, this is getting quite hard on wrists. I guess we can start QEMU with -monitor unix:mon.sock,server,nowait and send the 'migrate' command via socat STDIN UNIX-CONNECT:mon.sock 2. dump and transfer of *socket-side* MSS and window scale (I used hardcoded values): this needs more storage, so it needs to be transferred outside the flow table 3. dump, transfer and restore of TCP_REPAIR_WINDOW parameters (not strictly needed, but easy to add once we have appropriate storage) 4. perhaps some small bits of implementation for socket-originated connections (I tested only guest-originated ones so far) 5. UDP and ICMP flows (ping already happens to "survive" nicely, by the way) 6. man page for passt-repair, and man page changes for everything 7. packaging and Linux Security Module changes for passt-repair 8. error handling here and there, and repair rollback/migration abort 9. setting original receive/send buffer sizes and socket options (TCP_NODELAY) What clearly needs changes: a. we can't dump more stuff to the flow table, because we would exceed 128 bytes. We need to copy everything from tcp_tap_conn except for: - state in flow_common - in_epoll - sock - timer and on top of this we need: - values for TCPOPT_WINDOW and TCPOPT_MAXSEG - struct tcp_repair_window somewhat unexpectedly, this is actually bigger than a flow table entry. In any case, we need to implement a stream/per-entry migration right away. b. at this point, I guess we can throw the header away, and just keep a magic (0xB1BB1D1B0BB1D1B0 has a missing 0 at the end but, well, https://en.wikipedia.org/wiki/Bibbidi-Bobbidi-Boo is the Magic Song: can we keep it?) and a version number. The rest, let's go with big/network endianness I'd say, and 64-bit time_t c. the declarative data thing is very convenient but we need to fetch stuff from struct ctx, as shown by the hash_secret example. What's very convenient of this approach is the iovec / writev() / readv() idea. I'm not sure if we can maintain that convenience, though Patches that could be applied regardless of this series to make it more manageable: 1/20 tcp: Always pass NULL event with EPOLL_CTL_DEL 2/20 util: Rename and make global vu_remove_watch() 6/20 util: Add read_remainder() and read_all_buf() 8/20 Introduce passt-repair 16/20 vhost_user: Turn vhost-user message reports to trace() 17/20 vhost_user: Make source quit after reporting migration state 18/20 tcp: Get our socket port using getsockname() when connecting from guest 19/20 tcp: Add HOSTSIDE(x), HOSTFLOW(x) macros Patches that we can throw away with the changes outlined above: 3/20 icmp, udp: Pad time_t timestamp to 64-bit to ease state migration 4/20 flow, flow_table: Pad flow table entries to 128 bytes, hash entries to 32 bits 15/20 flow, flow_table: Export declaration of hash table David Gibson (6): tcp: Always pass NULL event with EPOLL_CTL_DEL util: Rename and make global vu_remove_watch() migrate: vu_migrate_{source,target}() aren't actually vu speciic migrate: Move repair_sock_init() to vu_init() migrate: Make more handling common rather than vhost-user specific migrate: Don't handle the migration channel through epoll Stefano Brivio (14): icmp, udp: Pad time_t timestamp to 64-bit to ease state migration flow, flow_table: Pad flow table entries to 128 bytes, hash entries to 32 bits flow_table: Use size in extern declaration for flowtab util: Add read_remainder() and read_all_buf() Introduce facilities for guest migration on top of vhost-user infrastructure Introduce passt-repair Add interfaces and configuration bits for passt-repair flow, tcp: Basic pre-migration source handler to dump sequence numbers flow, flow_table: Export declaration of hash table vhost_user: Turn vhost-user message reports to trace() vhost_user: Make source quit after reporting migration state tcp: Get our socket port using getsockname() when connecting from guest tcp: Add HOSTSIDE(x), HOSTFLOW(x) macros Implement target side of migration .gitignore | 1 + Makefile | 24 +-- conf.c | 44 +++++- epoll_type.h | 6 +- flow.c | 97 +++++++++++- flow.h | 20 ++- flow_table.h | 22 ++- icmp.c | 2 +- icmp_flow.h | 6 +- migrate.c | 408 +++++++++++++++++++++++++++++++++++++++++++++++++ migrate.h | 84 ++++++++++ passt-repair.c | 117 ++++++++++++++ passt.1 | 11 ++ passt.c | 17 ++- passt.h | 17 +++ repair.c | 193 +++++++++++++++++++++++ repair.h | 16 ++ tap.c | 64 +------- tcp.c | 198 +++++++++++++++++++++++- tcp_conn.h | 7 + tcp_internal.h | 10 +- tcp_splice.c | 4 +- udp_flow.c | 2 +- udp_flow.h | 6 +- util.c | 155 +++++++++++++++++++ util.h | 4 + vhost_user.c | 94 +++--------- virtio.h | 4 - vu_common.c | 62 +++----- vu_common.h | 2 +- 30 files changed, 1469 insertions(+), 228 deletions(-) create mode 100644 migrate.c create mode 100644 migrate.h create mode 100644 passt-repair.c create mode 100644 repair.c create mode 100644 repair.h -- 2.43.0

2 40

[PATCH] tcp: Don't reset outbound connection on SYN retries
by Stefano Brivio 03 Feb '25

03 Feb '25

Reported by somebody on IRC: if the server has considerable latency, it might happen that the client retries sending SYN segments for the same flow while we're still in a TAP_SYN_RCVD, non-ESTABLISHED state. In that case, we should go with the blanket assumption that we need to reset the connection on any unexpected segment: RFC 9293 explicitly mentions this case in Figure 8: Recovery from Old Duplicate SYN, section 3.5. It doesn't make sense for us to set a specific sequence number, socket-side, but we should definitely wait and see. Ignoring the duplicate SYN segment should also be compatible with section 3.10.7.3. SYN-SENT STATE, which mentions updating sequences socket-side (which we can't do anyway), but certainly not reset the connection. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- tcp.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tcp.c b/tcp.c index 7787381..51ad692 100644 --- a/tcp.c +++ b/tcp.c @@ -1920,6 +1920,9 @@ int tcp_tap_handler(const struct ctx *c, uint8_t pif, sa_family_t af, /* Establishing connection from tap */ if (conn->events & TAP_SYN_RCVD) { + if (th->syn && !th->ack && !th->fin) + return 1; /* SYN retry: ignore and keep waiting */ + if (!(conn->events & TAP_SYN_ACK_SENT)) goto reset; -- 2.43.0

2 1

[PATCH] tcp_splice, udp_flow: fcntl64() support on PPC64 depends on glibc version
by Stefano Brivio 03 Feb '25

03 Feb '25

I explicitly added fcntl64() to the list of allowed system calls for PPC64 a while ago, and now it turns out it's not available in recent Debian builds. The warning from seccomp.sh is harmless because we unconditionally try to enable fcntl() anyway, but take care of it anyway. Link: https://buildd.debian.org/status/fetch.php?pkg=passt&arch=ppc64&ver=0.0%7Eg… Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- tcp_splice.c | 2 +- udp_flow.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tcp_splice.c b/tcp_splice.c index 5db1d62..f048a82 100644 --- a/tcp_splice.c +++ b/tcp_splice.c @@ -28,7 +28,7 @@ * - FIN_SENT_0: FIN (write shutdown) sent to accepted socket * - FIN_SENT_1: FIN (write shutdown) sent to target socket * - * #syscalls:pasta pipe2|pipe fcntl arm:fcntl64 ppc64:fcntl64 i686:fcntl64 + * #syscalls:pasta pipe2|pipe fcntl arm:fcntl64 ppc64:fcntl64|fcntl i686:fcntl64 */ #include <sched.h> diff --git a/udp_flow.c b/udp_flow.c index 7fae81d..83c2568 100644 --- a/udp_flow.c +++ b/udp_flow.c @@ -174,7 +174,7 @@ cancel: * @s_in: Source socket address, filled in by recvmmsg() * @now: Timestamp * - * #syscalls fcntl arm:fcntl64 ppc64:fcntl64 i686:fcntl64 + * #syscalls fcntl arm:fcntl64 ppc64:fcntl64|fcntl i686:fcntl64 * * Return: sidx for the destination side of the flow for this packet, or * FLOW_SIDX_NONE if we couldn't find or create a flow. -- 2.43.0

2 1

[PATCH] vhost_user: On 32-bit ARM, mmap() is not available, mmap2() is used instead
by Stefano Brivio 03 Feb '25

03 Feb '25

Link: https://buildd.debian.org/status/fetch.php?pkg=passt&arch=armel&ver=0.0%7Eg… Link: https://buildd.debian.org/status/fetch.php?pkg=passt&arch=armhf&ver=0.0%7Eg… Fixes: 31117b27c6c9 ("vhost-user: introduce vhost-user API") Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- vhost_user.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/vhost_user.c b/vhost_user.c index 19ede8a..f9a9eb3 100644 --- a/vhost_user.c +++ b/vhost_user.c @@ -419,7 +419,7 @@ static bool map_ring(struct vu_dev *vdev, struct vu_virtq *vq) * * Return: False as no reply is requested * - * #syscalls:vu mmap munmap + * #syscalls:vu mmap|mmap2 munmap */ static bool vu_set_mem_table_exec(struct vu_dev *vdev, struct vhost_user_msg *msg) -- 2.43.0

2 1

[PATCH 0/2] Trivial cleanups to EPOLL_CTL_DEL handling
by David Gibson 03 Feb '25

03 Feb '25

2 3

[PATCH 0/2] Fancier version handling for migration
by David Gibson 31 Jan '25

31 Jan '25

This series introduces more flexible version handling for our device state migration stream. This is based on my previous assorted migration improvements series. David Gibson (2): migrate: Merge protocol version information into one table migrate: More sophisticated versioning migrate.c | 213 +++++++++++++++++++++++++++++++++++------------------- migrate.h | 73 +++++++++++-------- 2 files changed, 179 insertions(+), 107 deletions(-) -- 2.48.1

2 6

[net,v3] tcp: correct handling of extreme memory squeeze
by jmaloy＠redhat.com 30 Jan '25

30 Jan '25

From: Jon Maloy <jmaloy(a)redhat.com> Testing with iperf3 using the "pasta" protocol splicer has revealed a bug in the way tcp handles window advertising in extreme memory squeeze situations. Under memory pressure, a socket endpoint may temporarily advertise a zero-sized window, but this is not stored as part of the socket data. The reasoning behind this is that it is considered a temporary setting which shouldn't influence any further calculations. However, if we happen to stall at an unfortunate value of the current window size, the algorithm selecting a new value will consistently fail to advertise a non-zero window once we have freed up enough memory. This means that this side's notion of the current window size is different from the one last advertised to the peer, causing the latter to not send any data to resolve the sitution. The problem occurs on the iperf3 server side, and the socket in question is a completely regular socket with the default settings for the fedora40 kernel. We do not use SO_PEEK or SO_RCVBUF on the socket. The following excerpt of a logging session, with own comments added, shows more in detail what is happening: // tcp_v4_rcv(->) // tcp_rcv_established(->) [5201<->39222]: ==== Activating log @ net/ipv4/tcp_input.c/tcp_data_queue()/5257 ==== [5201<->39222]: tcp_data_queue(->) [5201<->39222]: DROPPING skb [265600160..265665640], reason: SKB_DROP_REASON_PROTO_MEM [rcv_nxt 265600160, rcv_wnd 262144, snt_ack 265469200, win_now 131184] [copied_seq 259909392->260034360 (124968), unread 5565800, qlen 85, ofoq 0] [OFO queue: gap: 65480, len: 0] [5201<->39222]: tcp_data_queue(<-) [5201<->39222]: __tcp_transmit_skb(->) [tp->rcv_wup: 265469200, tp->rcv_wnd: 262144, tp->rcv_nxt 265600160] [5201<->39222]: tcp_select_window(->) [5201<->39222]: (inet_csk(sk)->icsk_ack.pending & ICSK_ACK_NOMEM) ? --> TRUE [tp->rcv_wup: 265469200, tp->rcv_wnd: 262144, tp->rcv_nxt 265600160] returning 0 [5201<->39222]: tcp_select_window(<-) [5201<->39222]: ADVERTISING WIN 0, ACK_SEQ: 265600160 [5201<->39222]: [__tcp_transmit_skb(<-) [5201<->39222]: tcp_rcv_established(<-) [5201<->39222]: tcp_v4_rcv(<-) // Receive queue is at 85 buffers and we are out of memory. // We drop the incoming buffer, although it is in sequence, and decide // to send an advertisement with a window of zero. // We don't update tp->rcv_wnd and tp->rcv_wup accordingly, which means // we unconditionally shrink the window. [5201<->39222]: tcp_recvmsg_locked(->) [5201<->39222]: __tcp_cleanup_rbuf(->) tp->rcv_wup: 265469200, tp->rcv_wnd: 262144, tp->rcv_nxt 265600160 [5201<->39222]: [new_win = 0, win_now = 131184, 2 * win_now = 262368] [5201<->39222]: [new_win >= (2 * win_now) ? --> time_to_ack = 0] [5201<->39222]: NOT calling tcp_send_ack() [tp->rcv_wup: 265469200, tp->rcv_wnd: 262144, tp->rcv_nxt 265600160] [5201<->39222]: __tcp_cleanup_rbuf(<-) [rcv_nxt 265600160, rcv_wnd 262144, snt_ack 265469200, win_now 131184] [copied_seq 260040464->260040464 (0), unread 5559696, qlen 85, ofoq 0] returning 6104 bytes [5201<->39222]: tcp_recvmsg_locked(<-) // After each read, the algorithm for calculating the new receive // window in __tcp_cleanup_rbuf() finds it is too small to advertise // or to update tp->rcv_wnd. // Meanwhile, the peer thinks the window is zero, and will not send // any more data to trigger an update from the interrupt mode side. [5201<->39222]: tcp_recvmsg_locked(->) [5201<->39222]: __tcp_cleanup_rbuf(->) tp->rcv_wup: 265469200, tp->rcv_wnd: 262144, tp->rcv_nxt 265600160 [5201<->39222]: [new_win = 262144, win_now = 131184, 2 * win_now = 262368] [5201<->39222]: [new_win >= (2 * win_now) ? --> time_to_ack = 0] [5201<->39222]: NOT calling tcp_send_ack() [tp->rcv_wup: 265469200, tp->rcv_wnd: 262144, tp->rcv_nxt 265600160] [5201<->39222]: __tcp_cleanup_rbuf(<-) [rcv_nxt 265600160, rcv_wnd 262144, snt_ack 265469200, win_now 131184] [copied_seq 260099840->260171536 (71696), unread 5428624, qlen 83, ofoq 0] returning 131072 bytes [5201<->39222]: tcp_recvmsg_locked(<-) // The above pattern repeats again and again, since nothing changes // between the reads. [...] [5201<->39222]: tcp_recvmsg_locked(->) [5201<->39222]: __tcp_cleanup_rbuf(->) tp->rcv_wup: 265469200, tp->rcv_wnd: 262144, tp->rcv_nxt 265600160 [5201<->39222]: [new_win = 262144, win_now = 131184, 2 * win_now = 262368] [5201<->39222]: [new_win >= (2 * win_now) ? --> time_to_ack = 0] [5201<->39222]: NOT calling tcp_send_ack() [tp->rcv_wup: 265469200, tp->rcv_wnd: 262144, tp->rcv_nxt 265600160] [5201<->39222]: __tcp_cleanup_rbuf(<-) [rcv_nxt 265600160, rcv_wnd 262144, snt_ack 265469200, win_now 131184] [copied_seq 265600160->265600160 (0), unread 0, qlen 0, ofoq 0] returning 54672 bytes [5201<->39222]: tcp_recvmsg_locked(<-) // The receive queue is empty, but no new advertisement has been sent. // The peer still thinks the receive window is zero, and sends nothing. // We have ended up in a deadlock situation. Furthermore, we have observed that in these situations this side may send out an updated 'th->ack_seq´ which is not stored in tp->rcv_wup as it should be. Backing ack_seq seems to be harmless, but is of course still wrong from a protocol viewpoint. We fix this by updating the socket state correctly when a packet has been dropped because of memory exhaustion and we have to advertize a zero window. Further testing shows that the connection recovers neatly from the squeeze situation, and traffic can continue indefinitely. Fixes: e2142825c120 ("net: tcp: send zero-window ACK when no memory") Cc: Menglong Dong <menglong8.dong(a)gmail.com> Reviewed-by: Stefano Brivio <sbrivio(a)redhat.com> Signed-off-by: Jon Maloy <jmaloy(a)redhat.com> --- v1: -Posted on Apr 6, 2024: https://lore.kernel.org/all/20240406182107.261472-1-jmaloy@redhat.com/#r v2: -Improved commit log to clarify how we end up in this situation. -After feedback from Eric Dumazet, removed references to use of SO_PEEK and SO_PEEK_OFF which may lead to a misunderstanding about how this situation occurs. Those flags are used at the peer side's incoming connection, and not on this one. v3: -Clearing tp->pred_flags when we do a zero advertisement. -Edited (manually) and shortened the log documenting the problem. --- net/ipv4/tcp_output.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index 0e5b9a654254..bc95d2a5924f 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -265,11 +265,14 @@ static u16 tcp_select_window(struct sock *sk) u32 cur_win, new_win; /* Make the window 0 if we failed to queue the data because we - * are out of memory. The window is temporary, so we don't store - * it on the socket. + * are out of memory. */ - if (unlikely(inet_csk(sk)->icsk_ack.pending & ICSK_ACK_NOMEM)) + if (unlikely(inet_csk(sk)->icsk_ack.pending & ICSK_ACK_NOMEM)) { + tp->pred_flags = 0; + tp->rcv_wnd = 0; + tp->rcv_wup = tp->rcv_nxt; return 0; + } cur_win = tcp_receive_window(tp); new_win = __tcp_select_window(sk); -- 2.48.0

7 8