- dev - passt

[PATCH] tcp: Pass union tcp_conn pointer to destroy and splice timer functions
by Stefano Brivio 21 Nov '22

21 Nov '22

The pointers are actually the same, but we later pass the container union to tcp_table_compact(), which might zero the size of the whole union, and this confuses Coverity Scan. Given that we have pointers to the container union to start with, just pass those instead, all the way down to tcp_table_compact(). Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- tcp.c | 17 +++++++++-------- tcp_conn.h | 4 ++-- tcp_splice.c | 16 ++++++++++------ 3 files changed, 21 … [View More]insertions(+), 16 deletions(-) diff --git a/tcp.c b/tcp.c index 8874789..ff12dfa 100644 --- a/tcp.c +++ b/tcp.c @@ -1376,16 +1376,18 @@ void tcp_table_compact(struct ctx *c, union tcp_conn *hole) /** * tcp_conn_destroy() - Close sockets, trigger hash table removal and compaction * @c: Execution context - * @conn: Connection pointer + * @conn_union: Connection pointer (container union) */ -static void tcp_conn_destroy(struct ctx *c, struct tcp_tap_conn *conn) +static void tcp_conn_destroy(struct ctx *c, union tcp_conn *conn_union) { + struct tcp_tap_conn *conn = &conn_union->tap; + close(conn->sock); if (conn->timer != -1) close(conn->timer); tcp_hash_remove(c, conn); - tcp_table_compact(c, (union tcp_conn *)conn); + tcp_table_compact(c, conn_union); } static void tcp_rst_do(struct ctx *c, struct tcp_tap_conn *conn); @@ -1535,13 +1537,12 @@ void tcp_defer_handler(struct ctx *c) for (conn = tc + c->tcp.conn_count - 1; conn >= tc; conn--) { if (conn->c.spliced) { if (conn->splice.flags & CLOSING) - tcp_splice_destroy(c, &conn->splice); + tcp_splice_destroy(c, conn); } else { if (conn->tap.events == CLOSED) - tcp_conn_destroy(c, &conn->tap); + tcp_conn_destroy(c, conn); } } - } /** @@ -3395,10 +3396,10 @@ void tcp_timer(struct ctx *c, const struct timespec *ts) for (conn = tc + c->tcp.conn_count - 1; conn >= tc; conn--) { if (conn->c.spliced) { - tcp_splice_timer(c, &conn->splice); + tcp_splice_timer(c, conn); } else { if (conn->tap.events == CLOSED) - tcp_conn_destroy(c, &conn->tap); + tcp_conn_destroy(c, conn); } } diff --git a/tcp_conn.h b/tcp_conn.h index 4a8be29..a146276 100644 --- a/tcp_conn.h +++ b/tcp_conn.h @@ -181,8 +181,8 @@ extern union tcp_conn tc[]; void tcp_splice_conn_update(struct ctx *c, struct tcp_splice_conn *new); void tcp_table_compact(struct ctx *c, union tcp_conn *hole); -void tcp_splice_destroy(struct ctx *c, struct tcp_splice_conn *conn); -void tcp_splice_timer(struct ctx *c, struct tcp_splice_conn *conn); +void tcp_splice_destroy(struct ctx *c, union tcp_conn *conn_union); +void tcp_splice_timer(struct ctx *c, union tcp_conn *conn_union); void tcp_splice_pipe_refill(const struct ctx *c); diff --git a/tcp_splice.c b/tcp_splice.c index e2f0ce1..72b1672 100644 --- a/tcp_splice.c +++ b/tcp_splice.c @@ -251,10 +251,12 @@ void tcp_splice_conn_update(struct ctx *c, struct tcp_splice_conn *new) /** * tcp_splice_destroy() - Close spliced connection and pipes, clear * @c: Execution context - * @conn: Connection pointer + * @conn_union: Spliced connection (container union) */ -void tcp_splice_destroy(struct ctx *c, struct tcp_splice_conn *conn) +void tcp_splice_destroy(struct ctx *c, union tcp_conn *conn_union) { + struct tcp_splice_conn *conn = &conn_union->splice; + if (conn->events & SPLICE_ESTABLISHED) { /* Flushing might need to block: don't recycle them. */ if (conn->pipe_a_b[0] != -1) { @@ -283,7 +285,7 @@ void tcp_splice_destroy(struct ctx *c, struct tcp_splice_conn *conn) debug("TCP (spliced): index %li, CLOSED", CONN_IDX(conn)); c->tcp.splice_conn_count--; - tcp_table_compact(c, (union tcp_conn *)conn); + tcp_table_compact(c, conn_union); } /** @@ -824,12 +826,14 @@ void tcp_splice_init(struct ctx *c) /** * tcp_splice_timer() - Timer for spliced connections * @c: Execution context - * @conn: Spliced connection + * @conn_union: Spliced connection (container union) */ -void tcp_splice_timer(struct ctx *c, struct tcp_splice_conn *conn) +void tcp_splice_timer(struct ctx *c, union tcp_conn *conn_union) { + struct tcp_splice_conn *conn = &conn_union->splice; + if (conn->flags & CLOSING) { - tcp_splice_destroy(c, conn); + tcp_splice_destroy(c, conn_union); return; } -- 2.35.1 [View Less]

2 1

UDP "splicing" fairly broken :(
by David Gibson 18 Nov '22

18 Nov '22

In preparation for trying to implement dual stack sockets for UDP, I've been getting my head around how the UDP splicing works. Alas, I'm pretty sure that it's broken if there's not a one-to-one correspondence between init side source ports and ns side destination ports. That will typically be the case, but given its UDP there's no guarantee. In addition, UDP servers in the ns will not see the correct port numbers with getpeername(). That's also true of spliced TCP connections (see https://… [View More]

2 4

[PATCH] tcp: Explicit bound check on options length field
by Stefano Brivio 18 Nov '22

18 Nov '22

For some reason, Coverity only reports this (harmless) warning after David's series unifying IPv4 and IPv6 sockets for TCP: an untrusted loop bound (CWE-606) in tcp_opt_get(), coming from the fact that we use indeed the value of a TCP header field as loop bound. Note, though, that the loop already checks we're not exceeding the length of the option field, and this field is used as 8-bit unsigned value, so we can't really look for options past the end of the header. In any case, make Coverity … [View More]

1 0

[PATCH passt 0/5] Add fuzzing
by Richard W.M. Jones 17 Nov '22

17 Nov '22

(Note this patch series does not work so far and needs some help, read on ...) Patches 1 & 2 are general cleanup. The rest of the patches aim to add fuzzing support for Passt using AFL++, Clang and ASAN. I used the same approach as with libnbd: https://gitlab.com/nbdkit/libnbd/-/tree/master/fuzzing Firstly (patch 3) I added an --fd option. This is useful for fuzzing, but also generally useful. It allows a controller process to open a connected stream socket and pass that down to … [View More]

2 16

[PATCH 00/32] Use dual stack sockets to listen for inbound TCP connections
by David Gibson 17 Nov '22

17 Nov '22

When forwarding many ports, passt can consume a lot of kernel memory because of the many listening sockets it opens. There are not a lot of ways we can reduce that, but here's one. Currently we create separate listening sockets for each port for both IPv4 and IPv6. However in Linux (and probably other platforms), it's possible to listen for both IPv4 and IPv6 connections on an IPv6 socket. This series uses such dual stack sockets to halve the number of listening sockets needed for TCP. … [View More]When forwarding all TCP and UDP ports, this reduces the kernel memory used from around 677 MiB to around 487 MiB (kernel 6.0.8 on an x86_64 Fedora 37 machine). This should also be possible for UDP, but that will require a mostly separate implementation. David Gibson (32): clang-tidy: Suppress warning about assignments in if statements style: Minor corrections to function comments tcp_splice: #include tcp_splice.h in tcp_splice.c tcp: Remove unused TCP_MAX_SOCKS constant tcp: Better helpers for converting between connection pointer and index tcp_splice: Helpers for converting from index to/from tcp_splice_conn tcp: Move connection state structures into a shared header tcp: Add connection union type tcp: Improved helpers to update connections after moving tcp: Unify spliced and non-spliced connection tables tcp: Unify tcp_defer_handler and tcp_splice_defer_handler() tcp: Partially unify tcp_timer() and tcp_splice_timer() tcp: Unify the IN_EPOLL flag tcp: Separate helpers to create ns listening sockets tcp: Unify part of spliced and non-spliced conn_from_sock path tcp: Use the same sockets to listen for spliced and non-spliced connections tcp: Remove splice from tcp_epoll_ref tcp: Don't store hash bucket in connection structures inany: Helper functions for handling addresses which could be IPv4 or IPv6 tcp: Hash IPv4 and IPv4-mapped-IPv6 addresses the same tcp: Take tcp_hash_insert() address from struct tcp_conn tcp: Simplify tcp_hash_match() to take an inany_addr tcp: Unify initial sequence number calculation for IPv4 and IPv6 tcp: Have tcp_seq_init() take its parameters from struct tcp_conn tcp: Fix small errors in tcp_seq_init() time handling tcp: Remove v6 flag from tcp_epoll_ref tcp: NAT IPv4-mapped IPv6 addresses like IPv4 addresses tcp_splice: Allow splicing of connections from IPv4-mapped loopback tcp: Consolidate tcp_sock_init[46] util: Allow sock_l4() to open dual stack sockets util: Always return -1 on error in sock_l4() tcp: Use dual stack sockets for port forwarding when possible Makefile | 11 +- conf.c | 12 +- inany.h | 94 +++++ siphash.c | 2 + tap.c | 6 +- tcp.c | 978 ++++++++++++++++++++++----------------------------- tcp.h | 11 +- tcp_conn.h | 192 ++++++++++ tcp_splice.c | 337 ++++++++---------- tcp_splice.h | 12 +- util.c | 19 +- 11 files changed, 887 insertions(+), 787 deletions(-) create mode 100644 inany.h create mode 100644 tcp_conn.h -- 2.38.1 [View Less]

2 56

[PATCH] test/lib/test: Clean up iperf3 JSON files before starting the server
by Stefano Brivio 17 Nov '22

17 Nov '22

...instead of doing it after the test. Now that we have pre-built guest images, we might also have old JSON files from previous, interrupted test runs. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- test/lib/test | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/test/lib/test b/test/lib/test index df32178..29612c2 100755 --- a/test/lib/test +++ b/test/lib/test @@ -31,6 +31,8 @@ test_iperf3() { __procs="$((${1} - 1))"; shift __time="${1}"; shift + … [View More]

2 2

[PATCH v3] util, pasta: Add do_clone() wrapper around __clone2() and clone()
by Stefano Brivio 17 Nov '22

17 Nov '22

Spotted in Debian's buildd logs: on ia64, clone(2) is not available: the glibc wrapper is named __clone2() and it takes, additionally, the size of the stack area passed by the caller. Add a do_clone() wrapper handling the different cases, and also taking care of pointing the child's stack in the middle of the allocated area: on PA-RISC (hppa), handled by clone(), the stack grows up, and on ia64 the stack grows down, but the register backing store grows up -- and I think it might be actually … [View More]used here. Suggested-by: David Gibson <david(a)gibson.dropbear.id.au> Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- v3: Don't call it __clone(), it conflicts with glibc on static builds v2: Add a wrapper instead of two __ia64__ ifdefs pasta.c | 10 +++++----- util.c | 21 +++++++++++++++++++++ util.h | 8 +++++--- 3 files changed, 31 insertions(+), 8 deletions(-) diff --git a/pasta.c b/pasta.c index db86317..528f02a 100644 --- a/pasta.c +++ b/pasta.c @@ -226,11 +226,11 @@ void pasta_start_ns(struct ctx *c, uid_t uid, gid_t gid, arg.argv = sh_argv; } - pasta_child_pid = clone(pasta_spawn_cmd, - ns_fn_stack + sizeof(ns_fn_stack) / 2, - CLONE_NEWIPC | CLONE_NEWPID | CLONE_NEWNET | - CLONE_NEWUTS, - (void *)&arg); + pasta_child_pid = do_clone(pasta_spawn_cmd, ns_fn_stack, + sizeof(ns_fn_stack), + CLONE_NEWIPC | CLONE_NEWPID | CLONE_NEWNET | + CLONE_NEWUTS, + (void *)&arg); if (pasta_child_pid == -1) { perror("clone"); diff --git a/util.c b/util.c index be102e3..2125a67 100644 --- a/util.c +++ b/util.c @@ -482,3 +482,24 @@ int write_file(const char *path, const char *buf) close(fd); return len == 0 ? 0 : -1; } + +/** + * do_clone() - Wrapper of __clone2() for ia64, clone() for other architectures + * @fn: Entry point for child + * @stack_area: Stack area for child: we'll point callees to the middle of it + * @stack_size: Total size of stack area, passed to callee divided by two + * @flags: clone() system call flags + * @arg: Argument to @fn + * + * Return: thread ID of child, -1 on failure + */ +int do_clone(int (*fn)(void *), char *stack_area, size_t stack_size, int flags, + void *arg) +{ +#ifdef __ia64__ + return __clone2(fn, stack_area + stack_size / 2, stack_size / 2, + flags, arg); +#else + return clone(fn, stack_area + stack_size / 2, flags, arg); +#endif +} diff --git a/util.h b/util.h index 2d4e1ff..02b55a9 100644 --- a/util.h +++ b/util.h @@ -81,13 +81,15 @@ (((struct in_addr *)(a))->s_addr == ((struct in_addr *)b)->s_addr) #define NS_FN_STACK_SIZE (RLIMIT_STACK_VAL * 1024 / 8) +int do_clone(int (*fn)(void *), char *stack_area, size_t stack_size, int flags, + void *arg); #define NS_CALL(fn, arg) \ do { \ char ns_fn_stack[NS_FN_STACK_SIZE]; \ \ - clone((fn), ns_fn_stack + sizeof(ns_fn_stack) / 2, \ - CLONE_VM | CLONE_VFORK | CLONE_FILES | SIGCHLD, \ - (void *)(arg)); \ + do_clone((fn), ns_fn_stack, sizeof(ns_fn_stack), \ + CLONE_VM | CLONE_VFORK | CLONE_FILES | SIGCHLD,\ + (void *)(arg)); \ } while (0) #if __BYTE_ORDER == __BIG_ENDIAN -- 2.35.1 [View Less]

2 2

vsock regression
by David Gibson 17 Nov '22

17 Nov '22

A word of warning, if you try to run the passt tests on a recent qemu git build. A change to default options causes vsock to break. You can work around it by adding a `queue_reset=false` option to the vsock device, or reverting to qemu commit 3c37f8b8d1e5 or earlier. I've written up the details here: https://gitlab.com/qemu-project/qemu/-/issues/1318 Possibly I should also ping some people on qemu-devel. -- David Gibson | I'll have my music baroque, and my code david AT gibson.… [View More]

1 0

passt: new version 2022_11_16.ace074c available
by Stefano Brivio 17 Nov '22

17 Nov '22

The new version with tag 2022_11_16.ace074c includes the following changes: ace074c README: The upcoming version of Podman adds support for pasta ab6f825 util, pasta: Add do_clone() wrapper around __clone2() and clone() 7d8c2fb test/lib/test: Clean up iperf3 JSON files before starting the server 25dab96 tap: Revert recently added checks in tap_handler_passt() b27d6d1 arp, tap, util: Don't use perror() after seccomp filter is installed 5f74465 Remove contrib/debian, … [View More]

1 0

[PATCH] README: The upcoming version of Podman adds support for pasta
by Stefano Brivio 16 Nov '22

16 Nov '22

Don't update the demo yet: we could just wait until Podman 4.3.2 is out and packaged. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- README.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 3d0a5b4..8947702 100644 --- a/README.md +++ b/README.md @@ -330,10 +330,9 @@ speeding up local connections, and usually requiring NAT. _pasta_: * ✅ qemu, libvirt support with [`qrap` wrapper](/passt/tree/qrap.c) * ✅ out-of-tree … [View More]

1 0