- dev - passt

[net-next v4] tcp: add support for SO_PEEK_OFF socket option
by jmaloy＠redhat.com 12 Apr '24

12 Apr '24

From: Jon Maloy <jmaloy(a)redhat.com> When reading received messages from a socket with MSG_PEEK, we may want to read the contents with an offset, like we can do with pread/preadv() when reading files. Currently, it is not possible to do that. In this commit, we add support for the SO_PEEK_OFF socket option for TCP, in a similar way it is done for Unix Domain sockets. In the iperf3 log examples shown below, we can observe a throughput improvement of 15-20 % in the direction host->… [View More]namespace when using the protocol splicer 'pasta' (https://passt.top). This is a consistent result. pasta(1) and passt(1) implement user-mode networking for network namespaces (containers) and virtual machines by means of a translation layer between Layer-2 network interface and native Layer-4 sockets (TCP, UDP, ICMP/ICMPv6 echo). Received, pending TCP data to the container/guest is kept in kernel buffers until acknowledged, so the tool routinely needs to fetch new data from socket, skipping data that was already sent. At the moment this is implemented using a dummy buffer passed to recvmsg(). With this change, we don't need a dummy buffer and the related buffer copy (copy_to_user()) anymore. passt and pasta are supported in KubeVirt and libvirt/qemu. jmaloy@freyr:~/passt$ perf record -g ./pasta --config-net -f SO_PEEK_OFF not supported by kernel. jmaloy@freyr:~/passt# iperf3 -s ----------------------------------------------------------- Server listening on 5201 (test #1) ----------------------------------------------------------- Accepted connection from 192.168.122.1, port 44822 [ 5] local 192.168.122.180 port 5201 connected to 192.168.122.1 port 44832 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 1.02 GBytes 8.78 Gbits/sec [ 5] 1.00-2.00 sec 1.06 GBytes 9.08 Gbits/sec [ 5] 2.00-3.00 sec 1.07 GBytes 9.15 Gbits/sec [ 5] 3.00-4.00 sec 1.10 GBytes 9.46 Gbits/sec [ 5] 4.00-5.00 sec 1.03 GBytes 8.85 Gbits/sec [ 5] 5.00-6.00 sec 1.10 GBytes 9.44 Gbits/sec [ 5] 6.00-7.00 sec 1.11 GBytes 9.56 Gbits/sec [ 5] 7.00-8.00 sec 1.07 GBytes 9.20 Gbits/sec [ 5] 8.00-9.00 sec 667 MBytes 5.59 Gbits/sec [ 5] 9.00-10.00 sec 1.03 GBytes 8.83 Gbits/sec [ 5] 10.00-10.04 sec 30.1 MBytes 6.36 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.04 sec 10.3 GBytes 8.78 Gbits/sec receiver ----------------------------------------------------------- Server listening on 5201 (test #2) ----------------------------------------------------------- ^Ciperf3: interrupt - the server has terminated jmaloy@freyr:~/passt# logout [ perf record: Woken up 23 times to write data ] [ perf record: Captured and wrote 5.696 MB perf.data (35580 samples) ] jmaloy@freyr:~/passt$ jmaloy@freyr:~/passt$ perf record -g ./pasta --config-net -f SO_PEEK_OFF supported by kernel. jmaloy@freyr:~/passt# iperf3 -s ----------------------------------------------------------- Server listening on 5201 (test #1) ----------------------------------------------------------- Accepted connection from 192.168.122.1, port 52084 [ 5] local 192.168.122.180 port 5201 connected to 192.168.122.1 port 52098 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 1.32 GBytes 11.3 Gbits/sec [ 5] 1.00-2.00 sec 1.19 GBytes 10.2 Gbits/sec [ 5] 2.00-3.00 sec 1.26 GBytes 10.8 Gbits/sec [ 5] 3.00-4.00 sec 1.36 GBytes 11.7 Gbits/sec [ 5] 4.00-5.00 sec 1.33 GBytes 11.4 Gbits/sec [ 5] 5.00-6.00 sec 1.21 GBytes 10.4 Gbits/sec [ 5] 6.00-7.00 sec 1.31 GBytes 11.2 Gbits/sec [ 5] 7.00-8.00 sec 1.25 GBytes 10.7 Gbits/sec [ 5] 8.00-9.00 sec 1.33 GBytes 11.5 Gbits/sec [ 5] 9.00-10.00 sec 1.24 GBytes 10.7 Gbits/sec [ 5] 10.00-10.04 sec 56.0 MBytes 12.1 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.04 sec 12.9 GBytes 11.0 Gbits/sec receiver ----------------------------------------------------------- Server listening on 5201 (test #2) ----------------------------------------------------------- ^Ciperf3: interrupt - the server has terminated logout [ perf record: Woken up 20 times to write data ] [ perf record: Captured and wrote 5.040 MB perf.data (33411 samples) ] jmaloy@freyr:~/passt$ The perf record confirms this result. Below, we can observe that the CPU spends significantly less time in the function ____sys_recvmsg() when we have offset support. Without offset support: ---------------------- jmaloy@freyr:~/passt$ perf report -q --symbol-filter=do_syscall_64 \ -p ____sys_recvmsg -x --stdio -i perf.data | head -1 46.32% 0.00% passt.avx2 [kernel.vmlinux] [k] do_syscall_64 ____sys_recvmsg With offset support: ---------------------- jmaloy@freyr:~/passt$ perf report -q --symbol-filter=do_syscall_64 \ -p ____sys_recvmsg -x --stdio -i perf.data | head -1 28.12% 0.00% passt.avx2 [kernel.vmlinux] [k] do_syscall_64 ____sys_recvmsg Suggested-by: Paolo Abeni <pabeni(a)redhat.com> Reviewed-by: Stefano Brivio <sbrivio(a)redhat.com> Signed-off-by: Jon Maloy <jmaloy(a)redhat.com> --- v3: - Applied changes suggested by Stefano Brivio and Paolo Abeni v4: - Same as v3. Posting was delayed because I first had to debug an issue that turned out to not be directly related to this change. Signed-off-by: Jon Maloy <jmaloy(a)redhat.com> --- net/ipv4/af_inet.c | 1 + net/ipv4/tcp.c | 16 ++++++++++------ 2 files changed, 11 insertions(+), 6 deletions(-) diff --git a/net/ipv4/af_inet.c b/net/ipv4/af_inet.c index 55bd72997b31..a7cfeda28bb2 100644 --- a/net/ipv4/af_inet.c +++ b/net/ipv4/af_inet.c @@ -1072,6 +1072,7 @@ const struct proto_ops inet_stream_ops = { #endif .splice_eof = inet_splice_eof, .splice_read = tcp_splice_read, + .set_peek_off = sk_set_peek_off, .read_sock = tcp_read_sock, .read_skb = tcp_read_skb, .sendmsg_locked = tcp_sendmsg_locked, diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 92ee60492314..c0d6fd576d32 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -1416,8 +1416,6 @@ static int tcp_peek_sndq(struct sock *sk, struct msghdr *msg, int len) struct sk_buff *skb; int copied = 0, err = 0; - /* XXX -- need to support SO_PEEK_OFF */ - skb_rbtree_walk(skb, &sk->tcp_rtx_queue) { err = skb_copy_datagram_msg(skb, 0, msg, skb->len); if (err) @@ -2328,6 +2326,7 @@ static int tcp_recvmsg_locked(struct sock *sk, struct msghdr *msg, size_t len, int target; /* Read at least this many bytes */ long timeo; struct sk_buff *skb, *last; + u32 peek_offset = 0; u32 urg_hole = 0; err = -ENOTCONN; @@ -2361,7 +2360,8 @@ static int tcp_recvmsg_locked(struct sock *sk, struct msghdr *msg, size_t len, seq = &tp->copied_seq; if (flags & MSG_PEEK) { - peek_seq = tp->copied_seq; + peek_offset = max(sk_peek_offset(sk, flags), 0); + peek_seq = tp->copied_seq + peek_offset; seq = &peek_seq; } @@ -2464,11 +2464,11 @@ static int tcp_recvmsg_locked(struct sock *sk, struct msghdr *msg, size_t len, } if ((flags & MSG_PEEK) && - (peek_seq - copied - urg_hole != tp->copied_seq)) { + (peek_seq - peek_offset - copied - urg_hole != tp->copied_seq)) { net_dbg_ratelimited("TCP(%s:%d): Application bug, race in MSG_PEEK\n", current->comm, task_pid_nr(current)); - peek_seq = tp->copied_seq; + peek_seq = tp->copied_seq + peek_offset; } continue; @@ -2509,7 +2509,10 @@ static int tcp_recvmsg_locked(struct sock *sk, struct msghdr *msg, size_t len, WRITE_ONCE(*seq, *seq + used); copied += used; len -= used; - + if (flags & MSG_PEEK) + sk_peek_offset_fwd(sk, used); + else + sk_peek_offset_bwd(sk, used); tcp_rcv_space_adjust(sk); skip_copy: @@ -3010,6 +3013,7 @@ int tcp_disconnect(struct sock *sk, int flags) __skb_queue_purge(&sk->sk_receive_queue); WRITE_ONCE(tp->copied_seq, tp->rcv_nxt); WRITE_ONCE(tp->urg_data, 0); + sk_set_peek_off(sk, -1); tcp_write_queue_purge(sk); tcp_fastopen_active_disable_ofo_check(sk); skb_rbtree_purge(&tp->out_of_order_queue); -- 2.42.0 [View Less]

3 2

[PATCH] conf: We're interested in the MAC address, not in the MAC itself
by Stefano Brivio 12 Apr '24

12 Apr '24

Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- conf.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/conf.c b/conf.c index 9e0318a..90d3436 100644 --- a/conf.c +++ b/conf.c @@ -625,7 +625,7 @@ static unsigned int conf_ip4(unsigned int ifi, int rc = nl_link_get_mac(nl_sock, ifi, mac); if (rc < 0) { char ifname[IFNAMSIZ]; - err("Couldn't discover MAC for %s: %s", + err("Couldn't discover MAC address for %s: %s", if_indextoname(ifi,… [View More]

2 1

[PATCH] pasta, util: Align stack area for clones to maximum natural alignment
by Stefano Brivio 12 Apr '24

12 Apr '24

Given that we use this stack pointer as a location to store arbitrary data types from the cloned process, we need to guarantee that its alignment matches any of those possible data types. runsisi reports that pasta gets a SIGBUS in pasta_open_ns() on aarch64, where the alignment requirement for stack pointers is a 16 bytes (same as the size of a long double), and similar requirements actually apply to most architectures we run on. Reported-by: runsisi <runsisi(a)hust.edu.cn> Link: https:… [View More]

2 1

[PATCH v2] conf: Don't fail if the template interface doesn't have a MAC address
by Stefano Brivio 12 Apr '24

12 Apr '24

...simply resort to using locally-administered address (LAA) as host-side source, instead. Pick 02:00:00:00:00:00, to make it clear that we don't actually care about that address, and also to matchthe 00 (Administratively Assigned Identifier) quadrant of SLAP (RFC 8948). Otherwise, pasta refuses to start if the template is a tun or Wireguard interface. Link: https://bugs.passt.top/show_bug.cgi?id=49 Link: https://github.com/containers/podman/issues/22320 Signed-off-by: Stefano Brivio <… [View More]

1 0

[PATCH] treewide: Compilers' name for armv6l and armv7l is "arm"
by Stefano Brivio 12 Apr '24

12 Apr '24

When I switched from 'uname -m' to 'gcc -dumpmachine' to fetch the architecture name for, among others, seccomp.sh, I didn't realise that "armv6l" and "armv7l" are just Linux kernel names -- compilers just call that "arm". Fix the "syscalls" annotation we use to define seccomp profiles accordingly, otherwise pasta will be terminated on sigreturn() on armv6l and armv7l. Fixes: 213c397492bd ("passt, pasta: Run-time selection of AVX2 build") Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com… [View More]> --- fwd.c | 2 +- log.c | 4 ++-- passt.c | 5 ++--- pasta.c | 4 ++-- tcp_splice.c | 2 +- 5 files changed, 8 insertions(+), 9 deletions(-) diff --git a/fwd.c b/fwd.c index a235d13..b3d5a37 100644 --- a/fwd.c +++ b/fwd.c @@ -38,7 +38,7 @@ * @exclude: Bitmap of ports to exclude from setting (and clear) * * #syscalls:pasta lseek - * #syscalls:pasta ppc64le:_llseek ppc64:_llseek armv6l:_llseek armv7l:_llseek + * #syscalls:pasta ppc64le:_llseek ppc64:_llseek arm:_llseek */ static void procfs_scan_listen(int fd, unsigned int lstate, uint8_t *map, const uint8_t *exclude) diff --git a/log.c b/log.c index bdd31b4..e3630c3 100644 --- a/log.c +++ b/log.c @@ -220,7 +220,7 @@ void logfile_init(const char *name, const char *path, size_t size) * @fd: Log file descriptor * @now: Current timestamp * - * #syscalls lseek ppc64le:_llseek ppc64:_llseek armv6l:_llseek armv7l:_llseek + * #syscalls lseek ppc64le:_llseek ppc64:_llseek arm:_llseek */ static void logfile_rotate_fallocate(int fd, const struct timespec *now) { @@ -257,7 +257,7 @@ static void logfile_rotate_fallocate(int fd, const struct timespec *now) * @fd: Log file descriptor * @now: Current timestamp * - * #syscalls lseek ppc64le:_llseek ppc64:_llseek armv6l:_llseek armv7l:_llseek + * #syscalls lseek ppc64le:_llseek ppc64:_llseek arm:_llseek * #syscalls ftruncate */ static void logfile_rotate_move(int fd, const struct timespec *now) diff --git a/passt.c b/passt.c index 59ab501..e93b6be 100644 --- a/passt.c +++ b/passt.c @@ -192,10 +192,9 @@ void exit_handler(int signal) * #syscalls read write writev * #syscalls socket bind connect getsockopt setsockopt s390x:socketcall close * #syscalls recvfrom sendto shutdown - * #syscalls armv6l:recv armv7l:recv ppc64le:recv - * #syscalls armv6l:send armv7l:send ppc64le:send + * #syscalls arm:recv ppc64le:recv arm:send ppc64le:send * #syscalls accept4|accept listen epoll_ctl epoll_wait|epoll_pwait epoll_pwait - * #syscalls clock_gettime armv6l:clock_gettime64 armv7l:clock_gettime64 + * #syscalls clock_gettime arm:clock_gettime64 */ int main(int argc, char **argv) { diff --git a/pasta.c b/pasta.c index 61feaa9..e73b5af 100644 --- a/pasta.c +++ b/pasta.c @@ -12,8 +12,8 @@ * Author: Stefano Brivio <sbrivio(a)redhat.com> * * #syscalls:pasta clone waitid exit exit_group rt_sigprocmask - * #syscalls:pasta rt_sigreturn|sigreturn armv6l:sigreturn armv7l:sigreturn - * #syscalls:pasta ppc64:sigreturn s390x:sigreturn + * #syscalls:pasta rt_sigreturn|sigreturn + * #syscalls:pasta arm:sigreturn ppc64:sigreturn s390x:sigreturn */ #include <sched.h> diff --git a/tcp_splice.c b/tcp_splice.c index d066112..42b7be0 100644 --- a/tcp_splice.c +++ b/tcp_splice.c @@ -28,7 +28,7 @@ * - FIN_SENT_0: FIN (write shutdown) sent to accepted socket * - FIN_SENT_1: FIN (write shutdown) sent to target socket * - * #syscalls:pasta pipe2|pipe fcntl armv6l:fcntl64 armv7l:fcntl64 ppc64:fcntl64 + * #syscalls:pasta pipe2|pipe fcntl arm:fcntl64 ppc64:fcntl64 */ #include <sched.h> -- 2.43.0 [View Less]

1 0

[PATCH] conf: Don't fail if the template interface doesn't have a MAC address
by Stefano Brivio 12 Apr '24

12 Apr '24

...simply resort to using locally-administered address (LAA) as host-side source, instead. Pick 02:00:00:00:00:00, to make it clear that we don't actually care about that address, and also to matchthe 00 (Administratively Assigned Identifier) quadrant of SLAP (RFC 8948). Otherwise, pasta refuses to start if the template is a tun or Wireguard interface. Link: https://bugs.passt.top/show_bug.cgi?id=49 Link: https://github.com/containers/podman/issues/22320 Signed-off-by: Stefano Brivio <… [View More]

1 0

[net-next 0/2] tcp: add support for SO_PEEK_OFF socket option
by jmaloy＠redhat.com 08 Apr '24

08 Apr '24

From: Jon Maloy <jmaloy(a)redhat.com> We add support for the SO_PEEK_OFF socket option as a new feature in TCP. In a separate patch, we fix a bug that was revealed while testing this feature. Jon Maloy (2): tcp: add support for SO_PEEK_OFF socket option tcp: correct handling of extreme menory squeeze net/ipv4/af_inet.c | 1 + net/ipv4/tcp.c | 16 ++++++++++------ net/ipv4/tcp_output.c | 14 +++++++++----- 3 files changed, 20 insertions(+), 11 deletions(-) -- 2.42.0

5 9

[PATCH v2] RFC: Benchmarking script
by David Gibson 06 Apr '24

06 Apr '24

Although we make some performance measurements in our regular testsuite, those are designed more for getting a quick(ish) rough idea of the performance, rather than a more precise measurement. This patch adds a Python script in contrib/benchmark which can make more detailed benchmarks. It can test both passt & pasta themselves, but also various other scenarios for comparison, such as kernel veth, qemu -net tap and slirp (either qemu -net user or slirp4netns). It does some basic … [View More]statistics on the results to get at least a rough error estimate. --- contrib/benchmark/.gitignore | 3 + contrib/benchmark/benchmark.py | 462 +++++++++++++++++++++++++++++++++ 2 files changed, 465 insertions(+) create mode 100644 contrib/benchmark/.gitignore create mode 100755 contrib/benchmark/benchmark.py diff --git a/contrib/benchmark/.gitignore b/contrib/benchmark/.gitignore new file mode 100644 index 00000000..7e9f3c7c --- /dev/null +++ b/contrib/benchmark/.gitignore @@ -0,0 +1,3 @@ +*.json +*.ssh +*.hosts diff --git a/contrib/benchmark/benchmark.py b/contrib/benchmark/benchmark.py new file mode 100755 index 00000000..0a6599ee --- /dev/null +++ b/contrib/benchmark/benchmark.py @@ -0,0 +1,462 @@ +#! /usr/bin/python3 +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Benchmark for passt/pasta and other things for comparison +# +# Copyright Red Hat +# Author: David Gibson <david(a)gibson.dropbear.id.au> +# + +# How to use +# +# For best results, run on an idle system with external networks +# disabled. +# +# 1. Adjust TIME, OMIT, INTERVAL globals to select the length of the +# test +# 2. Run +# $ ./benchmark.py <scenarios> +# +# Scenarios: +# +# loopback - Direct transfer over lo interface +# Run within a network namespace both to isolate and remove impact +# of any netfilter rules. +# +# veth_{in,out} - Transfer over kernel veth device +# 'in' is from parent netns to child, 'out' is reverse +# +# pasta_{in,out} - Transfer over pasta managed netns +# +# slirp4netns_{in,out} - Transfer over slirp4netns managed netns +# +# qemu_user_{in,out} - Transfer host <-> qemu guest with -net user +# +# passt_{in,out} - Transfer host <-> qemu guest with passt +# +# qemu_tap_{in,out} - Transfer host <-> qemu guest with -net tap +# + +import sys +import os +import time +import socket +import math +import contextlib +import json +import subprocess +import statistics + +TIME, OMIT, INTERVAL = 6, 1, 1 +# TIME, OMIT, INTERVAL = 600, 10, 5 + +NSTOOL = "../../test/nstool" +PASST = "../../passt" +PASTA = "../../pasta" +MBUTO = "../../test/mbuto.img" +GUEST_KEY = "../../test/guest-key" + +DEBUG = True + +SERVER = ["iperf3", "-s", "-1", "-I", "server.pid"] +PORT = 5201 + +IP1 = "192.0.2.1" +IP2 = "192.0.2.2" +IP3 = "192.0.2.3" +IP4 = "192.0.2.4" + + +def dbg(fmt, *args, **kwargs): + if DEBUG: + print("DBG: " + fmt.format(*args, **kwargs), file=sys.stderr) + + +# Base class for command runners +class Cmd: + def popen(self, args, **kwargs): + return subprocess.Popen(self.cmd_prefix + args, **kwargs) + + def run(self, args, **kwargs): + return subprocess.run(self.cmd_prefix + args, **kwargs) + + +class HostCmd(Cmd): + def __init__(self): + self.cmd_prefix = [] + + +class NsTool(contextlib.AbstractContextManager, Cmd): + def __init__(self, ctl, parent, holdprefix): + try: + os.remove(ctl) + except FileNotFoundError: + pass + self.ctl = ctl + self.parent = parent + self.proc = parent.popen(holdprefix + [NSTOOL, "hold", ctl], + stdout=subprocess.DEVNULL, stderr=None) + self.cmd_prefix = [NSTOOL, "exec", ctl, "--"] + + def __enter__(self): + self.parent.run([NSTOOL, "info", "-w", self.ctl], + stdout=subprocess.DEVNULL).check_returncode() + self.proc.__enter__() + return self + + def __exit__(self, exc_type, exc_value, traceback): + return self.proc.__exit__(exc_type, exc_value, traceback) + + def relpid(self): + out = self.parent.run([NSTOOL, "info", "-p", self.ctl], + stdout=subprocess.PIPE) + return int(out.stdout) + + def stop(self): + return self.parent.run([NSTOOL, "stop", self.ctl]).check_returncode() + + +# We put everything into an isolated network namespace for a few reasons: +# 1. Make sure test traffic doesn't leak out onto external interfaces +# 2. Lets us allocate our own addresses freely +# 3. Avoids firewall rules which might be set up on the host affecting +# performance +(a)contextlib.contextmanager +def isolate(ctl): + with NsTool(ctl, HostCmd(), ["unshare", "-Unrp"]) as isons: + isons.run(["ip", "link", "set", "lo", "up"]).check_returncode() + yield isons + isons.stop() + + +(a)contextlib.contextmanager +def loopback(isons): + with isons.popen(SERVER): + yield (isons, "127.0.0.1") + + +(a)contextlib.contextmanager +def veth_ns(isons, ip_outer, ip_inner): + isons.run(["ip", "link", "add", "type", "veth"]) + with NsTool("veth_in.ctl", isons, ["unshare", "-n"]) as guestns: + dbg("veth_ns guest ns ready") + isons.run(["ip", "link", "set", "veth1", + "netns", str(guestns.relpid())]) + isons.run(["ip", "link", "set", "veth0", "up"]) + isons.run(["ip", "addr", "add", f"{ip_outer}/24", "dev", "veth0"]) + guestns.run(["ip", "link", "set", "veth1", "up"]) + guestns.run(["ip", "addr", "add", f"{ip_inner}/24", "dev", "veth1"]) + yield guestns + guestns.stop() + + +(a)contextlib.contextmanager +def veth_in(isons): + outer = IP3 + inner = IP4 + with veth_ns(isons, outer, inner) as guestns: + dbg("veth_in guestns ready") + with guestns.popen(SERVER): + yield (isons, inner) + + +(a)contextlib.contextmanager +def veth_out(isons): + outer = IP3 + inner = IP4 + with veth_ns(isons, outer, inner) as guestns: + dbg("veth_in guestns ready") + with isons.popen(SERVER): + yield (guestns, outer) + + +def passt_config_isons(isons): + isons.run(["ip", "link", "add", "type", "dummy"]).check_returncode() + isons.run(["ip", "link", "set", "dummy0", "up"]).check_returncode() + isons.run(["ip", "addr", "add", f"{IP1}/24", + "dev", "dummy0"]).check_returncode() + isons.run(["ip", "route", "add", "default", "via", str(IP2), + "dev", "dummy0"]).check_returncode() + + +(a)contextlib.contextmanager +def pasta_in(isons): + passt_config_isons(isons) + cmd = [PASTA, "--config-net", "-t", str(PORT), "--"] + SERVER + with isons.popen(cmd): + time.sleep(1) + yield (isons, IP1) + + +(a)contextlib.contextmanager +def pasta_out(isons): + passt_config_isons(isons) + pasta = NsTool("pasta_out.ctl", isons, [PASTA, "--config-net", "--"]) + with isons.popen(SERVER), pasta as pasta: + time.sleep(1) + yield (pasta, IP2) + pasta.stop() + + +class Slirp4NetNs(contextlib.AbstractContextManager): + def __init__(self, ns, name, pid): + self.apipath = f"{name}.api" + self.ns = ns + self.pid = pid + + def api(self, req): + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as sock: + sock.connect(self.apipath) + sock.sendall(req) + + def __enter__(self): + exit_r, exit_w = os.pipe() + self.exit = os.fdopen(exit_w, "wb") + cmd = ["slirp4netns", "-c", "-a", self.apipath, "-e", str(exit_r), + str(self.pid), "tap0"] + self.proc = self.ns.popen(cmd, pass_fds=(exit_r,)) + self.proc.__enter__() + time.sleep(1) + return self + + def __exit__(self, exc_type, exc_value, traceback): + self.exit.close() + return self.proc.__exit__(exc_type, exc_value, traceback) + + +(a)contextlib.contextmanager +def slirp4netns_in(isons): + with NsTool("slirp4netns_in.ctl", isons, ["unshare", "-n"]) as guestns: + with Slirp4NetNs(isons, "slirp4netns_in", guestns.relpid()) as slirp: + req = f''' +{{ + "execute": "add_hostfwd", + "arguments": {{ + "proto": "tcp", + "host_addr": "0.0.0.0", + "host_port": {PORT}, + "guest_addr": "10.0.2.100", + "guest_port": {PORT} + }} +}}''' + slirp.api(bytes(req, 'UTF-8')) + with guestns.popen(SERVER): + yield (isons, "127.0.0.1") + guestns.stop() + dbg("guestns closed") + dbg("slirp exited") + + +(a)contextlib.contextmanager +def slirp4netns_out(isons): + with isons.popen(SERVER), \ + NsTool("slirp4netns_in.ctl", isons, ["unshare", "-n"]) as guestns: + with Slirp4NetNs(isons, "slirp4netns_out", guestns.relpid()): + yield (guestns, "10.0.2.2") + guestns.stop() + + +class MbutoQemu(Cmd, contextlib.AbstractContextManager): + def __init__(self, parent, name, opts): + self.ns = parent + self.name = name + self.ssh_config = f"{name}.ssh" + self.pidfile = f"{name}.pid" + + cid = hash(name) & 0xffffffff + self.hosts = os.path.realpath(f"{name}.hosts") + try: + os.remove(self.hosts) + except FileNotFoundError: + pass + key = os.path.realpath(GUEST_KEY) + + with open(self.ssh_config, "w") as c: + c.write(f""" +Host {name} + User root + UserKnownHostsFile {self.hosts} + StrictHostKeyChecking no + IdentityFile {key} + IdentityAgent none + ProxyCommand socat - VSOCK-CONNECT:{cid}:22 +""") + + kernel = f"/boot/vmlinuz-{os.uname().release}" + + self.qemu_cmd = ["qemu-kvm", "-kernel", kernel, "-initrd", MBUTO, + "-append", '"console=ttyS0"', + "-nodefaults", "-nographic", "-serial", "stdio", + "-device", f"vhost-vsock-pci,guest-cid={cid}", + "-pidfile", self.pidfile] + list(opts) + + self.cmd_prefix = ["ssh", "-F", self.ssh_config, "-t", name, "--"] + + def __enter__(self): + dbg(f"Starting {self.qemu_cmd}") + self.qemu = self.ns.popen(self.qemu_cmd) + self.qemu.__enter__() + while self.run([":"], stderr=subprocess.DEVNULL).returncode != 0: + pass + + return self + + def __exit__(self, exc_type, exc_value, traceback): + pid = int(open(self.pidfile, "r").read()) + self.ns.run(["kill", str(pid)]) + ret = self.qemu.__exit__(exc_type, exc_value, traceback) + os.remove(self.ssh_config) + os.remove(self.hosts) + return ret + + def dhcp(self): + self.run(["ip", "link", "set", "lo", "up"]).check_returncode() + self.run(["ip", "link", "set", "eth0", "up"]).check_returncode() + self.run(["dhclient", "-v", "eth0"]).check_returncode() + + def static(self, ip): + self.run(["ip", "link", "set", "lo", "up"]).check_returncode() + self.run(["ip", "link", "set", "eth0", "up"]).check_returncode() + self.run(["ip", "addr", "add", ip, "dev", "eth0"]).check_returncode() + + +(a)contextlib.contextmanager +def qemu_user_in(isons): + opts = ["-m", "4G", + "-netdev", f"user,id=slirp,hostfwd=tcp:127.0.0.1:{PORT}-:{PORT}", + "-device", "virtio-net-pci,netdev=slirp"] + with MbutoQemu(isons, "slirp-qemu", opts) as qemu: + qemu.dhcp() + with qemu.popen(SERVER): + time.sleep(2) + dbg("Guest server ready") + yield (isons, "127.0.0.1") + + +(a)contextlib.contextmanager +def qemu_user_out(isons): + opts = ["-m", "4G", "-netdev", "user,id=slirp", + "-device", "virtio-net-pci,netdev=slirp"] + with MbutoQemu(isons, "slirp-qemu", opts) as qemu: + qemu.dhcp() + with isons.popen(SERVER): + time.sleep(2) + yield (qemu, "10.0.2.2") + + +(a)contextlib.contextmanager +def passt(ns, sock, opts=[]): + passt_cmd = [PASST, "-s", sock, "-1", "-f"] + opts + with ns.popen(passt_cmd): + yield + os.remove(sock) + + +(a)contextlib.contextmanager +def passt_qemu(ns, name, opts=[]): + sock = f"{name}.passt" + with passt(ns, sock, opts): + streamopts = f"id=passt,server=off,addr.type=unix,addr.path={sock}" + qemu_opts = ["-m", "4G", + "-netdev", f"stream,{streamopts}", + "-device", "virtio-net-pci,netdev=passt"] + with MbutoQemu(ns, name, qemu_opts) as qemu: + qemu.dhcp() + yield qemu + + +(a)contextlib.contextmanager +def passt_in(isons): + passt_config_isons(isons) + with passt_qemu(isons, "passt_in", ["-t", str(PORT)]) as qemu: + with qemu.popen(SERVER): + time.sleep(2) + dbg("Guest server ready") + yield (isons, "127.0.0.1") + + +(a)contextlib.contextmanager +def passt_out(isons): + passt_config_isons(isons) + with passt_qemu(isons, "passt_out") as qemu: + with isons.popen(SERVER): + time.sleep(2) + yield (qemu, IP2) + + +(a)contextlib.contextmanager +def tap_qemu(ns, name, opts): + opts += ["-netdev", "tap,id=tap,script=no,downscript=no", + "-device", "virtio-net-pci,netdev=tap"] + with MbutoQemu(ns, name, opts) as qemu: + ns.run(["ip", "link", "set", "tap0", "up"]).check_returncode() + ns.run(["ip", "addr", "add", f"{IP3}/24", + "dev", "tap0"]).check_returncode() + qemu.static(f"{IP4}/24") + yield qemu + + +(a)contextlib.contextmanager +def qemu_tap_in(isons): + opts = ["-m", "4G"] + with tap_qemu(isons, "tap_in", opts) as qemu: + with qemu.popen(SERVER): + time.sleep(2) + dbg("Guest server ready") + yield (isons, IP4) + + +(a)contextlib.contextmanager +def qemu_tap_out(isons): + opts = ["-m", "4G"] + with tap_qemu(isons, "tap_out", opts) as qemu: + with isons.popen(SERVER): + time.sleep(2) + yield (qemu, IP3) + + +def bench_one(name, setup): + print(f"Running benchmark {name}...") + with setup as (crun, addr): + dbg("Setup for {}", name) + ccmd = ["iperf3", "-c", addr, "-J", "-t", str(TIME), "-O", str(OMIT), + "-i", str(INTERVAL), "-l", "1M"] + out = crun.run(ccmd, stdout=subprocess.PIPE, stderr=None) + dbg("Client completed for {}", name) + out.check_returncode() + return out.stdout + + +def stats(name, out): + parse = json.loads(out) + intervals = parse['intervals'] + sums = [i['sum'] for i in intervals] + samples = [s['bits_per_second'] for s in sums if not s['omitted']] + mean = statistics.fmean(samples) + stdev = statistics.stdev(samples) + spread = stdev / mean + n = len(samples) + stderr = stdev / math.sqrt(float(n)) + relerr = stderr / abs(mean) + gbps = mean / 1000000000 + print(f"{name:16}:\t{gbps:8.2f} Gbps ± {relerr:.1%} " + \ + f"({n} intervals, {spread:.1%})") + + +ALL = ["loopback", "veth_in", "veth_out", "pasta_in", "pasta_out", + "slirp4netns_in", "slirp4netns_out", "qemu_user_in", "qemu_user_out", + "qemu_tap_in", "qemu_tap_out", "passt_in", "passt_out"] + +if __name__ == "__main__": + results = {} + cases = sys.argv[1:] + if not cases: + cases = ALL + for case in cases: + with isolate("isolate.ctl") as isons: + setup = globals()[case](isons) + out = bench_one(case, setup) + open(f"{case}.json", "wb").write(out) + results[case] = out + for name, out in results.items(): + stats(name, out) -- 2.44.0 [View Less]

2 2

passt: new version 2024_04_05.954589b available
by Stefano Brivio 05 Apr '24

05 Apr '24

The new version with tag 2024_04_05.954589b includes the following changes: 954589b test: Verify that podman tests are using the pasta binary we expect 489b28e test: catatonit may not be in $PATH f9fe3ae test: Build and download podman as a test asset e8b7821 test: Make sure to update mbuto repository ef2cb13 cppcheck: Explicitly give files to check 97e8b33 netlink: Ignore routes to link-local addresses for selecting interface 67a6258 util: Add helper to return name … [View More]

1 0

[net-next 0/2] tcp: add support for SO_PEEK_OFF socket option
by Jon Maloy 05 Apr '24

05 Apr '24

We add support for the SO_PEEK_OFF socket option as a new feature in TCP. In a separate patch, we fix a bug that was revealed while testing this feature. Jon Maloy (2): tcp: add support for SO_PEEK_OFF tcp: correct handling of extreme menory squeeze net/ipv4/af_inet.c | 1 + net/ipv4/tcp.c | 16 ++++++++++------ net/ipv4/tcp_output.c | 5 ++++- 3 files changed, 15 insertions(+), 7 deletions(-) -- 2.42.0

2 4