- dev - passt

[PATCH v2 00/11] Assorted test fixes, batch 7
by David Gibson 26 Sep '22

26 Sep '22

Here's another bundle of test fixes, focused on making the tests faster and more reliable. Based on my earlier series cleaning up the port forwarding data structures. Changes since v1: * Rebased and fixed conflict with another Makefile change * Dropped patch removing sleeps from layout functions David Gibson (11): test: Add wait_for() shell helper test: Remove unnecessary sleeps from shutdown tests test: More robust wait for pasta/passt to be ready test: Use --config-net for namespace setup test: Simplify data handling for transfer tests test: Remove unneccessary pane naming from layout_two_guests clang-tidy: Disable 'readability-identifier-length' cppcheck: Avoid excessive scanning due to system headers cppcheck: Run quietly Makefile: Simplify getting target triple for compiler cppcheck: Add target specific headers Makefile | 33 +++---- test/.gitignore | 1 + test/Makefile | 15 ++- test/lib/context | 4 +- test/lib/layout | 4 - test/lib/setup | 32 +++---- test/lib/util | 8 ++ test/passt.mbuto | 6 +- test/passt/shutdown | 1 - test/passt/tcp | 53 +++++------ test/passt/udp | 29 +++--- test/passt_in_ns/shutdown | 1 - test/passt_in_ns/tcp | 187 ++++++++++++++++---------------------- test/passt_in_ns/udp | 93 ++++++++----------- test/pasta/tcp | 79 +++++++--------- test/pasta/udp | 43 ++++----- test/two_guests/basic | 2 +- 17 files changed, 251 insertions(+), 340 deletions(-) -- 2.37.3

1 11

[PATCH v2 0/8] Clean up and fix bugs in port forwarding data structures
by David Gibson 24 Sep '22

24 Sep '22

The information we need to keep track of which ports are forwarded where is currently split across several different values and data structures in several different places. Worse, a number of those structures are incorrectly sized due to an off by one error, which could lead to buffer overruns. Fix the sizing, and re-organize the data structures in a way that should make it less likely to repeat that mistake. While we're there, correct a similar off-by-one mis-sizing of a number of other arrays. Changes since v1: * Use a define for the array size, rather than a typedef to handle the bitmaps of ports David Gibson (8): Improve types and names for port forwarding configuration Consolidate port forwarding configuration into a common structure udp: Delay initialization of UDP reversed port mapping table Don't use indirect remap functions for conf_ports() Pass entire port forwarding configuration substructure to conf_ports() Treat port numbers as unsigned Fix widespread off-by-one error dealing with port numbers icmp: Correct off by one errors dealing with number of echo request ids Makefile | 2 +- conf.c | 136 +++++++++++++++++++++++---------------------------- icmp.c | 5 +- passt.h | 1 + port_fwd.h | 34 +++++++++++++ tcp.c | 68 ++++++++------------------ tcp.h | 13 ++--- tcp_splice.c | 4 +- udp.c | 59 ++++++++++------------ udp.h | 27 +++++----- 10 files changed, 168 insertions(+), 181 deletions(-) create mode 100644 port_fwd.h -- 2.37.3

2 10

[PATCH] Fix the name of the qemu-system-* executable
by Dario Faggioli 24 Sep '22

24 Sep '22

Hello everyone! So, while playing with out passt package for openSUSE, my colleague Vasiliy found out an issue in how passt assumes the qemu binary to be called. In fact, in openSUSE, while we have a package that allows people to launch qemu by invoking `qemu-kvm`, we don't install it by default, and we therefore only have the standard upstream names in the filesystem, which have the ARCH part in lowercase. This patch, which we're currently carrying as a downstream one in our package, should fix the problem. Thanks and Regards, Dario --- Vasiliy Ulyanov (1): Fix the name of the qemu-system-* executable Makefile | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) -- Dario Faggioli, Ph.D http://about.me/dario.faggioli Virtualization Software Engineer SUSE Labs, SUSE https://www.suse.com/ ------------------------------------------------------------------- <<This happens because _I_ choose it to happen!>> (Raistlin Majere)

2 3

[PATCH 0/8] Clean up and fix bugs in port forwarding data structures
by David Gibson 24 Sep '22

24 Sep '22

The information we need to keep track of which ports are forwarded where is currently split across several different values and data structures in several different places. Worse, a number of those structures are incorrectly sized due to an off by one error, which could lead to buffer overruns. Fix the sizing, and re-organize the data structures in a way that should make it less likely to repeat that mistake. While we're there, correct a similar off-by-one mis-sizing of a number of other arrays. David Gibson (8): Improve types and names for port forwarding configuration Consolidate port forwarding configuration into a common structure udp: Delay initialization of UDP reversed port mapping table Don't use indirect remap functions for conf_ports() Pass entire port forwarding configuration substructure to conf_ports() Treat port numbers as unsigned Fix widespread off-by-one error dealing with port numbers icmp: Correct off by one errors dealing with number of echo request ids Makefile | 2 +- conf.c | 136 +++++++++++++++++++++++---------------------------- icmp.c | 5 +- passt.h | 1 + port_fwd.h | 34 +++++++++++++ tcp.c | 68 ++++++++------------------ tcp.h | 13 ++--- tcp_splice.c | 4 +- udp.c | 59 ++++++++++------------ udp.h | 27 +++++----- 10 files changed, 168 insertions(+), 181 deletions(-) create mode 100644 port_fwd.h -- 2.37.3

2 12

[PATCH 00/12] Assorted test fixes, batch 7
by David Gibson 24 Sep '22

24 Sep '22

Here's another bundle of test fixes, focused on making the tests faster and more reliable. Based on my earlier series cleaning up the port forwarding data structures. David Gibson (12): test: Add wait_for() shell helper test: Remove unnecessary sleeps from layout functions test: Remove unnecessary sleeps from shutdown tests test: More robust wait for pasta/passt to be ready test: Use --config-net for namespace setup test: Simplify data handling for transfer tests test: Remove unneccessary pane naming from layout_two_guests clang-tidy: Disable 'readability-identifier-length' cppcheck: Avoid excessive scanning due to system headers cppcheck: Run quietly Makefile: Simplify getting target triple for compiler cppcheck: Add target specific headers Makefile | 32 +++---- test/.gitignore | 1 + test/Makefile | 15 ++- test/lib/context | 4 +- test/lib/layout | 24 ----- test/lib/setup | 32 +++---- test/lib/util | 8 ++ test/passt.mbuto | 6 +- test/passt/shutdown | 1 - test/passt/tcp | 53 +++++------ test/passt/udp | 29 +++--- test/passt_in_ns/shutdown | 1 - test/passt_in_ns/tcp | 187 ++++++++++++++++---------------------- test/passt_in_ns/udp | 93 ++++++++----------- test/pasta/tcp | 79 +++++++--------- test/pasta/udp | 43 ++++----- test/two_guests/basic | 2 +- 17 files changed, 250 insertions(+), 360 deletions(-) -- 2.37.3

2 21

[PATCH 00/11] Fixes for demos, pre-push hook, README
by Stefano Brivio 24 Sep '22

24 Sep '22

After the recent rework of command dispatch and handling of temporary files, a few issues prevent demos from completing, and CI files from being uploaded. Fix those. On top of that, rebase to Podman's latest upstream so that the Podman demo can run again, shorten paths used to change directories in demo scripts, and fix a number of inconsistencies in the README. Given that the current state of the website is inconsistent, I'd be inclined to apply these right away, while posting to ease later review anyway. Stefano Brivio (11): contrib/podman: Rebase to latest upstream hooks/pre_push: Fix upload of CI's logs and terminal capture file test/demo: Use relative paths to change directories when possible test/demo: Avoid using port 5201 on the host test/lib: Drop perf_report_append() from perf_report test/lib: Don't try to write to perf.js when running demos README: Fix misspellings of openSUSE README: Point openSUSE links to Dario's OBS repository README: Fix indentation in "Try It" section README: Fix paragraph in Try It section of passt README: Add legend for Features section README.md | 24 ++- ...001-libpod-Add-pasta-networking-mode.patch | 95 +++++----- hooks/pre-push | 11 +- test/demo/passt | 45 +++-- test/demo/pasta | 20 +- test/demo/podman | 176 +++++++++--------- test/lib/perf_report | 13 +- 7 files changed, 198 insertions(+), 186 deletions(-) -- 2.35.1

1 11

[PATCH] test/passt.mbuto: Don't fail on missing guest public key
by Stefano Brivio 23 Sep '22

23 Sep '22

We won't necessarily run mbuto as part of regular tests: it can also be used for demos or out-of-tree tests. To keep the profile simple, leave the whole sshd setup there, which is otherwise harmless, but don't fail if guest-key.pub is missing in the current directory. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- test/passt.mbuto | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/passt.mbuto b/test/passt.mbuto index d29f456..51c9ada 100755 --- a/test/passt.mbuto +++ b/test/passt.mbuto @@ -68,7 +68,7 @@ EOF ln -s /run/sshd /usr/share/empty.sshd cat > /root/.ssh/authorized_keys <<EOF -'"$(cat guest-key.pub)"' +'"$(cat guest-key.pub 2>/dev/null || :)"' EOF chmod 600 /root/.ssh/authorized_keys chmod 700 /root -- 2.35.1

1 0

[PATCH 0/7] Fixes and workarounds for tests, Coverity warnings
by Stefano Brivio 23 Sep '22

23 Sep '22

With this series, I'm almost able to run the full test suite together with the new command dispatch mechanism. I still hit frequent failures in the passt_tcp performance test, so I'm not pushing this out at the moment, but as it's taking me a while, I'd rather share this earlier. Included are also two fixes for harmless (but ugly) issues discovered by Coverity. Stefano Brivio (7): test/perf: Always use /sbin/sysctl in tcp test test/perf: Switch performance test duration to 10 seconds instead of 30 tap: Check return value of accept4() before calling getsockopt() conf, tcp, udp: Arrays for ports need 2^16 values, not 2^16-8 test/lib: Restore IFS while executing directives in def blocks test/lib: Wait on iperf3 clients to be done, then send SIGINT to servers test/perf: Disable periodic throughput reports to avoid vhost hang conf.c | 2 +- tap.c | 6 ++++-- tcp.h | 4 ++-- test/lib/test | 20 ++++++++++---------- test/perf/passt_tcp | 10 +++++----- test/perf/passt_udp | 4 ++-- test/perf/pasta_tcp | 10 +++++----- test/perf/pasta_udp | 4 ++-- udp.h | 4 ++-- 9 files changed, 33 insertions(+), 31 deletions(-) -- 2.35.1

2 18

[PATCH 0/5] Some more hacks and workarounds to make tests pass again
by Stefano Brivio 23 Sep '22

23 Sep '22

Some of these are actual fixes, some are just hacks to keep tests running while we figure out issues. A part of this series should actually be reverted once we figure out why iperf3 clients and servers get stuck now and then. Stefano Brivio (5): test/lib: Run also iperf3 clients in background, revert to time-based wait test/lib: Wait for kernel to free up ports used by iperf3 before reusing them test/perf: Wait for neper servers in guest to be ready before starting client test/lib: Wait for DHCPv4 before starting DHCPv6 client in two_guests test test/distro: Update workarounds for Ubuntu 22.04 on s390x test/distro/ubuntu | 4 +++- test/lib/setup | 1 + test/lib/test | 11 ++++++++--- test/perf/passt_tcp | 4 ++++ test/perf/passt_udp | 2 ++ 5 files changed, 18 insertions(+), 4 deletions(-) -- 2.35.1

2 6

[PATCH] Makefile: Allow define overrides by prepending, not appending, CFLAGS
by Stefano Brivio 23 Sep '22

23 Sep '22

If we append CFLAGS to the ones passed via command line (if any), -D options we append will override -D options passed on command line (if any). For example, OpenSUSE build flags include -D_FORTIFY_SOURCE=3, and we want to have -D_FORTIFY_SOURCE=2, if and only if not overridden. The current behaviour implies we redefine _FORTIFY_SOURCE as 2, though. Instead of appending CFLAGS, prepend them by adding all the default build flags to another variable, a simply expanded one (defined with :=), named FLAGS, and pass that *before* CFLAGS in targets, so that defines from command line can override default flags. Reported-by: Dario Faggioli <dfaggioli(a)suse.com> Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- Makefile | 49 +++++++++++++++++++++++++------------------------ 1 file changed, 25 insertions(+), 24 deletions(-) diff --git a/Makefile b/Makefile index af3d1ff..6805c23 100644 --- a/Makefile +++ b/Makefile @@ -23,13 +23,13 @@ AUDIT_ARCH := $(shell echo $(AUDIT_ARCH) | sed 's/I[456]86/I386/') AUDIT_ARCH := $(shell echo $(AUDIT_ARCH) | sed 's/PPC64/PPC/') AUDIT_ARCH := $(shell echo $(AUDIT_ARCH) | sed 's/PPCLE/PPC64LE/') -CFLAGS += -Wall -Wextra -pedantic -std=c99 -D_XOPEN_SOURCE=700 -D_GNU_SOURCE -CFLAGS += -D_FORTIFY_SOURCE=2 -O2 -pie -fPIE -CFLAGS += -DPAGE_SIZE=$(shell getconf PAGE_SIZE) -CFLAGS += -DNETNS_RUN_DIR=\"/run/netns\" -CFLAGS += -DPASST_AUDIT_ARCH=AUDIT_ARCH_$(AUDIT_ARCH) -CFLAGS += -DRLIMIT_STACK_VAL=$(RLIMIT_STACK_VAL) -CFLAGS += -DARCH=\"$(TARGET_ARCH)\" +FLAGS := -Wall -Wextra -pedantic -std=c99 -D_XOPEN_SOURCE=700 -D_GNU_SOURCE +FLAGS += -D_FORTIFY_SOURCE=2 -O2 -pie -fPIE +FLAGS += -DPAGE_SIZE=$(shell getconf PAGE_SIZE) +FLAGS += -DNETNS_RUN_DIR=\"/run/netns\" +FLAGS += -DPASST_AUDIT_ARCH=AUDIT_ARCH_$(AUDIT_ARCH) +FLAGS += -DRLIMIT_STACK_VAL=$(RLIMIT_STACK_VAL) +FLAGS += -DARCH=\"$(TARGET_ARCH)\" PASST_SRCS = arch.c arp.c checksum.c conf.c dhcp.c dhcpv6.c icmp.c igmp.c \ isolation.c lineread.c mld.c ndp.c netlink.c packet.c passt.c pasta.c \ @@ -50,36 +50,36 @@ HEADERS = $(PASST_HEADERS) # from the pointer arithmetic used from the tcp_tap_handler() path to get the # remote connection address. ifeq ($(shell $(CC) -dumpversion),11) -ifneq (,$(filter -flto%,$(CFLAGS))) -ifneq (,$(filter -O2,$(CFLAGS))) - CFLAGS += -DTCP_HASH_NOINLINE - CFLAGS += -DSIPHASH_20B_NOINLINE +ifneq (,$(filter -flto%,$(FLAGS) $(CFLAGS))) +ifneq (,$(filter -O2,$(FLAGS) $(CFLAGS))) + FLAGS += -DTCP_HASH_NOINLINE + FLAGS += -DSIPHASH_20B_NOINLINE endif endif endif C := \#include <linux/tcp.h>\nstruct tcp_info x = { .tcpi_snd_wnd = 0 }; ifeq ($(shell printf "$(C)" | $(CC) -S -xc - -o - >/dev/null 2>&1; echo $$?),0) - CFLAGS += -DHAS_SND_WND + FLAGS += -DHAS_SND_WND endif C := \#include <linux/tcp.h>\nstruct tcp_info x = { .tcpi_bytes_acked = 0 }; ifeq ($(shell printf "$(C)" | $(CC) -S -xc - -o - >/dev/null 2>&1; echo $$?),0) - CFLAGS += -DHAS_BYTES_ACKED + FLAGS += -DHAS_BYTES_ACKED endif C := \#include <linux/tcp.h>\nstruct tcp_info x = { .tcpi_min_rtt = 0 }; ifeq ($(shell printf "$(C)" | $(CC) -S -xc - -o - >/dev/null 2>&1; echo $$?),0) - CFLAGS += -DHAS_MIN_RTT + FLAGS += -DHAS_MIN_RTT endif C := \#include <sys/random.h>\nint main(){int a=getrandom(0, 0, 0);} ifeq ($(shell printf "$(C)" | $(CC) -S -xc - -o - >/dev/null 2>&1; echo $$?),0) - CFLAGS += -DHAS_GETRANDOM + FLAGS += -DHAS_GETRANDOM endif ifeq ($(shell :|$(CC) -fstack-protector-strong -S -xc - -o - >/dev/null 2>&1; echo $$?),0) - CFLAGS += -fstack-protector-strong + FLAGS += -fstack-protector-strong endif prefix ?= /usr/local @@ -98,18 +98,19 @@ endif all: $(BIN) $(MANPAGES) docs -static: CFLAGS += -static -DGLIBC_NO_STATIC_NSS +static: FLAGS += -static -DGLIBC_NO_STATIC_NSS static: clean all seccomp.h: $(PASST_SRCS) $(PASST_HEADERS) @ EXTRA_SYSCALLS=$(EXTRA_SYSCALLS) ./seccomp.sh $^ passt: $(PASST_SRCS) $(PASST_HEADERS) seccomp.h - $(CC) $(CFLAGS) $(PASST_SRCS) -o passt $(LDFLAGS) + $(CC) $(FLAGS) $(CFLAGS) $(PASST_SRCS) -o passt $(LDFLAGS) -passt.avx2: CFLAGS += -Ofast -mavx2 -ftree-vectorize -funroll-loops +passt.avx2: FLAGS += -Ofast -mavx2 -ftree-vectorize -funroll-loops passt.avx2: $(PASST_SRCS) $(PASST_HEADERS) seccomp.h - $(CC) $(filter-out -O2,$(CFLAGS)) $(PASST_SRCS) -o passt.avx2 $(LDFLAGS) + $(CC) $(filter-out -O2,$(FLAGS) $(CFLAGS)) \ + $(PASST_SRCS) -o passt.avx2 $(LDFLAGS) passt.avx2: passt @@ -117,12 +118,12 @@ pasta.avx2 pasta.1 pasta: pasta%: passt% ln -s $< $@ qrap: $(QRAP_SRCS) passt.h - $(CC) $(CFLAGS) $(QRAP_SRCS) -o qrap $(LDFLAGS) + $(CC) $(FLAGS) $(CFLAGS) $(QRAP_SRCS) -o qrap $(LDFLAGS) valgrind: EXTRA_SYSCALLS="rt_sigprocmask rt_sigtimedwait rt_sigaction \ getpid gettid kill clock_gettime mmap munmap open \ unlink gettimeofday futex" -valgrind: CFLAGS:=-g -O0 $(filter-out -O%,$(CFLAGS)) +valgrind: FLAGS:=-g -O0 $(filter-out -O%,$(FLAGS)) valgrind: all .PHONY: clean @@ -261,7 +262,7 @@ clang-tidy: $(SRCS) $(HEADERS) -altera-struct-pack-align,\ -concurrency-mt-unsafe \ -config='{CheckOptions: [{key: bugprone-suspicious-string-compare.WarnOnImplicitComparison, value: "false"}]}' \ - --warnings-as-errors=* $(SRCS) -- $(filter-out -pie,$(CFLAGS)) + --warnings-as-errors=* $(SRCS) -- $(filter-out -pie,$(FLAGS) $(CFLAGS)) ifeq ($(shell $(CC) -v 2>&1 | grep -c "gcc version"),1) TARGET := $(shell ${CC} -v 2>&1 | sed -n 's/Target: $.*$/\1/p') @@ -304,5 +305,5 @@ cppcheck: $(SRCS) $(HEADERS) --suppress=unmatchedSuppression:udp.c \ --suppress=unmatchedSuppression:util.c \ --suppress=unmatchedSuppression:util.h \ - $(filter -D%,$(CFLAGS)) \ + $(filter -D%,$(FLAGS) $(CFLAGS)) \ . -- 2.35.1

2 3