- dev - passt

[PATCH 00/12] Assorted test fixes, batch 7
by David Gibson 24 Sep '22

24 Sep '22

Here's another bundle of test fixes, focused on making the tests faster and more reliable. Based on my earlier series cleaning up the port forwarding data structures. David Gibson (12): test: Add wait_for() shell helper test: Remove unnecessary sleeps from layout functions test: Remove unnecessary sleeps from shutdown tests test: More robust wait for pasta/passt to be ready test: Use --config-net for namespace setup test: Simplify data handling for transfer tests test: Remove unneccessary pane naming from layout_two_guests clang-tidy: Disable 'readability-identifier-length' cppcheck: Avoid excessive scanning due to system headers cppcheck: Run quietly Makefile: Simplify getting target triple for compiler cppcheck: Add target specific headers Makefile | 32 +++---- test/.gitignore | 1 + test/Makefile | 15 ++- test/lib/context | 4 +- test/lib/layout | 24 ----- test/lib/setup | 32 +++---- test/lib/util | 8 ++ test/passt.mbuto | 6 +- test/passt/shutdown | 1 - test/passt/tcp | 53 +++++------ test/passt/udp | 29 +++--- test/passt_in_ns/shutdown | 1 - test/passt_in_ns/tcp | 187 ++++++++++++++++---------------------- test/passt_in_ns/udp | 93 ++++++++----------- test/pasta/tcp | 79 +++++++--------- test/pasta/udp | 43 ++++----- test/two_guests/basic | 2 +- 17 files changed, 250 insertions(+), 360 deletions(-) -- 2.37.3

2 21

[PATCH 00/11] Fixes for demos, pre-push hook, README
by Stefano Brivio 24 Sep '22

24 Sep '22

After the recent rework of command dispatch and handling of temporary files, a few issues prevent demos from completing, and CI files from being uploaded. Fix those. On top of that, rebase to Podman's latest upstream so that the Podman demo can run again, shorten paths used to change directories in demo scripts, and fix a number of inconsistencies in the README. Given that the current state of the website is inconsistent, I'd be inclined to apply these right away, while posting to ease later review anyway. Stefano Brivio (11): contrib/podman: Rebase to latest upstream hooks/pre_push: Fix upload of CI's logs and terminal capture file test/demo: Use relative paths to change directories when possible test/demo: Avoid using port 5201 on the host test/lib: Drop perf_report_append() from perf_report test/lib: Don't try to write to perf.js when running demos README: Fix misspellings of openSUSE README: Point openSUSE links to Dario's OBS repository README: Fix indentation in "Try It" section README: Fix paragraph in Try It section of passt README: Add legend for Features section README.md | 24 ++- ...001-libpod-Add-pasta-networking-mode.patch | 95 +++++----- hooks/pre-push | 11 +- test/demo/passt | 45 +++-- test/demo/pasta | 20 +- test/demo/podman | 176 +++++++++--------- test/lib/perf_report | 13 +- 7 files changed, 198 insertions(+), 186 deletions(-) -- 2.35.1

1 11

[PATCH] test/passt.mbuto: Don't fail on missing guest public key
by Stefano Brivio 23 Sep '22

23 Sep '22

We won't necessarily run mbuto as part of regular tests: it can also be used for demos or out-of-tree tests. To keep the profile simple, leave the whole sshd setup there, which is otherwise harmless, but don't fail if guest-key.pub is missing in the current directory. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- test/passt.mbuto | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/passt.mbuto b/test/passt.mbuto index d29f456..51c9ada 100755 --- a/test/passt.mbuto +++ b/test/passt.mbuto @@ -68,7 +68,7 @@ EOF ln -s /run/sshd /usr/share/empty.sshd cat > /root/.ssh/authorized_keys <<EOF -'"$(cat guest-key.pub)"' +'"$(cat guest-key.pub 2>/dev/null || :)"' EOF chmod 600 /root/.ssh/authorized_keys chmod 700 /root -- 2.35.1

1 0

[PATCH 0/7] Fixes and workarounds for tests, Coverity warnings
by Stefano Brivio 23 Sep '22

23 Sep '22

With this series, I'm almost able to run the full test suite together with the new command dispatch mechanism. I still hit frequent failures in the passt_tcp performance test, so I'm not pushing this out at the moment, but as it's taking me a while, I'd rather share this earlier. Included are also two fixes for harmless (but ugly) issues discovered by Coverity. Stefano Brivio (7): test/perf: Always use /sbin/sysctl in tcp test test/perf: Switch performance test duration to 10 seconds instead of 30 tap: Check return value of accept4() before calling getsockopt() conf, tcp, udp: Arrays for ports need 2^16 values, not 2^16-8 test/lib: Restore IFS while executing directives in def blocks test/lib: Wait on iperf3 clients to be done, then send SIGINT to servers test/perf: Disable periodic throughput reports to avoid vhost hang conf.c | 2 +- tap.c | 6 ++++-- tcp.h | 4 ++-- test/lib/test | 20 ++++++++++---------- test/perf/passt_tcp | 10 +++++----- test/perf/passt_udp | 4 ++-- test/perf/pasta_tcp | 10 +++++----- test/perf/pasta_udp | 4 ++-- udp.h | 4 ++-- 9 files changed, 33 insertions(+), 31 deletions(-) -- 2.35.1

2 18

[PATCH 0/5] Some more hacks and workarounds to make tests pass again
by Stefano Brivio 23 Sep '22

23 Sep '22

Some of these are actual fixes, some are just hacks to keep tests running while we figure out issues. A part of this series should actually be reverted once we figure out why iperf3 clients and servers get stuck now and then. Stefano Brivio (5): test/lib: Run also iperf3 clients in background, revert to time-based wait test/lib: Wait for kernel to free up ports used by iperf3 before reusing them test/perf: Wait for neper servers in guest to be ready before starting client test/lib: Wait for DHCPv4 before starting DHCPv6 client in two_guests test test/distro: Update workarounds for Ubuntu 22.04 on s390x test/distro/ubuntu | 4 +++- test/lib/setup | 1 + test/lib/test | 11 ++++++++--- test/perf/passt_tcp | 4 ++++ test/perf/passt_udp | 2 ++ 5 files changed, 18 insertions(+), 4 deletions(-) -- 2.35.1

2 6

[PATCH] Makefile: Allow define overrides by prepending, not appending, CFLAGS
by Stefano Brivio 23 Sep '22

23 Sep '22

If we append CFLAGS to the ones passed via command line (if any), -D options we append will override -D options passed on command line (if any). For example, OpenSUSE build flags include -D_FORTIFY_SOURCE=3, and we want to have -D_FORTIFY_SOURCE=2, if and only if not overridden. The current behaviour implies we redefine _FORTIFY_SOURCE as 2, though. Instead of appending CFLAGS, prepend them by adding all the default build flags to another variable, a simply expanded one (defined with :=), named FLAGS, and pass that *before* CFLAGS in targets, so that defines from command line can override default flags. Reported-by: Dario Faggioli <dfaggioli(a)suse.com> Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- Makefile | 49 +++++++++++++++++++++++++------------------------ 1 file changed, 25 insertions(+), 24 deletions(-) diff --git a/Makefile b/Makefile index af3d1ff..6805c23 100644 --- a/Makefile +++ b/Makefile @@ -23,13 +23,13 @@ AUDIT_ARCH := $(shell echo $(AUDIT_ARCH) | sed 's/I[456]86/I386/') AUDIT_ARCH := $(shell echo $(AUDIT_ARCH) | sed 's/PPC64/PPC/') AUDIT_ARCH := $(shell echo $(AUDIT_ARCH) | sed 's/PPCLE/PPC64LE/') -CFLAGS += -Wall -Wextra -pedantic -std=c99 -D_XOPEN_SOURCE=700 -D_GNU_SOURCE -CFLAGS += -D_FORTIFY_SOURCE=2 -O2 -pie -fPIE -CFLAGS += -DPAGE_SIZE=$(shell getconf PAGE_SIZE) -CFLAGS += -DNETNS_RUN_DIR=\"/run/netns\" -CFLAGS += -DPASST_AUDIT_ARCH=AUDIT_ARCH_$(AUDIT_ARCH) -CFLAGS += -DRLIMIT_STACK_VAL=$(RLIMIT_STACK_VAL) -CFLAGS += -DARCH=\"$(TARGET_ARCH)\" +FLAGS := -Wall -Wextra -pedantic -std=c99 -D_XOPEN_SOURCE=700 -D_GNU_SOURCE +FLAGS += -D_FORTIFY_SOURCE=2 -O2 -pie -fPIE +FLAGS += -DPAGE_SIZE=$(shell getconf PAGE_SIZE) +FLAGS += -DNETNS_RUN_DIR=\"/run/netns\" +FLAGS += -DPASST_AUDIT_ARCH=AUDIT_ARCH_$(AUDIT_ARCH) +FLAGS += -DRLIMIT_STACK_VAL=$(RLIMIT_STACK_VAL) +FLAGS += -DARCH=\"$(TARGET_ARCH)\" PASST_SRCS = arch.c arp.c checksum.c conf.c dhcp.c dhcpv6.c icmp.c igmp.c \ isolation.c lineread.c mld.c ndp.c netlink.c packet.c passt.c pasta.c \ @@ -50,36 +50,36 @@ HEADERS = $(PASST_HEADERS) # from the pointer arithmetic used from the tcp_tap_handler() path to get the # remote connection address. ifeq ($(shell $(CC) -dumpversion),11) -ifneq (,$(filter -flto%,$(CFLAGS))) -ifneq (,$(filter -O2,$(CFLAGS))) - CFLAGS += -DTCP_HASH_NOINLINE - CFLAGS += -DSIPHASH_20B_NOINLINE +ifneq (,$(filter -flto%,$(FLAGS) $(CFLAGS))) +ifneq (,$(filter -O2,$(FLAGS) $(CFLAGS))) + FLAGS += -DTCP_HASH_NOINLINE + FLAGS += -DSIPHASH_20B_NOINLINE endif endif endif C := \#include <linux/tcp.h>\nstruct tcp_info x = { .tcpi_snd_wnd = 0 }; ifeq ($(shell printf "$(C)" | $(CC) -S -xc - -o - >/dev/null 2>&1; echo $$?),0) - CFLAGS += -DHAS_SND_WND + FLAGS += -DHAS_SND_WND endif C := \#include <linux/tcp.h>\nstruct tcp_info x = { .tcpi_bytes_acked = 0 }; ifeq ($(shell printf "$(C)" | $(CC) -S -xc - -o - >/dev/null 2>&1; echo $$?),0) - CFLAGS += -DHAS_BYTES_ACKED + FLAGS += -DHAS_BYTES_ACKED endif C := \#include <linux/tcp.h>\nstruct tcp_info x = { .tcpi_min_rtt = 0 }; ifeq ($(shell printf "$(C)" | $(CC) -S -xc - -o - >/dev/null 2>&1; echo $$?),0) - CFLAGS += -DHAS_MIN_RTT + FLAGS += -DHAS_MIN_RTT endif C := \#include <sys/random.h>\nint main(){int a=getrandom(0, 0, 0);} ifeq ($(shell printf "$(C)" | $(CC) -S -xc - -o - >/dev/null 2>&1; echo $$?),0) - CFLAGS += -DHAS_GETRANDOM + FLAGS += -DHAS_GETRANDOM endif ifeq ($(shell :|$(CC) -fstack-protector-strong -S -xc - -o - >/dev/null 2>&1; echo $$?),0) - CFLAGS += -fstack-protector-strong + FLAGS += -fstack-protector-strong endif prefix ?= /usr/local @@ -98,18 +98,19 @@ endif all: $(BIN) $(MANPAGES) docs -static: CFLAGS += -static -DGLIBC_NO_STATIC_NSS +static: FLAGS += -static -DGLIBC_NO_STATIC_NSS static: clean all seccomp.h: $(PASST_SRCS) $(PASST_HEADERS) @ EXTRA_SYSCALLS=$(EXTRA_SYSCALLS) ./seccomp.sh $^ passt: $(PASST_SRCS) $(PASST_HEADERS) seccomp.h - $(CC) $(CFLAGS) $(PASST_SRCS) -o passt $(LDFLAGS) + $(CC) $(FLAGS) $(CFLAGS) $(PASST_SRCS) -o passt $(LDFLAGS) -passt.avx2: CFLAGS += -Ofast -mavx2 -ftree-vectorize -funroll-loops +passt.avx2: FLAGS += -Ofast -mavx2 -ftree-vectorize -funroll-loops passt.avx2: $(PASST_SRCS) $(PASST_HEADERS) seccomp.h - $(CC) $(filter-out -O2,$(CFLAGS)) $(PASST_SRCS) -o passt.avx2 $(LDFLAGS) + $(CC) $(filter-out -O2,$(FLAGS) $(CFLAGS)) \ + $(PASST_SRCS) -o passt.avx2 $(LDFLAGS) passt.avx2: passt @@ -117,12 +118,12 @@ pasta.avx2 pasta.1 pasta: pasta%: passt% ln -s $< $@ qrap: $(QRAP_SRCS) passt.h - $(CC) $(CFLAGS) $(QRAP_SRCS) -o qrap $(LDFLAGS) + $(CC) $(FLAGS) $(CFLAGS) $(QRAP_SRCS) -o qrap $(LDFLAGS) valgrind: EXTRA_SYSCALLS="rt_sigprocmask rt_sigtimedwait rt_sigaction \ getpid gettid kill clock_gettime mmap munmap open \ unlink gettimeofday futex" -valgrind: CFLAGS:=-g -O0 $(filter-out -O%,$(CFLAGS)) +valgrind: FLAGS:=-g -O0 $(filter-out -O%,$(FLAGS)) valgrind: all .PHONY: clean @@ -261,7 +262,7 @@ clang-tidy: $(SRCS) $(HEADERS) -altera-struct-pack-align,\ -concurrency-mt-unsafe \ -config='{CheckOptions: [{key: bugprone-suspicious-string-compare.WarnOnImplicitComparison, value: "false"}]}' \ - --warnings-as-errors=* $(SRCS) -- $(filter-out -pie,$(CFLAGS)) + --warnings-as-errors=* $(SRCS) -- $(filter-out -pie,$(FLAGS) $(CFLAGS)) ifeq ($(shell $(CC) -v 2>&1 | grep -c "gcc version"),1) TARGET := $(shell ${CC} -v 2>&1 | sed -n 's/Target: $.*$/\1/p') @@ -304,5 +305,5 @@ cppcheck: $(SRCS) $(HEADERS) --suppress=unmatchedSuppression:udp.c \ --suppress=unmatchedSuppression:util.c \ --suppress=unmatchedSuppression:util.h \ - $(filter -D%,$(CFLAGS)) \ + $(filter -D%,$(FLAGS) $(CFLAGS)) \ . -- 2.35.1

2 3

all pending patches pushed
by Stefano Brivio 23 Sep '22

23 Sep '22

Tests finally run reliably again, so I pushed all the pending patches. Sorry for the delay. Updated CI videos and test logs will appear in a while. -- Stefano

1 0

[PATCH 0/2] test: Fix two issues made apparent by new command dispatch
by Stefano Brivio 19 Sep '22

19 Sep '22

This is based on pending series by David, but they're logically separated fixes, so I'm sending them as a different series. Patch 2/2 adds some 'sleep 1' commands that were previously missing, which actually goes against the intention of speeding up those tests. However, without those, a number of tests fail quite frequently due to races between starting server and client, and I could never get a clean run with the new command dispatch mechanism otherwise. We can probably find a more elegant (and time-saving) mechanism to make sure the server is ready (based e.g. on parsing its output), but at the moment I'm just trying to make the test suite work again for me in its entirety. Stefano Brivio (2): test/perf: Check for /sbin/sysctl with which(1), not simply sysctl test/passt_in_ns: Consistent sleep commands before starting socat client test/passt_in_ns/tcp | 29 +++++++++++++++++++++++++---- test/passt_in_ns/udp | 18 ++++++++++++++++-- test/perf/passt_tcp | 4 ++-- test/perf/passt_udp | 2 +- test/perf/pasta_tcp | 2 +- 5 files changed, 45 insertions(+), 10 deletions(-) -- 2.35.1

2 13

[PATCH 0/2] doc/demo: Fix two issues seen on Fedora 36
by Stefano Brivio 17 Sep '22

17 Sep '22

Stefano Brivio (2): doc/demo: Drop /sbin from dhclient command, pass script file explicitly doc/demo: Clone and use mbuto in init namespace doc/demo.sh | 30 +++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-) -- 2.35.1

2 5

[PATCH] README: Update Availability and Try It sections with new packages
by Stefano Brivio 17 Sep '22

17 Sep '22

We now have official packages for Fedora, unofficial (Fedora Copr) for other common RPM-based distributions, and the existing packages with static builds for Debian, and for other RPM-based distributions. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- README.md | 57 +++++++++++++++++++++++++++++++------------------------ 1 file changed, 32 insertions(+), 25 deletions(-) diff --git a/README.md b/README.md index 2d7ae3c..fb09c6a 100644 --- a/README.md +++ b/README.md @@ -339,15 +339,19 @@ speeding up local connections, and usually requiring NAT. _pasta_: * ⌚ drop-in replacement for VPNKit (rootless Docker) ### Availability -* ✅ convenience unofficial packages for Debian, RPM-based distributions on - x86_64 (static builds) +* ✅ official [packages](https://src.fedoraproject.org/rpms/passt) for Fedora +* ✅ unofficial + [packages](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) for CentOS + Stream, EPEL, Mageia, OpenSUSE +* ✅ unofficial packages from x86_64 static builds for other RPM-based + distributions and Debian * ✅ testing on non-x86_64 architectures (aarch64, armv7l, i386, ppc64, ppc64le, s390x) -* ✅ example Debian [package files](/passt/tree/contrib/debian), - example [`spec` file](/passt/tree/contrib/fedora) for Fedora +* ✅ example Debian [package files](/passt/tree/contrib/debian) * 🛠 official [OpenSUSE packages](https://build.opensuse.org/package/show/home:mnhauke/passt) -* ⌚ official packages for Debian, Fedora, etc. +* ⌚ official packages for Debian + ([RFP](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1010498)), Arch Linux ### Services * ✅ built-in [ARP proxy](/passt/tree/arp.c) @@ -530,16 +534,19 @@ See also the [test logs](/builds/latest/test/). cd passt make - * alternatively, [static builds](/builds/latest/x86_64/) for x86_64 as of - the latest commit are also available for convenience. Non-official - packages for Debian (and derivatives) and RPM-based distributions are also - available there. These binaries and packages are simply built with: + * alternatively, install one of the available packages: - make pkgs + * [Debian and Debian-based](/builds/latest/x86_64/) (unofficial, from static x86_64 builds) + * [Fedora](https://src.fedoraproject.org/rpms/passt) (official) + * [CentOS Stream](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) (unofficial) + * [EPEL](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) (unofficial) + * [Mageia](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) (unofficial) + * [OpenSUSE](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) (unofficial) + * [Other RPM-based distributions](/builds/latest/x86_64/) (unofficial, from static x86_64 builds) - * or use one of the unofficial - [repositories](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) for - CentOS Stream, EPEL, Fedora, Mageia and OpenSUSE Tumbleweed + Static binaries and packages are simply built with: + + make pkgs * have a look at the _man_ page for synopsis and options: @@ -565,10 +572,7 @@ See also the [test logs](/builds/latest/test/). echo working from/to the guest for IPv4 and IPv6 * to connect to a service on the VM, just connect to the same port directly - with the address of the network namespace. For example, to ssh to the guest, - from the main namespace on the host: - - ssh 192.0.2.2 + with the address of the current network namespace ### pasta @@ -578,16 +582,19 @@ See also the [test logs](/builds/latest/test/). cd passt make - * alternatively, [static builds](/builds/latest/x86_64/) for x86_64 as of - the latest commit are also available for convenience. Non-official - packages for Debian (and derivatives) and RPM-based distributions are also - available there. These binaries and packages are simply built with: + * alternatively, install one of the available packages: + + * [Debian and Debian-based](/builds/latest/x86_64/) (unofficial, from static x86_64 builds) + * [Fedora](https://src.fedoraproject.org/rpms/passt) (official) + * [CentOS Stream](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) (unofficial) + * [EPEL](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) (unofficial) + * [Mageia](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) (unofficial) + * [OpenSUSE](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) (unofficial) + * [Other RPM-based distributions](/builds/latest/x86_64/) (unofficial, from static x86_64 builds) - make pkgs + Static binaries and packages are simply built with: - * or use one of the unofficial - [repositories](https://copr.fedorainfracloud.org/coprs/sbrivio/passt/) for - CentOS Stream, EPEL, Fedora, Mageia and OpenSUSE Tumbleweed + make pkgs * have a look at the _man_ page for synopsis and options: -- 2.35.1

1 0