- dev - passt

[PATCH] test: Add external mbuto profile, drop udhcpc, and switch to it
by Stefano Brivio 02 Jul '22

02 Jul '22

This depends on a future change in mbuto to accept external profile files. Add a file defining what we need for tests and demos, dropping udhcpc, plus udhcpc and dhclient scripts as they're not needed anymore, and switch to it. Suggested-by: David Gibson <david(a)gibson.dropbear.id.au> Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- test/demo/passt | 2 +- test/lib/setup | 2 +- test/passt.mbuto | 34 ++++++++++++++++++++++++++++++++++ 3 files changed, 36 insertions(+), 2 deletions(-) create mode 100755 test/passt.mbuto diff --git a/test/demo/passt b/test/demo/passt index 0ece04f..b67ed26 100644 --- a/test/demo/passt +++ b/test/demo/passt @@ -56,7 +56,7 @@ say Let's create a small initramfs image for the guest. guest cd __TEMPDIR__ guest git clone git://mbuto.sh/mbuto guest cd mbuto -guest ./mbuto -f passt.img -p passt -c lz4 +guest ./mbuto -f passt.img -p __TEMPDIR__/passt/test/passt.mbuto -c lz4 sleep 2 nl diff --git a/test/lib/setup b/test/lib/setup index 0b6d915..cfbac3c 100755 --- a/test/lib/setup +++ b/test/lib/setup @@ -34,7 +34,7 @@ setup_passt() { pane_run GUEST "git -C ${__mbuto_dir} clone git://mbuto.sh/mbuto" pane_status GUEST - pane_run GUEST "${__mbuto_dir}/mbuto/mbuto -p passt -c lz4 -f mbuto.img" + pane_run GUEST "${__mbuto_dir}/mbuto/mbuto -p test/passt.mbuto -c lz4 -f mbuto.img" pane_status GUEST rm -rf "${__mbuto_dir}" diff --git a/test/passt.mbuto b/test/passt.mbuto new file mode 100755 index 0000000..5cafb30 --- /dev/null +++ b/test/passt.mbuto @@ -0,0 +1,34 @@ +#!/bin/sh +# +# SPDX-License-Identifier: AGPL-3.0-or-later +# +# PASST - Plug A Simple Socket Transport +# for qemu/UNIX domain socket mode +# +# test/passt.mbuto - mbuto (https://mbuto.sh) profile for test images +# +# Copyright (c) 2022 Red Hat GmbH +# Author: Stefano Brivio <sbrivio(a)redhat.com> + +PROGS="${PROGS:-ash,dash,bash ip mount ls insmod mkdir ln cat chmod lsmod + modprobe find grep mknod mv rm umount jq iperf3 dhclient busybox logger + sed tr chown sipcalc cut md5sum nc dd strace ping tail killall sleep + sysctl nproc tcp_rr tcp_crr udp_rr which tee seq bc}" + +KMODS="${KMODS:- virtio_net virtio_pci}" + +LINKS="${LINKS:- + ash,dash,bash /init + ash,dash,bash /bin/sh + ash,dash,bash /usr/bin/bash}" + +DIRS="${DIRS} /tmp" + +FIXUP="${FIXUP} + :> /etc/fstab + sh +m +" + +OUTPUT="KERNEL=__KERNEL__ +INITRD=__INITRD__ +" -- 2.35.1

2 3

[PATCH 0/5] Test fixes, batch 3
by David Gibson 28 Jun '22

28 Jun '22

Here's another batch of fixes to get more passt/pasta tests running on my system, this time focused on the distro test cases. A number of those tests now run on my Fedora 36 host. Unfortunately they still fail on the Fedora 30, aarch64 test for reasons I haven't figured out (one of the yum downloads times out with passt but not -net user). This series is based on my earlier series adding the new "lineread" subsystem. David Gibson (5): tests: qemu-system-ppc64le isn't a thing Invoke specific qemu-system-* binaries tests: Cache downloaded guest images Handle the case of a DNS server on localhost tests: Try multiple paths for aarch64 EDK2 bios image conf.c | 16 +++++++ passt.1 | 6 +-- test/.gitignore | 1 + test/demo/passt | 2 +- test/distro/debian | 60 +++++++++++--------------- test/distro/fedora | 100 +++++++++++++++++++------------------------ test/distro/opensuse | 23 +++++----- test/distro/ubuntu | 25 +++++------ test/lib/setup | 15 +++++-- test/lib/test | 12 ++++++ test/run | 4 ++ 11 files changed, 140 insertions(+), 124 deletions(-) -- 2.36.1

1 6

[PATCH 0/4] New line reading implementation
by David Gibson 24 Jun '22

24 Jun '22

As discussed on our call, the current line_read() implementation is pretty ugly and has some definite bugs. This series replaces it with a new implementation which should be more solid and more readable. The new implementation is larger than I'd really like, but it turns out its fiddlier to handle the various edge cases than you might think. David Gibson (4): Add cleaner line-by-line reading primitives Parse resolv.conf with new lineread implementation Use new lineread implementation for procfs_scan_listen() Remove unused line_read() Makefile | 8 ++-- conf.c | 22 +++++++---- lineread.c | 108 +++++++++++++++++++++++++++++++++++++++++++++++++++++ lineread.h | 23 ++++++++++++ util.c | 64 +++---------------------------- 5 files changed, 155 insertions(+), 70 deletions(-) create mode 100644 lineread.c create mode 100644 lineread.h -- 2.36.1

2 6

Weird timeout issue with passt
by David Gibson 23 Jun '22

23 Jun '22

I alluded to this in my last patchset, but here's the information I've gathered on the current problem I'm hitting running the passt distro tests. It's pretty weird. On at least two occasions the tests have stalled during the Fedora 30, aarch64 test, with the guest getting a timeout downloading the package lists before installing new packages. For me this is selecting the mirror mirror.2degrees.nz. I'm not sure which factors are are relevant to reproducing the problem though. * The problem seems to me that the download suddenly stops progressing * partway through, causing dnf to eventually time out * If I manually try a "dnf clean all && dnf makecache -v" using the same guest image, it doesn't fail every time, but it fails significantly more often than not * It doesn't fail on the same file every time * I haven't been able to reproduce manually downloading the failing file with curl (tried repeatedly) * If I restrict dnf to a single repository rather than the whole set, I haven't managed to reproduce the problem * If I use qemu's -net user slirp instead of passt with the same disk image , I haven't been able to reproduce the problem (tried a bunch of times) * I've reproduced with the guest using both IPv4 and IPv6 * I have reproduced what looks like the same problem with an x86 guest image under KVM (also Fedora 30), but it seems to happen much less often (seen once in 10 or more attempts) * Seems to reproduce fairly readily with an x86 guest under TCG though, so I'm guessing the difference is timing related. -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson

2 1

[PATCH 0/6] Assorted makefile cleanups
by David Gibson 20 Jun '22

20 Jun '22

These are not directly apropose anything, but I encountered a number of minor uglies in the Makefiles while I was working on other stuff, so heres a bunch of cleanups. David Gibson (6): Makefile: Avoid using wildcard sources Makefile: Use $(BIN) and $(MANPAGES) variable to simplify several targets Makefile: Simplify pasta* targets with a pattern rule Makefile: Tweak $(RM) usage Makefile: Don't create extraneous -.s file Makefile: Spell prefix as PREFIX Makefile | 77 +++++++++++++++++++++++++++--------------------------- seccomp.sh | 5 ++-- 2 files changed, 41 insertions(+), 41 deletions(-) -- 2.36.1

2 11

[PATCH 0/5] More test fixes for Fedora host
by David Gibson 18 Jun '22

18 Jun '22

Hi Stefano, Here's another batch of fixes for the passt/pasta tests. With this lot I'm now able to get the build and pasta tests passing on my Fedora workstation. The passt and distro tests still fail or crash, I'm continuing to look at those (it looks like there are some very fragile behaviours in qrap which means they're not even getting started). David Gibson (5): tests: Use more explicit netcat options for distro/fedora tests Don't abbreviate ip(8) arguments in examples and tests Tweak dhclient arguments for readability Use dhclient instead of udhcpc tests: Use nmap-ncat instead of openbsd netcat for pasta tests contrib/kata-containers/README.md | 2 +- doc/demo.sh | 8 ++--- passt.1 | 16 +++------ test/README.md | 2 +- test/demo/passt | 16 ++++----- test/demo/pasta | 24 ++++++------- test/demo/podman | 20 +++++------ test/dhcp/passt | 24 ++++++------- test/dhcp/pasta | 26 +++++++------- test/distro/fedora | 14 ++++---- test/distro/ubuntu | 4 +-- test/icmp/passt_in_ns | 4 +-- test/lib/setup | 12 +++---- test/ndp/passt | 12 +++---- test/ndp/pasta | 12 +++---- test/perf/passt_tcp | 8 ++--- test/perf/passt_udp | 8 ++--- test/perf/pasta_tcp | 6 ++-- test/perf/pasta_udp | 6 ++-- test/tcp/passt | 6 ++-- test/tcp/passt_in_ns | 22 ++++++------ test/tcp/pasta | 60 +++++++++++++++---------------- test/two_guests/basic | 26 +++++++------- test/udp/passt | 6 ++-- test/udp/passt_in_ns | 14 ++++---- test/udp/pasta | 35 +++++++++--------- 26 files changed, 193 insertions(+), 200 deletions(-) -- 2.36.1

2 11

[PATCH] tap: Add informational messages for UNIX domain socket connections
by Stefano Brivio 17 Jun '22

17 Jun '22

...namely, as connections are discarded or accepted. This was quite useful to debug an issue with libvirtd failing to start qemu (because passt refused the new connection) as a previous qemu instance was still active. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- tap.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/tap.c b/tap.c index 04ceade..23414db 100644 --- a/tap.c +++ b/tap.c @@ -856,12 +856,19 @@ static void tap_sock_unix_new(struct ctx *c) { struct epoll_event ev = { 0 }; int v = INT_MAX / 2; + struct ucred ucred; + socklen_t len; + + len = sizeof(ucred); /* Another client is already connected: accept and close right away. */ if (c->fd_tap != -1) { int discard = accept4(c->fd_tap_listen, NULL, NULL, SOCK_NONBLOCK); + if (!getsockopt(discard, SOL_SOCKET, SO_PEERCRED, &ucred, &len)) + info("discardingq connection from PID %i", ucred.pid); + if (discard != -1) close(discard); @@ -870,6 +877,9 @@ static void tap_sock_unix_new(struct ctx *c) c->fd_tap = accept4(c->fd_tap_listen, NULL, NULL, 0); + if (!getsockopt(c->fd_tap, SOL_SOCKET, SO_PEERCRED, &ucred, &len)) + info("accepted connection from PID %i", ucred.pid); + if (!c->low_rmem && setsockopt(c->fd_tap, SOL_SOCKET, SO_RCVBUF, &v, sizeof(v))) trace("tap: failed to set SO_RCVBUF to %i", v); -- 2.35.1

2 1

[PATCH] conf: In conf_runas(), on static builds, group information is also unused
by Stefano Brivio 16 Jun '22

16 Jun '22

Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- conf.c | 1 + 1 file changed, 1 insertion(+) diff --git a/conf.c b/conf.c index d615cf5..3a5d650 100644 --- a/conf.c +++ b/conf.c @@ -868,6 +868,7 @@ static int conf_runas(const char *opt, unsigned int *uid, unsigned int *gid) (void)ubuf; (void)gbuf; (void)pw; + (void)gr; return -EINVAL; #else -- 2.35.1

1 0

[PATCH] qrap: Add probe retry on connection reset from passt for KubeVirt integration
by Stefano Brivio 16 Jun '22

16 Jun '22

KubeVirt uses libvirt to start qrap in its current draft integration (https://github.com/kubevirt/kubevirt/pull/7849/), and libvirtd starts qrap three times every time a new virtual machine is created: once on domain creation, and twice on domain start (for "probing") and to finally start it for real. Very often, a subsequent invocation of qrap happen before the previously running instance of qemu terminates, which means that passt will refuse the new connection as the old one is still active. Introduce a single retry with a 100ms delay to work around this. This should be checked again once native libvirt support is there, and that point qrap will have no reason to exist anymore. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- qrap.c | 34 +++++++++++++++++++++++++++++++++- 1 file changed, 33 insertions(+), 1 deletion(-) diff --git a/qrap.c b/qrap.c index 17cc472..303e981 100644 --- a/qrap.c +++ b/qrap.c @@ -25,6 +25,7 @@ #include <net/if_arp.h> #include <netinet/in.h> #include <netinet/if_ether.h> +#include <time.h> #include "util.h" #include "passt.h" @@ -112,7 +113,7 @@ void usage(const char *name) int main(int argc, char **argv) { struct timeval tv = { .tv_sec = 0, .tv_usec = (long)(500 * 1000) }; - int i, s, qemu_argc = 0, addr_map = 0, has_dev = 0; + int i, s, qemu_argc = 0, addr_map = 0, has_dev = 0, retry_on_reset; char *qemu_argv[ARG_MAX], dev_str[ARG_MAX]; struct sockaddr_un addr = { .sun_family = AF_UNIX, @@ -233,6 +234,9 @@ int main(int argc, char **argv) valid_args: for (i = 1; i < UNIX_SOCK_MAX; i++) { + retry_on_reset = 1; + +retry: s = socket(AF_UNIX, SOCK_STREAM, 0); if (s < 0) { perror("socket"); @@ -254,6 +258,34 @@ valid_args: else break; + /* FIXME: in a KubeVirt environment, libvirtd invokes qrap three + * times in a strict sequence when a virtual machine needs to + * be started, namely, when: + * - the domain XML is saved + * - the domain is started (for "probing") + * - the virtual machine is started for real + * and it often happens that the qemu process is still running + * when qrap is invoked again, so passt will refuse the new + * connection because the previous one is still active. This + * overlap seems to be anywhere between 0 and 3ms. + * + * If we get a connection reset, retry, just once, after 100ms, + * to allow for the previous qemu instance to terminate and, in + * turn, for the connection to passt to be closed. + * + * This should be fixed in libvirt instead. It probably makes + * sense to check this behaviour once native libvirt support is + * there, and this implies native qemu support too, so at that + * point qrap will have no reason to exist anymore -- that is, + * this FIXME will probably remain until the tool itself is + * obsoleted. + */ + if (retry_on_reset && errno == ECONNRESET) { + retry_on_reset = 0; + usleep(100 * 1000); + goto retry; + } + fprintf(stderr, "Probe of %s failed\n", addr.sun_path); close(s); -- 2.35.1

1 0

passt & mbuto
by David Gibson 16 Jun '22

16 Jun '22

Hi again, I realized I wasn't quite right when I said that qrap problems where what was currently stopping me running the passt (not pasta) tests. I did hit qrap issues somewhere, but the current stumbling block is that mbuto looks for udhcpc to put into the guest image, which I can't easily put onto my host system. Now, in the short term, once my patch to remove usage of udhcpc from the passt/pasta tests is applied, we could just remove udhcpc from the mbuto profile as well. However, that raises a wider scope issue: The passt testing profile for mbuto appliances is in the mbuto tree, not the passt tree. That doesn't realy make sense, since it means any change to what we need for the passt tests requires a synchronized change with mbuto. Particularly for a pretty young and project like passt, that's not really tenable. Plus, slurping an external tool from some random URL in the tests is just kinda ugly. I'm not immediately sure how best to to address this: * We could make mbuto take the profiles as some sort of external file, so they can be provided by the user, rather than built into the mbuto repository. * We could just fork a copy of mbuto into the passt tree, making local modifications for the profile, and only manually updating it to match upstream mbuto changes. * We could use an entirely different and more established tool for building our testing guest images in passt (e.g. supermin, buildroot or just picking a standard distro guest image) -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson

2 4

2024

2023

2022

2021

dev