Ugly as hell, but we keep breaking things otherwise, and I keep forgetting to run this manually (as long as it's based on my local Podman setup, that's the only alternative). We need to clone the Podman repository as distribution packages don't contain test scripts, typically. While at it, build the latest version which is what really matters. As we're planning anyway to revamp the test framework, I'd be inclined to just add this without too many thoughts, and have it as a nice-to-have requirement reminder for the new framework. Link: https://github.com/containers/podman/pull/19699 Suggested-by: Paul Holzinger <pholzing(a)redhat.com> Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- test/README.md | 4 ++-- test/pasta_podman/bats | 22 ++++++++++++++++++++++ test/run | 4 ++++ 3 files changed, 28 insertions(+), 2 deletions(-) create mode 100644 test/pasta_podman/bats diff --git a/test/README.md b/test/README.md index 03c7f57..0936b04 100644 --- a/test/README.md +++ b/test/README.md @@ -28,8 +28,8 @@ on a system, i.e. common utilities such as a shell are not included here. Example for Debian, and possibly most Debian-based distributions: - build-essential git jq strace iperf3 qemu-system-x86 tmux sipcalc bc - clang-tidy cppcheck isc-dhcp-common psmisc linux-cpupower socat + build-essential git jq strace iperf3 qemu-system-x86 tmux sipcalc bats bc + catatonit clang-tidy cppcheck go isc-dhcp-common psmisc linux-cpupower socat netcat-openbsd fakeroot lz4 lm-sensors qemu-system-arm qemu-system-ppc qemu-system-misc qemu-system-x86 valgrind diff --git a/test/pasta_podman/bats b/test/pasta_podman/bats new file mode 100644 index 0000000..f36da7c --- /dev/null +++ b/test/pasta_podman/bats @@ -0,0 +1,22 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# +# PASST - Plug A Simple Socket Transport +# for qemu/UNIX domain socket mode +# +# PASTA - Pack A Subtle Tap Abstraction +# for network namespace/tap device mode +# +# test/pasta_podman/bats - Build Podman, run pasta system test with bats +# +# Copyright (c) 2022 Red Hat GmbH +# Author: Stefano Brivio <sbrivio(a)redhat.com> + +htools git make go bats catatonit ip jq socat + +test Podman system test with bats + +host git -C __STATEDIR__ clone https://github.com/containers/podman.git +host make -C __STATEDIR__/podman +hout WD pwd +host printf "[engine]\nhelper_binaries_dir=['__WD__']\n" > __STATEDIR__/containers.conf +host PODMAN="__STATEDIR__/podman/bin/podman" CONTAINERS_CONF_OVERRIDE="__STATEDIR__/containers.conf" bats __STATEDIR__/podman/test/system/505-networking-pasta.bats diff --git a/test/run b/test/run index 8f4f845..3b37663 100755 --- a/test/run +++ b/test/run @@ -82,6 +82,10 @@ run() { test pasta_options/log_to_file teardown pasta_options + setup build + test pasta_podman/bats + teardown build + setup memory test memory/passt teardown memory -- 2.39.2
On 23/08/2023 11:35, Stefano Brivio wrote:Ugly as hell, but we keep breaking things otherwise, and I keep forgetting to run this manually (as long as it's based on my local Podman setup, that's the only alternative). We need to clone the Podman repository as distribution packages don't contain test scripts, typically. While at it, build the latest version which is what really matters.We actually ship a podman-tests rpm which contains `test/system` from the podman repo. Of course this will contain the test version based on the current shipped version and not the main branch. But if you want to the with latest you have to clone from github of course.As we're planning anyway to revamp the test framework, I'd be inclined to just add this without too many thoughts, and have it as a nice-to-have requirement reminder for the new framework. Link: https://github.com/containers/podman/pull/19699 Suggested-by: Paul Holzinger <pholzing(a)redhat.com> Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- test/README.md | 4 ++-- test/pasta_podman/bats | 22 ++++++++++++++++++++++ test/run | 4 ++++ 3 files changed, 28 insertions(+), 2 deletions(-) create mode 100644 test/pasta_podman/bats diff --git a/test/README.md b/test/README.md index 03c7f57..0936b04 100644 --- a/test/README.md +++ b/test/README.md @@ -28,8 +28,8 @@ on a system, i.e. common utilities such as a shell are not included here. Example for Debian, and possibly most Debian-based distributions: - build-essential git jq strace iperf3 qemu-system-x86 tmux sipcalc bc - clang-tidy cppcheck isc-dhcp-common psmisc linux-cpupower socat + build-essential git jq strace iperf3 qemu-system-x86 tmux sipcalc bats bc + catatonit clang-tidy cppcheck go isc-dhcp-common psmisc linux-cpupower socat netcat-openbsd fakeroot lz4 lm-sensors qemu-system-arm qemu-system-ppc qemu-system-misc qemu-system-x86 valgrind diff --git a/test/pasta_podman/bats b/test/pasta_podman/bats new file mode 100644 index 0000000..f36da7c --- /dev/null +++ b/test/pasta_podman/bats @@ -0,0 +1,22 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# +# PASST - Plug A Simple Socket Transport +# for qemu/UNIX domain socket mode +# +# PASTA - Pack A Subtle Tap Abstraction +# for network namespace/tap device mode +# +# test/pasta_podman/bats - Build Podman, run pasta system test with bats +# +# Copyright (c) 2022 Red Hat GmbH +# Author: Stefano Brivio <sbrivio(a)redhat.com> + +htools git make go bats catatonit ip jq socat + +test Podman system test with bats + +host git -C __STATEDIR__ clone https://github.com/containers/podman.git +host make -C __STATEDIR__/podman +hout WD pwd +host printf "[engine]\nhelper_binaries_dir=['__WD__']\n" > __STATEDIR__/containers.conf +host PODMAN="__STATEDIR__/podman/bin/podman" CONTAINERS_CONF_OVERRIDE="__STATEDIR__/containers.conf" bats __STATEDIR__/podman/test/system/505-networking-pasta.batsI recommend to use the `CONTAINERS_HELPER_BINARY_DIR` to set the directory over containers.conf, with your containers.conf change you will unset all other helper directories. This may be fine for the pasta specific tests for now as we do not use other binaries such as netavark there but in the future it could change and leaves you wondering what broke.diff --git a/test/run b/test/run index 8f4f845..3b37663 100755 --- a/test/run +++ b/test/run @@ -82,6 +82,10 @@ run() { test pasta_options/log_to_file teardown pasta_options + setup build + test pasta_podman/bats + teardown build + setup memory test memory/passt teardown memory
On Wed, 23 Aug 2023 14:09:13 +0200 Paul Holzinger <pholzing(a)redhat.com> wrote:On 23/08/2023 11:35, Stefano Brivio wrote:Not that I have a particular motivation... but podman-tests doesn't exist on, say, Debian. Still, I actually have a slight preference to test with the latest HEAD, yes, so that we can unblock things quickly in case.Ugly as hell, but we keep breaking things otherwise, and I keep forgetting to run this manually (as long as it's based on my local Podman setup, that's the only alternative). We need to clone the Podman repository as distribution packages don't contain test scripts, typically. While at it, build the latest version which is what really matters.We actually ship a podman-tests rpm which contains `test/system` from the podman repo. Of course this will contain the test version based on the current shipped version and not the main branch. But if you want to the with latest you have to clone from github of course.Hah, thanks, that's how it's called, I couldn't find it. I'll post a new version.As we're planning anyway to revamp the test framework, I'd be inclined to just add this without too many thoughts, and have it as a nice-to-have requirement reminder for the new framework. Link: https://github.com/containers/podman/pull/19699 Suggested-by: Paul Holzinger <pholzing(a)redhat.com> Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- test/README.md | 4 ++-- test/pasta_podman/bats | 22 ++++++++++++++++++++++ test/run | 4 ++++ 3 files changed, 28 insertions(+), 2 deletions(-) create mode 100644 test/pasta_podman/bats diff --git a/test/README.md b/test/README.md index 03c7f57..0936b04 100644 --- a/test/README.md +++ b/test/README.md @@ -28,8 +28,8 @@ on a system, i.e. common utilities such as a shell are not included here. Example for Debian, and possibly most Debian-based distributions: - build-essential git jq strace iperf3 qemu-system-x86 tmux sipcalc bc - clang-tidy cppcheck isc-dhcp-common psmisc linux-cpupower socat + build-essential git jq strace iperf3 qemu-system-x86 tmux sipcalc bats bc + catatonit clang-tidy cppcheck go isc-dhcp-common psmisc linux-cpupower socat netcat-openbsd fakeroot lz4 lm-sensors qemu-system-arm qemu-system-ppc qemu-system-misc qemu-system-x86 valgrind diff --git a/test/pasta_podman/bats b/test/pasta_podman/bats new file mode 100644 index 0000000..f36da7c --- /dev/null +++ b/test/pasta_podman/bats @@ -0,0 +1,22 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# +# PASST - Plug A Simple Socket Transport +# for qemu/UNIX domain socket mode +# +# PASTA - Pack A Subtle Tap Abstraction +# for network namespace/tap device mode +# +# test/pasta_podman/bats - Build Podman, run pasta system test with bats +# +# Copyright (c) 2022 Red Hat GmbH +# Author: Stefano Brivio <sbrivio(a)redhat.com> + +htools git make go bats catatonit ip jq socat + +test Podman system test with bats + +host git -C __STATEDIR__ clone https://github.com/containers/podman.git +host make -C __STATEDIR__/podman +hout WD pwd +host printf "[engine]\nhelper_binaries_dir=['__WD__']\n" > __STATEDIR__/containers.conf +host PODMAN="__STATEDIR__/podman/bin/podman" CONTAINERS_CONF_OVERRIDE="__STATEDIR__/containers.conf" bats __STATEDIR__/podman/test/system/505-networking-pasta.batsI recommend to use the `CONTAINERS_HELPER_BINARY_DIR` to set the directory over containers.conf, with your containers.conf change you will unset all other helper directories.This may be fine for the pasta specific tests for now as we do not use other binaries such as netavark there but in the future it could change and leaves you wondering what broke.Sure, I didn't like that either. -- Stefano