This series, mostly from David, cleans up a number of atrocities in
the test suite, makes it more observable by reporting test failures
whenever a command fails, and a bit more robust by improving filtering
in pane_parse().
Further, the whole script now fails if an unhandled failure occurs. A
few assorted fixes are included as well.
David Gibson (9):
tests: Remove unused XVFB variable
tests: Add some debugging output for the test scripts themselves
tests: Don't use tmux update-environment
tests: Don't globally set tmux default-shell
tests: Improve control character filtering in pane_parse
tests: Don't ignore errors during script
tests: Add pane_status command to check for success of issued commands
tests: Simplify *tools commands using pane_status
tests: Simplify explicit checks for command success
Stefano Brivio (3):
tests: Update mbuto git URLs
test/perf/pasta_udp: Drop redundant assignment of ::1 to loopback
interface
test/run: Return 0 from run(), exit value already reflects failures
test/build/static_checkers | 6 ++--
test/demo/passt | 2 +-
test/distro/debian | 33 +++++++------------
test/distro/opensuse | 18 ++++-------
test/distro/ubuntu | 15 +++------
test/icmp/passt_in_ns | 4 ---
test/lib/setup | 59 ++++++++++++++++------------------
test/lib/term | 30 ++++++++++++++----
test/lib/test | 65 ++++++++++++++++++--------------------
test/perf/pasta_udp | 1 -
test/run | 7 ++--
11 files changed, 108 insertions(+), 132 deletions(-)
--
2.35.1
The project is now at mbuto.sh, and git transport is enabled. Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com> --- test/demo/passt | 2 +- test/lib/setup | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/test/demo/passt b/test/demo/passt index 8838363..cf75c9b 100644 --- a/test/demo/passt +++ b/test/demo/passt @@ -54,7 +54,7 @@ nl nl say Let's create a small initramfs image for the guest. guest cd __TEMPDIR__ -guest git clone https://mbuto.lameexcu.se/mbuto +guest git clone git://mbuto.sh/mbuto guest cd mbuto guest ./mbuto -f passt.img -p passt -c lz4 sleep 2 diff --git a/test/lib/setup b/test/lib/setup index 823c706..b954596 100755 --- a/test/lib/setup +++ b/test/lib/setup @@ -31,7 +31,7 @@ setup_passt() { layout_passt __mbuto_dir="$(mktemp -d)" - pane_run GUEST "git -C ${__mbuto_dir} clone https://mbuto.lameexcu.se/mbuto/" + pane_run GUEST "git -C ${__mbuto_dir} clone git://mbuto.sh/mbuto" pane_wait GUEST pane_run GUEST "${__mbuto_dir}/mbuto/mbuto -p passt -c lz4 -f mbuto.img" -- 2.35.1
From: David Gibson <david(a)gibson.dropbear.id.au>
The XVFB variable is initialized at the beginning of test/run then never
used again. I'm assuming it's a leftover from some ealier iteration.
Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
---
test/run | 3 ---
1 file changed, 3 deletions(-)
diff --git a/test/run b/test/run
index b2819ef..a8e1c98 100755
--- a/test/run
+++ b/test/run
@@ -35,9 +35,6 @@ DEBUG=${DEBUG:-0}
# If set, tell passt and pasta to take packet captures
PCAP=${PCAP:-0}
-# If set, run X terminals for CI and demos in Xvfb
-XVFB=${XVFB:-1}
-
COMMIT="$(git log --oneline --no-decorate -1)"
. lib/util
--
2.35.1
From: David Gibson <david(a)gibson.dropbear.id.au>
The DEBUG option for tests/run enables debugging options to passt/pasta,
however that doesn't help with debugging the test scripts themselves, which
are fairly fragile.
Extend the DEBUG option so it also prints information on each command in
the test scripts to make it easier to work out where things are falling
over.
Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
---
test/lib/test | 2 ++
1 file changed, 2 insertions(+)
diff --git a/test/lib/test b/test/lib/test
index 2854191..93c86e8 100755
--- a/test/lib/test
+++ b/test/lib/test
@@ -89,6 +89,8 @@ test_iperf3() {
test_one_line() {
__line="${1}"
+ [ ${DEBUG} -eq 1 ] && info DEBUG: "${__line}"
+
# Strip comments
__line="${__line%%#*}"
--
2.35.1
From: David Gibson <david(a)gibson.dropbear.id.au>
The semantics of tmux's update-environment option are a bit confusing.
It says it means the given variables are copied into the session
environment from the source environment, but it's not entirely clear
what the "source" environment means.
From my experimentation it appeast to be the environment from which
the tmux *server* is launched, not the one issuing the 'new-session'
command. That makes it pretty much useles, certainly in our case where
we have no way of knowing if the user has pre-existing tmux sessions.
Instead use the new-session -e option to explicitly pass in the variables
we want to propagate.
Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
---
test/lib/term | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/test/lib/term b/test/lib/term
index c9f5be3..42b0020 100755
--- a/test/lib/term
+++ b/test/lib/term
@@ -544,18 +544,19 @@ pause_continue() {
run_term() {
export SHELL="/bin/sh"
tmux set-option -g default-shell "/bin/sh"
- tmux set-option -g update-environment "PCAP DEBUG"
+
+ TMUX="tmux new-session -s passt_test -ePCAP=$PCAP -eDEBUG=$DEBUG"
if [ ${CI} -eq 1 ]; then
printf '\e[8;50;240t'
- asciinema rec --overwrite ci.uncut -c 'tmux new-session -s passt_test ./ci
from_term'
+ asciinema rec --overwrite ci.uncut -c "$TMUX ./ci from_term"
video_postprocess ci.uncut
elif [ ${DEMO} -eq 1 ]; then
printf '\e[8;40;130t'
- asciinema rec --overwrite demo.uncut -c 'tmux new-session -s passt_test ./run_demo
from_term'
+ asciinema rec --overwrite demo.uncut -c "$TMUX ./run_demo from_term"
video_postprocess demo.uncut
else
- tmux new-session -s passt_test ./run from_term
+ $TMUX ./run from_term
fi
}
--
2.35.1
From: David Gibson <david(a)gibson.dropbear.id.au>
run_term() uses tmux set-option -g to globally set the default shell.
Unfortunately this hits a chicken-and-egg problem that's common with many
of tmux's session options. If there isn't already a tmux server running,
we can't connect to set the option. If we attempt this after starting our
session (and therefore the server), then the session will already be
started with the previous default shell.
In any case it's not a good idea to set tmux global options, since that
might interfere with whatever else the user is doing in tmux. So, instead
set the default-shell option locally to the session after starting it. To
make sure we get the right shell for our initial script, explicitly invoke
/bin/sh to interpret it.
Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
---
test/lib/term | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/test/lib/term b/test/lib/term
index 42b0020..7dd9826 100755
--- a/test/lib/term
+++ b/test/lib/term
@@ -542,26 +542,25 @@ pause_continue() {
# run_term() - Start tmux session, running entry point, with recording if needed
run_term() {
- export SHELL="/bin/sh"
- tmux set-option -g default-shell "/bin/sh"
-
TMUX="tmux new-session -s passt_test -ePCAP=$PCAP -eDEBUG=$DEBUG"
if [ ${CI} -eq 1 ]; then
printf '\e[8;50;240t'
- asciinema rec --overwrite ci.uncut -c "$TMUX ./ci from_term"
+ asciinema rec --overwrite ci.uncut -c "$TMUX /bin/sh -c './ci
from_term'"
video_postprocess ci.uncut
elif [ ${DEMO} -eq 1 ]; then
printf '\e[8;40;130t'
- asciinema rec --overwrite demo.uncut -c "$TMUX ./run_demo from_term"
+ asciinema rec --overwrite demo.uncut -c "$TMUX /bin/sh -c './run_demo
from_term'"
video_postprocess demo.uncut
else
- $TMUX ./run from_term
+ $TMUX /bin/sh -c './run from_term'
fi
}
# term() - Set up terminal window and panes for regular tests or CI
term() {
+ tmux set-option default-shell "/bin/sh"
+
tmux set status-interval 1
tmux rename-window ''
@@ -596,6 +595,8 @@ term() {
# term_demo() - Set up terminal window and panes for demo
term_demo() {
+ tmux set-option default-shell "/bin/sh"
+
tmux set status-interval 1
tmux rename-window ''
--
2.35.1
From: David Gibson <david(a)gibson.dropbear.id.au>
pane_parse() attempts to grab the output from the last command issued
into a tmux pane. It strips out control characters using tr, which in
particular includes the final \r\n. However, this won't fully strip
out terminal escape sequences. In particular this breaks if the shell
in the pane is bash, with enable-bracketed-paste enabled in readline.
That issues terminal sequences to enable and disable bracketed paste
mode around every shell prompt.
We can work around this because these escapes are followed by a \r
(CR). More generally, it seems reasonable to assume that any terminal
shenanigans followed by a CR, but not an LF is supposed to be hidden.
So, use sed to strip everything before the second last CR. We still
need the tr to remove the final \r\n from the string (sed processes a
line at a time, and doesn't consider the CRLF part of the buffer it's
processing).
Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
[sbrivio: modify regexp to keep foo\r\r\n unchanged, by matching on at
least one CR and a non-CR afterwards: that's the usual output pattern
for bash on Debian 8 and Debian 9]
Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com>
---
test/lib/term | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/test/lib/term b/test/lib/term
index 7dd9826..0093f90 100755
--- a/test/lib/term
+++ b/test/lib/term
@@ -221,7 +221,7 @@ pane_wait() {
pane_parse() {
__pane_lc="$(echo "${1}" | tr [A-Z] [a-z])"
- __buf="$(tail -n2 ${LOGDIR}/pane_${__pane_lc}.log | head -n1 | tr -d -c
[:print:])"
+ __buf="$(tail -n2 ${LOGDIR}/pane_${__pane_lc}.log | head -n1 | sed
's/^[^\r]*\r\([^\r]\)/\1/' | tr -d '\r\n')"
[ "# $(eval printf '%s' \"\$${1}_LAST_CMD\")" !=
"${__buf}" ] && \
[ "$ $(eval printf '%s' \"\$${1}_LAST_CMD\")" !=
"${__buf}" ] &&
--
2.35.1
From: David Gibson <david(a)gibson.dropbear.id.au>
Most commands issued during the testing scripts aren't explicitly checked
for errors. Therefore, if they fail, the shell will just keep on
executing. This makes it difficult to figure out where things started
going wrong if things fall over.
Run the whole script with the set -e mode so that it will exit in the case
of any (unchecked) failing command. To make this work we do need to add
explicit checks / fallbacks for some commands which we expect to fail.
Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
[sbrivio: use sh -e instead of setting -e later, so that we don't miss
anything before set -e is issued]
Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com>
---
test/lib/test | 10 +++++-----
test/run | 2 +-
2 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/test/lib/test b/test/lib/test
index 93c86e8..2cfec9a 100755
--- a/test/lib/test
+++ b/test/lib/test
@@ -153,7 +153,7 @@ test_one_line() {
"htools")
pane_run HOST 'which '"${__arg}"' >/dev/null || echo
skip'
pane_wait HOST
- [ "$(pane_parse HOST)" = "skip" ] && TEST_ONE_skip=1
+ [ "$(pane_parse HOST)" = "skip" ] && TEST_ONE_skip=1 ||
true
;;
"passt")
pane_run PASST "${__arg}"
@@ -194,17 +194,17 @@ test_one_line() {
"gtools")
pane_run GUEST 'which '"${__arg}"' >/dev/null || echo
skip'
pane_wait GUEST
- [ "$(pane_parse GUEST)" = "skip" ] && TEST_ONE_skip=1
+ [ "$(pane_parse GUEST)" = "skip" ] && TEST_ONE_skip=1 ||
true
;;
"g1tools")
pane_run GUEST_1 'which '"${__arg}"' >/dev/null || echo
skip'
pane_wait GUEST_1
- [ "$(pane_parse GUEST_1)" = "skip" ] && TEST_ONE_skip=1
+ [ "$(pane_parse GUEST_1)" = "skip" ] && TEST_ONE_skip=1 ||
true
;;
"g2tools")
pane_run GUEST_2 'which '"${__arg}"' >/dev/null || echo
skip'
pane_wait GUEST_2
- [ "$(pane_parse GUEST_2)" = "skip" ] && TEST_ONE_skip=1
+ [ "$(pane_parse GUEST_2)" = "skip" ] && TEST_ONE_skip=1 ||
true
;;
"guest2")
pane_run GUEST_2 "${__arg}"
@@ -249,7 +249,7 @@ test_one_line() {
"nstools")
pane_run NS 'which '"${__arg}"' >/dev/null || echo
skip'
pane_wait NS
- [ "$(pane_parse NS)" = "skip" ] && TEST_ONE_skip=1
+ [ "$(pane_parse NS)" = "skip" ] && TEST_ONE_skip=1 || true
;;
"gout")
__varname="${__arg%% *}"
diff --git a/test/run b/test/run
index a8e1c98..737fa81 100755
--- a/test/run
+++ b/test/run
@@ -1,4 +1,4 @@
-#!/bin/sh
+#!/bin/sh -e
#
# SPDX-License-Identifier: AGPL-3.0-or-later
#
--
2.35.1
From: David Gibson <david(a)gibson.dropbear.id.au>
When we use pane_wait to wait for a command issued to a tmux pane to finish
we have no idea whether the command succeeded or not. This means that the
test scripts can keep running long after the point something vital has
failed, making it difficult to work out what went wrong.
Add a new pane_status command that checks for success of the issued command
and use it in most places instead of pane_wait. We still need explicit
pane_wait where we're gathering explicit output with pane_parse, because
the way we check the status with 'echo $?' means we lose track of that
output.
Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
[sbrivio:
- instead of quitting the script, make a test fail if a command
issued in a pane fails during a test, and loop until the status code is
numeric in pane_status() as a hack to make it a bit more robust
- retain usage of pane_wait() in iperf3 and teardown functions as we
interrupt iperf3, passt, and pasta, so a non-zero exit code is expected
- drop bogus ns_{1,2}_wait() calls in teardown_two_guests(), those
functions were never implemented
- use pane_status() for "guest" test directives too
]
Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com>
---
test/lib/setup | 57 +++++++++++++++++++++++---------------------------
test/lib/term | 14 +++++++++++++
test/lib/test | 38 ++++++++++++++++-----------------
3 files changed, 59 insertions(+), 50 deletions(-)
diff --git a/test/lib/setup b/test/lib/setup
index b954596..6139e8f 100755
--- a/test/lib/setup
+++ b/test/lib/setup
@@ -32,10 +32,10 @@ setup_passt() {
__mbuto_dir="$(mktemp -d)"
pane_run GUEST "git -C ${__mbuto_dir} clone git://mbuto.sh/mbuto"
- pane_wait GUEST
+ pane_status GUEST
pane_run GUEST "${__mbuto_dir}/mbuto/mbuto -p passt -c lz4 -f mbuto.img"
- pane_wait GUEST
+ pane_status GUEST
rm -rf "${__mbuto_dir}"
@@ -52,9 +52,9 @@ setup_passt() {
[ ${TRACE} -eq 1 ] && __opts="${__opts} --trace"
pane_run PASST "make clean"
- pane_wait PASST
+ pane_status PASST
pane_run PASST "make valgrind"
- pane_wait PASST
+ pane_status PASST
pane_run PASST "valgrind --max-stackframe=$((4 * 1024 * 1024)) --trace-children=yes
--vgdb=no --error-exitcode=1 --suppressions=test/valgrind.supp ./passt ${__opts} -f -t
10001 -u 10001 -P passt.pid"
sleep 5
@@ -66,7 +66,7 @@ setup_passt() {
'virtio-net.napi_tx=1"' \
" -device virtio-net-pci,netdev=hostnet0,x-txburst=16384" \
" -netdev socket,fd=5,id=hostnet0"
- pane_wait GUEST
+ pane_status GUEST
}
# setup_pasta() - Create a network and user namespace, connect pasta to it
@@ -80,7 +80,7 @@ setup_pasta() {
__tty_pid="$(pane_parse NS)"
pane_run NS "unshare -rUnpf /bin/sh"
- pane_wait NS
+ pane_status NS
pane_run PASST "pstree -p ${__tty_pid} | sed -n
's/.*(\([0-9].*\))$/\1/p'"
pane_wait PASST
@@ -131,24 +131,24 @@ setup_passt_in_ns() {
pane_run PASST "./pasta ${__opts} -t 10001,10002,10011,10012 -T 10003,10013 -u
10001,10002,10011,10012 -U 10003,10013 -P ${__pid_file}"
sleep 1
pane_run PASST ''
- pane_wait PASST
+ pane_status PASST
__pasta_pid="$(cat "${__pid_file}")"
__ns_pid="$(cat /proc/${__pasta_pid}/task/${__pasta_pid}/children | cut -f1
-d' ')"
rm "${__pid_file}"
pane_run GUEST "nsenter -t ${__ns_pid} -U -n --preserve-credentials"
pane_run NS "nsenter -t ${__ns_pid} -U -n -p --preserve-credentials"
- pane_wait GUEST
- pane_wait NS
+ pane_status GUEST
+ pane_status NS
pane_run NS "ip -j li sh | jq -rM '.[] | select(.link_type ==
\"ether\").ifname'"
pane_wait NS
__ifname="$(pane_parse NS)"
pane_run NS "/sbin/udhcpc -i ${__ifname}"
- pane_wait NS
+ pane_status NS
sleep 2
pane_run NS "/sbin/dhclient -6 ${__ifname}"
- pane_wait NS
+ pane_status NS
__opts=
[ ${PCAP} -eq 1 ] && __opts="${__opts} -p /tmp/passt_in_pasta.pcap"
@@ -157,15 +157,15 @@ setup_passt_in_ns() {
if [ ${VALGRIND} -eq 1 ]; then
pane_run PASST "make clean"
- pane_wait PASST
+ pane_status PASST
pane_run PASST "make valgrind"
- pane_wait PASST
+ pane_status PASST
pane_run PASST "valgrind --max-stackframe=$((4 * 1024 * 1024))
--trace-children=yes --vgdb=no --error-exitcode=1 --suppressions=test/valgrind.supp
./passt -f ${__opts} -t 10001,10011,10021,10031 -u 10001,10011,10021,10031 -P
passt.pid"
else
pane_run PASST "make clean"
- pane_wait PASST
+ pane_status PASST
pane_run PASST "make"
- pane_wait PASST
+ pane_status PASST
pane_run PASST "./passt -f ${__opts} -t 10001,10011,10021,10031 -u
10001,10011,10021,10031 -P passt.pid"
fi
sleep 5
@@ -178,7 +178,7 @@ setup_passt_in_ns() {
'virtio-net.napi_tx=1"' \
" -device virtio-net-pci,netdev=hostnet0,x-txburst=524288" \
" -netdev socket,fd=5,id=hostnet0"
- pane_wait GUEST
+ pane_status GUEST
}
# setup_two_guests() - Set up two namespace, run qemu and passt in both of them
@@ -216,8 +216,8 @@ setup_two_guests() {
pane_run PASST_1 ''
pane_run PASST_2 ''
- pane_wait PASST_1
- pane_wait PASST_2
+ pane_status PASST_1
+ pane_status PASST_2
__pasta1_pid="$(cat "${__pid1_file}")"
__ns1_pid="$(cat /proc/${__pasta1_pid}/task/${__pasta1_pid}/children | cut -f1
-d' ')"
rm "${__pid1_file}"
@@ -234,13 +234,13 @@ setup_two_guests() {
pane_run GUEST_1 "/sbin/udhcpc -i ${__ifname}"
pane_run GUEST_2 "/sbin/udhcpc -i ${__ifname}"
- pane_wait GUEST_1
- pane_wait GUEST_2
+ pane_status GUEST_1
+ pane_status GUEST_2
sleep 2
pane_run GUEST_1 "/sbin/dhclient -6 ${__ifname}"
pane_run GUEST_2 "/sbin/dhclient -6 ${__ifname}"
- pane_wait GUEST_1
- pane_wait GUEST_2
+ pane_status GUEST_1
+ pane_status GUEST_2
__opts=
[ ${PCAP} -eq 1 ] && __opts="${__opts} -p /tmp/passt_1.pcap"
@@ -257,7 +257,7 @@ setup_two_guests() {
pane_run PASST_2 "./passt -f ${__opts} -t 10004 -u 10004"
pane_run GUEST_2 'cp mbuto.img mbuto_2.img'
- pane_wait GUEST_2
+ pane_status GUEST_2
pane_run GUEST_1 './qrap 5 kvm -m '${VMEM}' -cpu host -smp '${VCPUS}
\
' -kernel ' "/boot/vmlinuz-$(uname -r)" \
@@ -275,8 +275,8 @@ setup_two_guests() {
'virtio-net.napi_tx=1"' \
" -device virtio-net-pci,netdev=hostnet0,x-txburst=16384" \
" -netdev socket,fd=5,id=hostnet0"
- pane_wait GUEST_1
- pane_wait GUEST_2
+ pane_status GUEST_1
+ pane_status GUEST_2
}
# teardown_passt() - Kill qemu, remove passt PID file
@@ -301,7 +301,7 @@ teardown_passt_in_ns() {
tmux send-keys -t ${PANE_GUEST} "C-d"
[ ${VALGRIND} -eq 0 ] && tmux send-keys -t ${PANE_PASST} "C-c"
- [ ${VALGRIND} -eq 0 ] && pane_wait GUEST
+ [ ${VALGRIND} -eq 0 ] && pane_status GUEST
tmux send-keys -t ${PANE_PASST} "C-d"
tmux send-keys -t ${PANE_NS} "C-d"
@@ -331,13 +331,8 @@ teardown_two_guests() {
pane_wait PASST_2
tmux send-keys -t ${PANE_PASST_2} "C-d"
- tmux send-keys -t ${PANE_NS_1} "C-d"
- tmux send-keys -t ${PANE_NS_2} "C-d"
-
pane_wait GUEST_1
pane_wait GUEST_2
- ns_1_wait
- ns_2_wait
pane_wait PASST_1
pane_wait PASST_2
}
diff --git a/test/lib/term b/test/lib/term
index 0093f90..004c2a2 100755
--- a/test/lib/term
+++ b/test/lib/term
@@ -228,6 +228,20 @@ pane_parse() {
printf '%s' "${__buf}" || printf '@EMPTY@'
}
+# pane_status() - Wait for command to complete and return its exit status
+# $1: Pane name
+pane_status() {
+ pane_wait "${1}"
+ __status="$(pane_parse "${1}")"
+ while ! [ "${__status}" -eq "${__status}" ]; do
+ sleep 1
+ pane_run "${1}" 'echo $?'
+ pane_wait "${1}"
+ __status="$(pane_parse "${1}")"
+ done
+ return ${__status}
+}
+
# status_file_end() - Display and log messages when tests from one file are done
status_file_end() {
[ -z "${STATUS_FILE}" ] && return
diff --git a/test/lib/test b/test/lib/test
index 2cfec9a..4262343 100755
--- a/test/lib/test
+++ b/test/lib/test
@@ -52,7 +52,7 @@ test_iperf3() {
pane_run "${__pane}" 'for i in $(seq 0 '${__procs}');
do' \
':> s${i}.bw; done'
- pane_wait "${__pane}"
+ pane_status "${__pane}"
if [ ${UDP_CLIENT} -eq 0 ]; then
pane_run "${__pane}" 'for i in $(seq 0 '${__procs}');' \
@@ -70,7 +70,7 @@ test_iperf3() {
'done'
fi
- pane_wait "${__pane}"
+ pane_status "${__pane}"
sleep 45
pane_run "${__pane}" 'for i in $(seq 0 '${__procs}');
do' \
'kill -INT $(cat s${i}.pid) 2>/dev/null; done'
@@ -83,7 +83,7 @@ test_iperf3() {
pane_parse "${__pane}"
pane_run "${__pane}" 'for i in $(seq 0 '${__procs}');
do' \
'rm -f s${i}.bw; done'
- pane_wait "${__pane}"
+ pane_status "${__pane}"
}
test_one_line() {
@@ -139,13 +139,13 @@ test_one_line() {
;;
"host")
pane_run HOST "${__arg}"
- pane_wait HOST
+ pane_status HOST || TEST_ONE_nok=1
;;
"hostb")
pane_run HOST "${__arg}"
;;
"hostw")
- pane_wait HOST
+ pane_status HOST || TEST_ONE_nok=1
;;
"hint")
tmux send-keys -t ${PANE_HOST} "C-c"
@@ -157,13 +157,13 @@ test_one_line() {
;;
"passt")
pane_run PASST "${__arg}"
- pane_wait PASST
+ pane_status PASST || TEST_ONE_nok=1
;;
"passtb")
pane_run PASST "${__arg}"
;;
"passtw")
- pane_wait PASST
+ pane_status PASST || TEST_ONE_nok=1
;;
"pout")
__varname="${__arg%% *}"
@@ -173,23 +173,23 @@ test_one_line() {
;;
"guest")
pane_run GUEST "${__arg}"
- pane_wait GUEST
+ pane_status GUEST || TEST_ONE_nok=1
;;
"guestb")
pane_run GUEST "${__arg}"
;;
"guestw")
- pane_wait GUEST
+ pane_status GUEST || TEST_ONE_nok=1
;;
"guest1")
pane_run GUEST_1 "${__arg}"
- pane_wait GUEST_1
+ pane_status GUEST_1 || TEST_ONE_nok=1
;;
"guest1b")
pane_run GUEST_1 "${__arg}"
;;
"guest1w")
- pane_wait GUEST_1
+ pane_status GUEST_1 || TEST_ONE_nok=1
;;
"gtools")
pane_run GUEST 'which '"${__arg}"' >/dev/null || echo
skip'
@@ -208,25 +208,25 @@ test_one_line() {
;;
"guest2")
pane_run GUEST_2 "${__arg}"
- pane_wait GUEST_2
+ pane_status GUEST_2 || TEST_ONE_nok=1
;;
"guest2b")
pane_run GUEST_2 "${__arg}"
;;
"guest2w")
- pane_wait GUEST_2
+ pane_status GUEST_2 || TEST_ONE_nok=1
;;
"ns")
pane_run NS "${__arg}"
- pane_wait NS
+ pane_status NS || TEST_ONE_nok=1
;;
"ns1")
pane_run NS1 "${__arg}"
- pane_wait NS1
+ pane_status NS1 || TEST_ONE_nok=1
;;
"ns2")
pane_run NS2 "${__arg}"
- pane_wait NS2
+ pane_status NS2 || TEST_ONE_nok=1
;;
"nsb")
pane_run NS "${__arg}"
@@ -238,13 +238,13 @@ test_one_line() {
pane_run NS2 "${__arg}"
;;
"nsw")
- pane_wait NS
+ pane_status NS || TEST_ONE_nok=1
;;
"ns1w")
- pane_wait NS1
+ pane_status NS1 || TEST_ONE_nok=1
;;
"ns2w")
- pane_wait NS2
+ pane_status NS2 || TEST_ONE_nok=1
;;
"nstools")
pane_run NS 'which '"${__arg}"' >/dev/null || echo
skip'
--
2.35.1
From: David Gibson <david(a)gibson.dropbear.id.au>
Now that we have pane_status to check the success of commands issued to
panes, we can more easily check for the success of the 'which' commands
used to check tool availability, rather than constructing, then parsing
special "skip" output.
Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
---
test/lib/test | 25 ++++++++++---------------
1 file changed, 10 insertions(+), 15 deletions(-)
diff --git a/test/lib/test b/test/lib/test
index 4262343..a5583e3 100755
--- a/test/lib/test
+++ b/test/lib/test
@@ -151,9 +151,8 @@ test_one_line() {
tmux send-keys -t ${PANE_HOST} "C-c"
;;
"htools")
- pane_run HOST 'which '"${__arg}"' >/dev/null || echo
skip'
- pane_wait HOST
- [ "$(pane_parse HOST)" = "skip" ] && TEST_ONE_skip=1 ||
true
+ pane_run HOST 'which '"${__arg}"' >/dev/null'
+ pane_status HOST || TEST_ONE_skip=1
;;
"passt")
pane_run PASST "${__arg}"
@@ -192,19 +191,16 @@ test_one_line() {
pane_status GUEST_1 || TEST_ONE_nok=1
;;
"gtools")
- pane_run GUEST 'which '"${__arg}"' >/dev/null || echo
skip'
- pane_wait GUEST
- [ "$(pane_parse GUEST)" = "skip" ] && TEST_ONE_skip=1 ||
true
+ pane_run GUEST 'which '"${__arg}"' >/dev/null'
+ pane_status GUEST || TEST_ONE_skip=1
;;
"g1tools")
- pane_run GUEST_1 'which '"${__arg}"' >/dev/null || echo
skip'
- pane_wait GUEST_1
- [ "$(pane_parse GUEST_1)" = "skip" ] && TEST_ONE_skip=1 ||
true
+ pane_run GUEST_1 'which '"${__arg}"' >/dev/null'
+ pane_status GUEST_1 || TEST_ONE_skip=1
;;
"g2tools")
- pane_run GUEST_2 'which '"${__arg}"' >/dev/null || echo
skip'
- pane_wait GUEST_2
- [ "$(pane_parse GUEST_2)" = "skip" ] && TEST_ONE_skip=1 ||
true
+ pane_run GUEST_2 'which '"${__arg}"' >/dev/null'
+ pane_status GUEST_2 || TEST_ONE_skip=1
;;
"guest2")
pane_run GUEST_2 "${__arg}"
@@ -247,9 +243,8 @@ test_one_line() {
pane_status NS2 || TEST_ONE_nok=1
;;
"nstools")
- pane_run NS 'which '"${__arg}"' >/dev/null || echo
skip'
- pane_wait NS
- [ "$(pane_parse NS)" = "skip" ] && TEST_ONE_skip=1 || true
+ pane_run NS 'which '"${__arg}"' >/dev/null'
+ pane_status NS || TEST_ONE_skip=1
;;
"gout")
__varname="${__arg%% *}"
--
2.35.1
From: David Gibson <david(a)gibson.dropbear.id.au> A number of individual test cases use '*out' commands to check for success of specific commands they've issued. Now that the test harness is testing for success of all issued commands as a matter of course, we no longer need to do this. Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au> --- test/build/static_checkers | 6 ++---- test/distro/debian | 33 +++++++++++---------------------- test/distro/opensuse | 18 ++++++------------ test/distro/ubuntu | 15 +++++---------- test/icmp/passt_in_ns | 4 ---- 5 files changed, 24 insertions(+), 52 deletions(-) diff --git a/test/build/static_checkers b/test/build/static_checkers index d18dea8..34c8dff 100644 --- a/test/build/static_checkers +++ b/test/build/static_checkers @@ -14,9 +14,7 @@ htools clang-tidy cppcheck test Run clang-tidy -hout RET make clang-tidy; echo $? -check [ __RET__ -eq 0 ] +host make clang-tidy test Run cppcheck -hout RET make cppcheck; echo $? -check [ __RET__ -eq 0 ] +host make cppcheck diff --git a/test/distro/debian b/test/distro/debian index 344f453..f748dea 100644 --- a/test/distro/debian +++ b/test/distro/debian @@ -61,8 +61,7 @@ host apt-get update host apt-get -y install make gcc netcat-openbsd host make clean -hout RET CFLAGS="-Wno-missing-field-initializers -Wno-missing-braces -Wno-type-limits" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Wno-missing-field-initializers -Wno-missing-braces -Wno-type-limits" make # TODO: pasta test skipped for the moment: clone() as called by NS_CALL hangs # with wrapper provided by glibc 2.19, probably wrong argument order. @@ -89,8 +88,7 @@ host apt-get update host apt-get -y install make gcc netcat-openbsd host make clean -hout RET CFLAGS="-Werror" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Werror" make host sysctl -w kernel.unprivileged_userns_clone=1 distro_quick_pasta_test @@ -114,8 +112,7 @@ host apt-get update host apt-get -y install make gcc netcat-openbsd host make clean -hout RET CFLAGS="-Werror" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Werror" make host sysctl -w kernel.unprivileged_userns_clone=1 distro_quick_pasta_test @@ -146,8 +143,7 @@ host apt-get update host apt-get -y install make gcc netcat-openbsd host make clean -hout RET CFLAGS="-Werror" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Werror" make host sysctl -w kernel.unprivileged_userns_clone=1 distro_quick_pasta_test @@ -179,8 +175,7 @@ host apt-get update host apt-get -y install make gcc netcat-openbsd host make clean -hout RET CFLAGS="-Werror" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Werror" make host sysctl -w kernel.unprivileged_userns_clone=1 distro_quick_pasta_test @@ -204,8 +199,7 @@ host apt-get update host apt-get -y install make gcc netcat-openbsd host make clean -hout RET CFLAGS="-Werror" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Werror" make distro_quick_pasta_test @@ -234,8 +228,7 @@ host apt-get update host apt-get -y install make gcc netcat-openbsd host make clean -hout RET CFLAGS="-Werror" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Werror" make distro_quick_pasta_test @@ -265,8 +258,7 @@ host apt-get update host apt-get -y install make gcc netcat-openbsd host make clean -hout RET CFLAGS="-Werror" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Werror" make distro_quick_pasta_test @@ -289,8 +281,7 @@ host apt-get update host apt-get -y install make gcc netcat-openbsd host make clean -hout RET CFLAGS="-Werror" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Werror" make distro_quick_pasta_test @@ -311,8 +302,7 @@ host apt-get update host apt-get -y install make gcc netcat-openbsd host make clean -hout RET CFLAGS="-Werror" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Werror" make distro_quick_pasta_test @@ -334,8 +324,7 @@ host apt-get update host apt-get -y install make gcc netcat-openbsd host make clean -hout RET CFLAGS="-Werror" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Werror" make distro_quick_pasta_test diff --git a/test/distro/opensuse b/test/distro/opensuse index a2c0944..39f059a 100644 --- a/test/distro/opensuse +++ b/test/distro/opensuse @@ -60,8 +60,7 @@ host echo "DNSSERVERS='__DNS6__'" | netconfig modify -s dns_resolver -i eth0 host for i in $(seq 1 10); do zypper install -y gcc make netcat-openbsd && break; done; echo host make clean -hout RET CFLAGS="-Werror" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Werror" make distro_quick_pasta_test @@ -89,8 +88,7 @@ host echo "DNSSERVERS='__DNS6__'" | netconfig modify -s dns_resolver -i eth0 host for i in $(seq 1 10); do zypper install -y gcc make netcat-openbsd && break; done; echo host make clean -hout RET CFLAGS="-Werror" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Werror" make distro_quick_pasta_test @@ -117,8 +115,7 @@ host echo "DNSSERVERS='__DNS6__'" | netconfig modify -s dns_resolver -i eth0 host for i in $(seq 1 10); do zypper install -y gcc make netcat-openbsd && break; done; echo host make clean -hout RET CFLAGS="-Werror" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Werror" make distro_quick_pasta_test @@ -143,8 +140,7 @@ sleep 10 host zypper install -y gcc make netcat-openbsd; echo host make clean -hout RET CFLAGS="-Werror" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Werror" make distro_quick_pasta_test @@ -172,8 +168,7 @@ sleep 10 host zypper install -y gcc make netcat-openbsd; echo host make clean -hout RET CFLAGS="-Werror" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Werror" make distro_quick_pasta_test @@ -200,8 +195,7 @@ host echo "DNSSERVERS='__DNS6__'" | netconfig modify -s dns_resolver -i ens2 host for i in $(seq 1 10); do zypper install -y gcc make netcat-openbsd && break; done; echo host make clean -hout RET CFLAGS="-Werror" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Werror" make distro_quick_pasta_test diff --git a/test/distro/ubuntu b/test/distro/ubuntu index 781daab..3ce156d 100644 --- a/test/distro/ubuntu +++ b/test/distro/ubuntu @@ -62,8 +62,7 @@ host apt-get update host apt-get -y install make gcc netcat-openbsd host make clean -hout RET CFLAGS="-Wno-missing-field-initializers -Wno-missing-braces -Wno-type-limits" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Wno-missing-field-initializers -Wno-missing-braces -Wno-type-limits" make # TODO: pasta test skipped for the moment: clone() as called by NS_CALL hangs # with wrapper provided by glibc 2.19, probably wrong argument order. @@ -96,8 +95,7 @@ host apt-get update host apt-get -y install make gcc netcat-openbsd host make clean -hout RET CFLAGS="-Wno-missing-field-initializers -Wno-missing-braces -Wno-type-limits -Wno-sign-compare" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Wno-missing-field-initializers -Wno-missing-braces -Wno-type-limits -Wno-sign-compare" make # TODO: pasta test skipped for the moment: clone() as called by NS_CALL hangs # with wrapper provided by glibc 2.19, probably wrong argument order. @@ -127,8 +125,7 @@ host apt-get update host apt-get -y install make gcc netcat-openbsd host make clean -hout RET CFLAGS="-Wno-missing-field-initializers -Wno-missing-braces -Wno-type-limits -Wno-sign-compare" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Wno-missing-field-initializers -Wno-missing-braces -Wno-type-limits -Wno-sign-compare" make # TODO: pasta test skipped for the moment: clone() as called by NS_CALL hangs # with wrapper provided by glibc 2.19, probably wrong argument order. @@ -160,8 +157,7 @@ host dhclient host apt-get -y install make gcc netcat-openbsd host make clean -hout RET CFLAGS="-Werror" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Werror" make distro_quick_pasta_test @@ -195,8 +191,7 @@ host apt-get update host apt-get -y install make gcc netcat-openbsd host make clean -hout RET CFLAGS="-Werror" make; echo $? -check [ __RET__ -eq 0 ] +host CFLAGS="-Werror" make host export SHELL="/bin/dash" host dash diff --git a/test/icmp/passt_in_ns b/test/icmp/passt_in_ns index be5be29..38a9a4f 100644 --- a/test/icmp/passt_in_ns +++ b/test/icmp/passt_in_ns @@ -24,14 +24,10 @@ test ICMP echo: guest to ns nsout IFNAME_NS ip -j li sh | jq -rM '.[] | select(.link_type == "ether").ifname' ns ip addr add 192.0.2.1/32 dev __IFNAME_NS__ guest ping -c1 -w1 192.0.2.1 -gout RET echo $? ns ip addr del 192.0.2.1/32 dev __IFNAME_NS__ -check [ __RET__ -eq 0 ] test ICMPv6 echo: guest to ns ns ip addr add 2001:db8::1 dev __IFNAME_NS__ && sleep 2 # DAD gout IFNAME ip -j li sh | jq -rM '.[] | select(.link_type == "ether").ifname' guest ping -c1 -w1 2001:db8::1 -gout RET echo $? ns ip addr del 2001:db8::1 dev __IFNAME_NS__ -check [ __RET__ -eq 0 ] -- 2.35.1
There are a few occurrences of this assignment, which are needed to
re-add ::1 as loopback address after the MTU has been increased
back from a value below 1280 bytes.
This one, however, is redundant, and causes an error in the
execution.
Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com>
---
test/perf/pasta_udp | 1 -
1 file changed, 1 deletion(-)
diff --git a/test/perf/pasta_udp b/test/perf/pasta_udp
index 39a6e24..8b68b78 100644
--- a/test/perf/pasta_udp
+++ b/test/perf/pasta_udp
@@ -62,7 +62,6 @@ lat __LAT__ 200 150
tr UDP throughput over IPv4: ns to host
ns ip link set dev lo mtu 1500
-ns ip addr add ::1 dev lo
iperf3c ns 127.0.0.1 100${i}3 __THREADS__ __OPTS__ -b 3G
iperf3s BW host 100${i}3 __THREADS__
bw __BW__ 1.0 1.5
--
2.35.1
There's no need to return non-zero if there have been failures in
run(), because the exit value is already determined from the number
of failures reported in the log file.
Return zero, so that this doesn't cause the script to fail, given we
now run it with -e.
Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com>
---
test/run | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/test/run b/test/run
index 737fa81..d9e5107 100755
--- a/test/run
+++ b/test/run
@@ -110,7 +110,7 @@ run() {
"Interrupted, press any key to quit" \
9
- return ${STATUS_FAIL}
+ return 0
}
# demo() - Simpler path for demo purposes
--
2.35.1
On Tue, May 17, 2022 at 02:36:02PM +0200, Stefano Brivio wrote:This series, mostly from David, cleans up a number of atrocities in the test suite, makes it more observable by reporting test failures whenever a command fails, and a bit more robust by improving filtering in pane_parse(). Further, the whole script now fails if an unhandled failure occurs. A few assorted fixes are included as well.Unsurprisingly, these all LGTM. I notice they're not in the main git tree yet; is that intentional? -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
On Wed, 18 May 2022 13:17:31 +1000 David Gibson <david(a)gibson.dropbear.id.au> wrote:On Tue, May 17, 2022 at 02:36:02PM +0200, Stefano Brivio wrote:Yes, I post patches to the list and leave a bit of time in case anybody wants to review or comment. Also, the pre-push hook runs CI and demos so that takes a bit as well. -- StefanoThis series, mostly from David, cleans up a number of atrocities in the test suite, makes it more observable by reporting test failures whenever a command fails, and a bit more robust by improving filtering in pane_parse(). Further, the whole script now fails if an unhandled failure occurs. A few assorted fixes are included as well.Unsurprisingly, these all LGTM. I notice they're not in the main git tree yet; is that intentional?