[PATCH v4 0/5] Test cases to display memory usage in kernel and userspace

Stefano Brivio

3 Nov 2022 3 Nov '22

7:33 a.m.

Note that 2/5 is not actually needed for the purposes of these "test cases" (which always succeed, at the moment), it just fixes a condition I hit while writing them. v4: Fold changes by David: use a dummy interface for the purpose of starting passt, and drop setup and teardown steps related to the "outer" instance of passt, now gone. Install passt in /bin instead of /usr/bin, but also add a link v3: In 5/5, drop useless conditionals and teardown steps as pointed out by David v2: Actually include passt.mem.mbuto in 5/5, and move the introduction of INITRAMFS_MEM for the setup function there Stefano Brivio (5): test/lib: Move screen-scraping setup and layout functions to _ugly files tap: Support for detection of existing sockets on ramfs test/lib/perf_report: Use own flag to track initialisation test/lib: Add "td" directive, handled by table_value() test: Add memory/passt test cases tap.c | 3 +- test/Makefile | 5 +- test/lib/layout | 69 ---------------- test/lib/layout_ugly | 113 ++++++++++++++++++++++++++ test/lib/perf_report | 9 ++- test/lib/setup | 23 ------ test/lib/setup_ugly | 58 ++++++++++++++ test/lib/term | 28 +++++++ test/lib/test | 3 + test/memory/passt | 187 +++++++++++++++++++++++++++++++++++++++++++ test/passt.mem.mbuto | 44 ++++++++++ test/run | 6 ++ 12 files changed, 451 insertions(+), 97 deletions(-) create mode 100644 test/lib/layout_ugly create mode 100755 test/lib/setup_ugly create mode 100644 test/memory/passt create mode 100755 test/passt.mem.mbuto -- 2.35.1

Show replies by date

Stefano Brivio

3 Nov 3 Nov

7:33 a.m.

New subject: [PATCH v4 1/5] test/lib: Move screen-scraping setup and layout functions to _ugly files

I'm going to add yet another one of those, for which I have no quick solution. It's a regression in some sense, but at least if we make this regression more observable and defined, it should be easier to find a comprehensive solution later, within this or another testing framework. Signed-off-by: Stefano Brivio Reviewed-by: David Gibson --- test/lib/layout | 69 ------------------------------------ test/lib/layout_ugly | 83 ++++++++++++++++++++++++++++++++++++++++++++ test/lib/setup | 23 ------------ test/lib/setup_ugly | 38 ++++++++++++++++++++ test/run | 2 ++ 5 files changed, 123 insertions(+), 92 deletions(-) create mode 100644 test/lib/layout_ugly create mode 100755 test/lib/setup_ugly diff --git a/test/lib/layout b/test/lib/layout index fcd1db4..b190060 100644 --- a/test/lib/layout +++ b/test/lib/layout @@ -13,75 +13,6 @@ # Copyright (c) 2021 Red Hat GmbH # Author: Stefano Brivio -# layout_host() - Simple host commands layout with info and host panes -layout_host() { - sleep 3 - - tmux kill-pane -a -t 0 - cmd_write 0 clear - - tmux split-window -h -l '35%' -t passt_test:1.0 - - PANE_HOST=0 - PANE_INFO=1 - - get_info_cols - - tmux send-keys -l -t ${PANE_INFO} 'while cat '"$STATEBASE/log_pipe"'; do :; done' - tmux send-keys -t ${PANE_INFO} -N 100 C-m - tmux select-pane -t ${PANE_INFO} -T "test log" - - if context_exists host; then - pane_watch_contexts 0 host host - else - tmux pipe-pane -O -t ${PANE_HOST} "cat >> ${LOGDIR}/pane_host.log" - tmux select-pane -t ${PANE_HOST} -T "host" - fi - - info_layout "host commands only" - - sleep 1 -} - -# layout_pasta_simple() - Panes for host and pasta -layout_pasta_simple() { - sleep 3 - - tmux kill-pane -a -t 0 - cmd_write 0 clear - - tmux split-window -v -t passt_test - tmux split-window -h -t passt_test - - PANE_PASST=0 - PANE_HOST=1 - PANE_INFO=2 - - get_info_cols - - tmux send-keys -l -t ${PANE_INFO} 'while cat '"$STATEBASE/log_pipe"'; do :; done' - tmux send-keys -t ${PANE_INFO} -N 100 C-m - tmux select-pane -t ${PANE_INFO} -T "test log" - - if context_exists host; then - pane_watch_contexts ${PANE_HOST} host host - else - tmux pipe-pane -O -t ${PANE_HOST} "cat >> ${LOGDIR}/pane_host.log" - tmux select-pane -t ${PANE_HOST} -T "host" - fi - - if context_exists passt; then - pane_watch_contexts ${PANE_PASST} host host - else - tmux pipe-pane -O -t ${PANE_PASST} "cat >> ${LOGDIR}/pane_passt.log" - tmux select-pane -t ${PANE_PASST} -T "pasta" - fi - - info_layout "single pasta instance" - - sleep 1 -} - # layout_pasta() - Panes for host, pasta, and separate one for namespace layout_pasta() { sleep 3 diff --git a/test/lib/layout_ugly b/test/lib/layout_ugly new file mode 100644 index 0000000..9397b7d --- /dev/null +++ b/test/lib/layout_ugly @@ -0,0 +1,83 @@ +#!/bin/sh +# +# SPDX-License-Identifier: AGPL-3.0-or-later +# +# PASST - Plug A Simple Socket Transport +# for qemu/UNIX domain socket mode +# +# PASTA - Pack A Subtle Tap Abstraction +# for network namespace/tap device mode +# +# test/lib/layout_ugly - screen-scraped tmux pane layouts +# +# Copyright (c) 2022 Red Hat GmbH +# Author: Stefano Brivio + +# layout_host() - Simple host commands layout with info and host panes +layout_host() { + sleep 3 + + tmux kill-pane -a -t 0 + cmd_write 0 clear + + tmux split-window -h -l '35%' -t passt_test:1.0 + + PANE_HOST=0 + PANE_INFO=1 + + get_info_cols + + tmux send-keys -l -t ${PANE_INFO} 'while cat '"$STATEBASE/log_pipe"'; do :; done' + tmux send-keys -t ${PANE_INFO} -N 100 C-m + tmux select-pane -t ${PANE_INFO} -T "test log" + + if context_exists host; then + pane_watch_contexts 0 host host + else + tmux pipe-pane -O -t ${PANE_HOST} "cat >> ${LOGDIR}/pane_host.log" + tmux select-pane -t ${PANE_HOST} -T "host" + fi + + info_layout "host commands only" + + sleep 1 +} + +# layout_pasta_simple() - Panes for host and pasta +layout_pasta_simple() { + sleep 3 + + tmux kill-pane -a -t 0 + cmd_write 0 clear + + tmux split-window -v -t passt_test + tmux split-window -h -t passt_test + + PANE_PASST=0 + PANE_HOST=1 + PANE_INFO=2 + + get_info_cols + + tmux send-keys -l -t ${PANE_INFO} 'while cat '"$STATEBASE/log_pipe"'; do :; done' + tmux send-keys -t ${PANE_INFO} -N 100 C-m + tmux select-pane -t ${PANE_INFO} -T "test log" + + if context_exists host; then + pane_watch_contexts ${PANE_HOST} host host + else + tmux pipe-pane -O -t ${PANE_HOST} "cat >> ${LOGDIR}/pane_host.log" + tmux select-pane -t ${PANE_HOST} -T "host" + fi + + if context_exists passt; then + pane_watch_contexts ${PANE_PASST} host host + else + tmux pipe-pane -O -t ${PANE_PASST} "cat >> ${LOGDIR}/pane_passt.log" + tmux select-pane -t ${PANE_PASST} -T "pasta" + fi + + info_layout "single pasta instance" + + sleep 1 +} diff --git a/test/lib/setup b/test/lib/setup index e2d0ff0..ab6d8d5 100755 --- a/test/lib/setup +++ b/test/lib/setup @@ -26,11 +26,6 @@ setup_build() { layout_host } -# setup_distro() - Set up pane layout for distro tests -setup_distro() { - layout_host -} - # setup_passt() - Start qemu and passt setup_passt() { context_setup_host host @@ -106,13 +101,6 @@ setup_pasta() { wait_for [ -f "${STATESETUP}/passt.pid" ] } -# setup_pasta_options() - Set up layout and host context without starting pasta -setup_pasta_options() { - context_setup_host host - - layout_pasta_simple -} - # setup_passt_in_ns() - Set up namespace (with pasta), run qemu and passt into it setup_passt_in_ns() { context_setup_host host @@ -285,11 +273,6 @@ teardown_build() { teardown_context_watch ${PANE_HOST} host } -# teardown_distro() - Nothing to do, yet -teardown_distro() { - : -} - # teardown_passt() - Kill qemu, remove passt PID file teardown_passt() { kill $(cat "${STATESETUP}/qemu.pid") @@ -311,12 +294,6 @@ teardown_pasta() { teardown_context_watch ${PANE_NS} unshare ns } -# teardown_pasta_options() - Tear down pasta and host context, no namespace -teardown_pasta_options() { - teardown_context_watch ${PANE_HOST} host - teardown_context_watch ${PANE_PASST} passt -} - # teardown_passt_in_ns() - Exit namespace, kill qemu and pasta, remove pid file teardown_passt_in_ns() { context_run ns kill $(cat "${STATESETUP}/qemu.pid") diff --git a/test/lib/setup_ugly b/test/lib/setup_ugly new file mode 100755 index 0000000..764177e --- /dev/null +++ b/test/lib/setup_ugly @@ -0,0 +1,38 @@ +#!/bin/sh +# +# SPDX-License-Identifier: AGPL-3.0-or-later +# +# PASST - Plug A Simple Socket Transport +# for qemu/UNIX domain socket mode +# +# PASTA - Pack A Subtle Tap Abstraction +# for network namespace/tap device mode +# +# test/lib/setup_ugly - Setup functions using screen-scraping instead of context +# +# Copyright (c) 2022 Red Hat GmbH +# Author: Stefano Brivio + +# setup_distro() - Set up pane layout for distro tests +setup_distro() { + layout_host +} + +# setup_pasta_options() - Set up layout and host context without starting pasta +setup_pasta_options() { + context_setup_host host + + layout_pasta_simple +} + +# teardown_distro() - Nothing to do, yet +teardown_distro() { + : +} + +# teardown_pasta_options() - Tear down pasta and host context, no namespace +teardown_pasta_options() { + teardown_context_watch ${PANE_HOST} host + teardown_context_watch ${PANE_PASST} passt +} + diff --git a/test/run b/test/run index cb3b6c0..c3486b9 100755 --- a/test/run +++ b/test/run @@ -39,10 +39,12 @@ COMMIT="$(git log --oneline --no-decorate -1)" . lib/util . lib/setup +. lib/setup_ugly . lib/context . lib/term . lib/perf_report . lib/layout +. lib/layout_ugly . lib/test . lib/video -- 2.35.1

Stefano Brivio

7:33 a.m.

New subject: [PATCH v4 2/5] tap: Support for detection of existing sockets on ramfs

On ramfs, connecting to a non-existent UNIX domain socket yields EACCESS, instead of ENOENT. This is visible if we use passt directly on rootfs (a ramfs instance) from an initramfs image. It's probably wrong for ramfs to return EACCES, but given the simplicity of the filesystem, I doubt we should try to fix it there at the possible cost of added complexity. Also, this whole beauty should go away once qrap-less usage is established, so just accept EACCES as indication that a conflicting socket does not, in fact, exist. Signed-off-by: Stefano Brivio Reviewed-by: David Gibson --- tap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tap.c b/tap.c index 4dcff4f..0a3ccce 100644 --- a/tap.c +++ b/tap.c @@ -912,7 +912,8 @@ static void tap_sock_unix_init(struct ctx *c) } ret = connect(ex, (const struct sockaddr *)&addr, sizeof(addr)); - if (!ret || (errno != ENOENT && errno != ECONNREFUSED)) { + if (!ret || (errno != ENOENT && errno != ECONNREFUSED && + errno != EACCES)) { if (*c->sock_path) { err("Socket path %s already in use", path); exit(EXIT_FAILURE); -- 2.35.1

Stefano Brivio

7:33 a.m.

New subject: [PATCH v4 3/5] test/lib/perf_report: Use own flag to track initialisation

Instead of just disabling performance reports if running in demo mode. This allows us to use table functions outside of performance reports. Signed-off-by: Stefano Brivio Reviewed-by: David Gibson --- test/lib/perf_report | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/test/lib/perf_report b/test/lib/perf_report index ed71ef9..4ffb14b 100755 --- a/test/lib/perf_report +++ b/test/lib/perf_report @@ -13,6 +13,7 @@ # Copyright (c) 2021 Red Hat GmbH # Author: Stefano Brivio +PERF_INIT=0 PERF_LINK_COUNT=0 PERF_JS="${LOGDIR}/web/perf.js" @@ -129,6 +130,7 @@ perf_init() { mkdir -p "$(dirname "${PERF_JS}")" echo "${PERF_TEMPLATE_HTML}" > "${PERF_JS}" perf_report_sub commit "$(echo ${COMMIT} | sed "s/'/\\\'/g")" + PERF_INIT=1 } # perf_fill_lines() - Fill multiple "LINE" directives in template, matching rows @@ -178,6 +180,7 @@ perf_fill_lines() { # perf_finish() - Add trailing backslashes and process ending templates perf_finish() { + PERF_INIT=0 perf_fill_lines sed -i 's/^.*$/&\\/g' "${PERF_JS}" echo "${PERF_TEMPLATE_JS}" >> "${PERF_JS}" @@ -215,7 +218,7 @@ perf_report() { # perf_th() - Table header for a set of tests perf_th() { - [ ${DEMO} -eq 1 ] && return + [ ${PERF_INIT} -eq 0 ] && return shift @@ -231,7 +234,7 @@ perf_th() { # perf_tr() - Main table row perf_tr() { - [ ${DEMO} -eq 1 ] && return + [ ${PERF_INIT} -eq 0 ] && return __line_no=0 shift @@ -244,7 +247,7 @@ perf_tr() { # perf_td() - Single cell with test result perf_td() { - [ ${DEMO} -eq 1 ] && return + [ ${PERF_INIT} -eq 0 ] && return __rewind="${1}" shift -- 2.35.1

Stefano Brivio

7:33 a.m.

New subject: [PATCH v4 4/5] test/lib: Add "td" directive, handled by table_value()

This can be used for generic cell values with an arbitrary scale. Signed-off-by: Stefano Brivio Reviewed-by: David Gibson --- test/lib/term | 28 ++++++++++++++++++++++++++++ test/lib/test | 3 +++ 2 files changed, 31 insertions(+) diff --git a/test/lib/term b/test/lib/term index 1b42df8..8515c50 100755 --- a/test/lib/term +++ b/test/lib/term @@ -552,6 +552,34 @@ table_end() { info_n "\n${__buf}'\n" } +# table_value() - Print generic table value in its own cell +# $1: Value, can be '-' to indicate a filler +# $2: Scale, exponent of 10 +# $3: Error value, scaled: if value is less than this, print in red +# $4: Warning value, scaled: if value is less than this, print in yellow +table_value() { + [ "${1}" = "-" ] && table_cell 1 "-" && perf_td 0 "" && return 0 + if [ "${2}" != "0" ]; then + __v="$(echo "scale=1; x=( ${1} + 10^$((${2} - 1)) / 2 ) / 10^${2}; if ( x < 1 && x > 0 ) print 0; x" | bc -l)" + else + __v="${1}" + fi + perf_td 0 "${__v}" + + __red="${3}" + __yellow="${4}" + if [ "$(echo "${__v} < ${__red}" | bc -l)" = "1" ]; then + table_cell ${#__v} "${PR_RED}${__v}${PR_NC}" + return 1 + elif [ "$(echo "${__v} < ${__yellow}" | bc -l)" = "1" ]; then + table_cell ${#__v} "${PR_YELLOW}${__v}${PR_NC}" + return 1 + else + table_cell ${#__v} "${PR_GREEN}${__v}${PR_NC}" + return 0 + fi +} + table_value_throughput() { [ "${1}" = "-" ] && table_cell 1 "-" && perf_td 0 "" && return 0 __v="$(echo "scale=1; x=( ${1} + 10^8 / 2 ) / 10^9; if ( x < 1 && x > 0 ) print 0; x" | bc -l)" diff --git a/test/lib/test b/test/lib/test index 4c271a5..df32178 100755 --- a/test/lib/test +++ b/test/lib/test @@ -278,6 +278,9 @@ test_one_line() { "te") table_end ;; + "td") + table_value ${__arg} || TEST_ONE_perf_nok=1 + ;; "bw") table_value_throughput ${__arg} || TEST_ONE_perf_nok=1 ;; -- 2.35.1

Stefano Brivio

7:33 a.m.

New subject: [PATCH v4 5/5] test: Add memory/passt test cases

These show a summary of memory usage in kernel and userspace with different port forwarding configurations, details of userspace usage using 'nm' (passt only uses statically allocated memory), and details of kernel memory from slab reporting facilities. This adds a new test image, mbuto.mem.img, with harcoded IPv4 and IPv6 addresses and routes, and just the tools we need to start and stop passt, to report from /proc/slabinfo, /proc/meminfo, and to print and parse symbol sizes using nm(1). passt can't pivot_root() for sandboxing purposes on ramfs, so we need to create another filesystem and chroot into it, first. We don't want to use pane context functions, as we're checking memory usage for sockets: resort to screen-scraping. Configure a dummy interface to provide passt with an appearance of working IPv4 and IPv6 connectivity, contributed by David. Signed-off-by: Stefano Brivio --- test/Makefile | 5 +- test/lib/layout_ugly | 30 +++++++ test/lib/setup_ugly | 20 +++++ test/memory/passt | 187 +++++++++++++++++++++++++++++++++++++++++++ test/passt.mem.mbuto | 44 ++++++++++ test/run | 4 + 6 files changed, 289 insertions(+), 1 deletion(-) create mode 100644 test/memory/passt create mode 100755 test/passt.mem.mbuto diff --git a/test/Makefile b/test/Makefile index 91498ff..9f2bc42 100644 --- a/test/Makefile +++ b/test/Makefile @@ -55,7 +55,7 @@ UBUNTU_IMGS = $(UBUNTU_OLD_IMGS) $(UBUNTU_NEW_IMGS) DOWNLOAD_ASSETS = mbuto \ $(DEBIAN_IMGS) $(FEDORA_IMGS) $(OPENSUSE_IMGS) $(UBUNTU_IMGS) TESTDATA_ASSETS = small.bin big.bin medium.bin -LOCAL_ASSETS = mbuto.img QEMU_EFI.fd \ +LOCAL_ASSETS = mbuto.img mbuto.mem.img QEMU_EFI.fd \ $(DEBIAN_IMGS:%=prepared-%) $(FEDORA_IMGS:%=prepared-%) \ $(UBUNTU_NEW_IMGS:%=prepared-%) \ nsholder guest-key guest-key.pub \ @@ -76,6 +76,9 @@ guest-key guest-key.pub: mbuto.img: passt.mbuto mbuto guest-key.pub $(TESTDATA_ASSETS) ./mbuto/mbuto -p ./$< -c lz4 -f $@ +mbuto.mem.img: passt.mem.mbuto mbuto ../passt.avx2 + ./mbuto/mbuto -p ./$< -c lz4 -f $@ + nsholder: nsholder.c $(CC) $(CFLAGS) -o $@ $^ diff --git a/test/lib/layout_ugly b/test/lib/layout_ugly index 9397b7d..22f6169 100644 --- a/test/lib/layout_ugly +++ b/test/lib/layout_ugly @@ -81,3 +81,33 @@ layout_pasta_simple() { sleep 1 } + +# layout_memory() - Screen-scraped panes for memory usage tests, big guest pane +layout_memory() { + sleep 3 + + tmux kill-pane -a -t 0 + cmd_write 0 clear + + tmux split-window -h -l '35%' -t passt_test + + PANE_GUEST=0 + PANE_INFO=1 + + get_info_cols + + tmux send-keys -l -t ${PANE_INFO} 'while cat '"$STATEBASE/log_pipe"'; do :; done' + tmux send-keys -t ${PANE_INFO} -N 100 C-m + tmux select-pane -t ${PANE_INFO} -T "test log" + + if context_exists guest; then + pane_watch_contexts ${PANE_GUEST} guest guest + else + tmux pipe-pane -O -t ${PANE_GUEST} "cat >> ${LOGDIR}/pane_guest.log" + tmux select-pane -t ${PANE_GUEST} -T "guest" + fi + + info_layout "memory usage" + + sleep 1 +} diff --git a/test/lib/setup_ugly b/test/lib/setup_ugly index 764177e..bcf07ef 100755 --- a/test/lib/setup_ugly +++ b/test/lib/setup_ugly @@ -13,6 +13,8 @@ # Copyright (c) 2022 Red Hat GmbH # Author: Stefano Brivio +INITRAMFS_MEM="${BASEPATH}/mbuto.mem.img" + # setup_distro() - Set up pane layout for distro tests setup_distro() { layout_host @@ -25,6 +27,20 @@ setup_pasta_options() { layout_pasta_simple } +# setup_memory() - Start qemu in guest pane, and passt in passt context +setup_memory() { + layout_memory + + pane_or_context_run guest 'qemu-system-$(uname -m)' \ + ' -machine accel=kvm' \ + ' -m '${VMEM}' -cpu host -smp '${VCPUS} \ + ' -kernel ' "/boot/vmlinuz-$(uname -r)" \ + ' -initrd '${INITRAMFS_MEM}' -nographic -serial stdio' \ + ' -nodefaults' \ + ' -append "console=ttyS0 mitigations=off apparmor=0"' \ + " -pidfile ${STATESETUP}/qemu.pid" +} + # teardown_distro() - Nothing to do, yet teardown_distro() { : @@ -36,3 +52,7 @@ teardown_pasta_options() { teardown_context_watch ${PANE_PASST} passt } +# teardown_passt() - Kill qemu with ^C, remove passt PID file +teardown_memory() { + kill $(cat "${STATESETUP}/qemu.pid") +} diff --git a/test/memory/passt b/test/memory/passt new file mode 100644 index 0000000..fa89821 --- /dev/null +++ b/test/memory/passt @@ -0,0 +1,187 @@ +# SPDX-License-Identifier: AGPL-3.0-or-later +# +# PASST - Plug A Simple Socket Transport +# for qemu/UNIX domain socket mode +# +# PASTA - Pack A Subtle Tap Abstraction +# for network namespace/tap device mode +# +# test/memory/passt - Show memory usage of passt in kernel and userspace +# +# Copyright (c) 2022 Red Hat GmbH +# Author: Stefano Brivio + +gtools sed cat diff nm sort kill tee head tail chroot unshare mount mkdir cp + +def meminfo_row +gout DIFF meminfo_diff /tmp/meminfo.before /tmp/meminfo.after __WHAT__ +tl __NAME__ +td __DIFF__ 3 0 0 +endef + +def meminfo_reverse_row +gout DIFF meminfo_diff /tmp/meminfo.after /tmp/meminfo.before __WHAT__ +tl __NAME__ +td __DIFF__ 3 0 0 +endef + +def nm_row +gout SIZE nm_size /tmp/nm.size __WHAT__ +tl __WHAT__ +td __SIZE__ 6 0 0 +endef + +def slab_row +gout COUNT slab_diff_count /tmp/slabinfo.before /tmp/slabinfo.after __WHAT__ +gout SIZE slab_size /tmp/slabinfo.before __WHAT__ +gout DIFF slab_diff_size /tmp/slabinfo.before /tmp/slabinfo.after __WHAT__ +tl __WHAT__ +td __COUNT__ 0 0 0 +td __SIZE__ 0 0 0 +td __DIFF__ 6 0 0 +endef + +def start_stop_diff +guest sed /proc/slabinfo -ne 's/^$[^ ]* *[^ ]* *[^ ]* *[^ ]*$.*/\\\1/p' > /tmp/slabinfo.before +guest cat /proc/meminfo > /tmp/meminfo.before +guest /usr/bin/passt.avx2 -l /tmp/log -s /tmp/sock -P /tmp/pid __OPTS__ --netns-only +sleep 2 +guest cat /proc/meminfo > /tmp/meminfo.after +guest sed /proc/slabinfo -ne 's/^$[^ ]* *[^ ]* *[^ ]* *[^ ]*$.*/\\\1/p' > /tmp/slabinfo.after +guest kill \$(cat /tmp/pid) +guest diff -y --suppress-common-lines /tmp/meminfo.before /tmp/meminfo.after || : +guest nm -td -Sr --size-sort -P /usr/bin/passt.avx2 | head -30 | tee /tmp/nm.size +guest sed /proc/slabinfo -ne 's/$.*<objsize>$.*$/\1/p' | tail -1; (diff -y --suppress-common-lines /tmp/slabinfo.before /tmp/slabinfo.after | sort -grk8) +endef + +def summary +info Memory usage summary +info +th type MiB +set WHAT MemFree +set NAME used +meminfo_reverse_row +set WHAT AnonPages +set NAME userspace +meminfo_row +set WHAT Slab +set NAME kernel +meminfo_row +te +endef + + +guest mkdir /test +guest mount -t tmpfs none /test +guest mkdir /test/proc /test/dev /test/tmp +guest mount -o bind /proc /test/proc +guest mount -o bind /dev /test/dev +guest cp -Lr /bin /lib /lib64 /usr /sbin /test/ + +guest ulimit -Hn 300000 +guest unshare -rUm -R /test +guest chroot . + +guest meminfo_size() { grep "^$2:" $1 | tr -s ' ' | cut -f2 -d ' '; } +guest meminfo_diff() { echo $(( $(meminfo_size $2 $3) - $(meminfo_size $1 $3) )); } + +guest nm_size() { grep -m1 "^$2 " $1 | cut -f4 -d ' '; } + +guest slab_count() { grep "^$2 " $1 | tr -s ' ' | cut -f3 -d ' '; } +guest slab_size() { grep "^$2 " $1 | tr -s ' ' | cut -f4 -d ' '; } +guest slab_diff_count() { echo $(( $(slab_count $2 $3) - $(slab_count $1 $3) )); } +guest slab_diff_size() { echo $(( $(slab_count $2 $3) * $(slab_size $2 $3) - $(slab_count $1 $3) * $(slab_size $1 $3) )); } + + +test Memory usage: all TCP and UDP ports forwarded, IPv4 and IPv6 +set OPTS -t all -u all +start_stop_diff +summary + +info Userspace memory detail +info +th symbol MiB +set WHAT tcp_buf_discard +nm_row +set WHAT tcp6_l2_buf +nm_row +set WHAT tcp4_l2_buf +nm_row +set WHAT tc +nm_row +set WHAT pkt_buf +nm_row +set WHAT udp_splice_map +nm_row +set WHAT udp6_l2_buf +nm_row +set WHAT udp4_l2_buf +nm_row +set WHAT udp_tap_map +nm_row +set WHAT icmp_id_map +nm_row +set WHAT udp_splice_buf +nm_row +set WHAT tc_hash +nm_row +set WHAT pool_tap6_storage +nm_row +set WHAT pool_tap4_storage +nm_row +set WHAT tap6_l4 +nm_row +set WHAT tap4_l4 +nm_row +te + +info Kernel memory detail +info +th objects count size MiB +set WHAT pid +slab_row +set WHAT dentry +slab_row +set WHAT Acpi-Parse +slab_row +set WHAT kmalloc-64 +slab_row +set WHAT kmalloc-32 +slab_row +set WHAT lsm_file_cache +slab_row +set WHAT filp +slab_row +set WHAT anon_vma_chain +slab_row +set WHAT ep_head +slab_row +set WHAT sock_inode_cache +slab_row +set WHAT signal_cache +slab_row +set WHAT TCPv6 +slab_row +set WHAT TCP +slab_row +set WHAT UDPv6 +slab_row +te + + +test Memory usage: all TCP ports forwarded, IPv4 +set OPTS -t all -4 +start_stop_diff +summary + + +test Memory usage: all TCP and UDP ports forwarded, IPv4 +set OPTS -t all -u all -4 +start_stop_diff +summary + + +test Memory usage: no ports forwarded +set OPTS -t none -u none +start_stop_diff +summary diff --git a/test/passt.mem.mbuto b/test/passt.mem.mbuto new file mode 100755 index 0000000..dc06015 --- /dev/null +++ b/test/passt.mem.mbuto @@ -0,0 +1,44 @@ +#!/bin/sh +# +# SPDX-License-Identifier: AGPL-3.0-or-later +# +# PASST - Plug A Simple Socket Transport +# for qemu/UNIX domain socket mode +# +# test/passt.mem.mbuto - mbuto (https://mbuto.sh) profile for memory usage tests +# +# Copyright (c) 2022 Red Hat GmbH +# Author: Stefano Brivio + +PROGS="${PROGS:-ash,dash,bash chmod ip mount insmod mkdir ln cat chmod modprobe + grep mknod sed chown sleep bc ls ps mount unshare chroot cp kill diff + head tail sort tr tee cut nm which}" + +KMODS="${KMODS:- dummy}" + +NODES="${NODES:-console kmsg null ptmx random urandom zero}" + +LINKS="${LINKS:- + ash,dash,bash /init + ash,dash,bash /bin/sh}" + +DIRS="${DIRS} /tmp /sbin" + +COPIES="${COPIES} ../passt.avx2,/bin/passt.avx2" + +FIXUP="${FIXUP}"' +ln -s /bin /usr/bin +chmod 777 /tmp +ip link add eth0 type dummy +ip link set eth0 up +ip address add 192.0.2.2/24 dev eth0 +ip address add 2001:db8::2/64 dev eth0 +ip route add default via 192.0.2.1 +ip -6 route add default via 2001:db8::1 dev eth0 +sleep 2 +sh +m +' + +OUTPUT="KERNEL=__KERNEL__ +INITRD=__INITRD__ +" diff --git a/test/run b/test/run index c3486b9..e07513f 100755 --- a/test/run +++ b/test/run @@ -79,6 +79,10 @@ run() { test pasta_options/log_to_file teardown pasta_options + setup memory + test memory/passt + teardown memory + setup passt test passt/ndp test passt/dhcp -- 2.35.1

David Gibson

4 Nov 4 Nov

1:27 a.m.

New subject: [PATCH v4 5/5] test: Add memory/passt test cases

On Thu, Nov 03, 2022 at 07:33:28AM +0100, Stefano Brivio wrote:

...

These show a summary of memory usage in kernel and userspace with different port forwarding configurations, details of userspace usage using 'nm' (passt only uses statically allocated memory), and details of kernel memory from slab reporting facilities.

This adds a new test image, mbuto.mem.img, with harcoded IPv4 and IPv6 addresses and routes, and just the tools we need to start and stop passt, to report from /proc/slabinfo, /proc/meminfo, and to print and parse symbol sizes using nm(1).

passt can't pivot_root() for sandboxing purposes on ramfs, so we need to create another filesystem and chroot into it, first.

We don't want to use pane context functions, as we're checking memory usage for sockets: resort to screen-scraping.

Configure a dummy interface to provide passt with an appearance of working IPv4 and IPv6 connectivity, contributed by David.

Signed-off-by: Stefano Brivio

Reviewed-by: David Gibson

...

--- test/Makefile | 5 +- test/lib/layout_ugly | 30 +++++++ test/lib/setup_ugly | 20 +++++ test/memory/passt | 187 +++++++++++++++++++++++++++++++++++++++++++ test/passt.mem.mbuto | 44 ++++++++++ test/run | 4 + 6 files changed, 289 insertions(+), 1 deletion(-) create mode 100644 test/memory/passt create mode 100755 test/passt.mem.mbuto

diff --git a/test/Makefile b/test/Makefile index 91498ff..9f2bc42 100644 --- a/test/Makefile +++ b/test/Makefile @@ -55,7 +55,7 @@ UBUNTU_IMGS = $(UBUNTU_OLD_IMGS) $(UBUNTU_NEW_IMGS) DOWNLOAD_ASSETS = mbuto \ $(DEBIAN_IMGS) $(FEDORA_IMGS) $(OPENSUSE_IMGS) $(UBUNTU_IMGS) TESTDATA_ASSETS = small.bin big.bin medium.bin -LOCAL_ASSETS = mbuto.img QEMU_EFI.fd \ +LOCAL_ASSETS = mbuto.img mbuto.mem.img QEMU_EFI.fd \ $(DEBIAN_IMGS:%=prepared-%) $(FEDORA_IMGS:%=prepared-%) \ $(UBUNTU_NEW_IMGS:%=prepared-%) \ nsholder guest-key guest-key.pub \ @@ -76,6 +76,9 @@ guest-key guest-key.pub: mbuto.img: passt.mbuto mbuto guest-key.pub $(TESTDATA_ASSETS) ./mbuto/mbuto -p ./$< -c lz4 -f $@

+mbuto.mem.img: passt.mem.mbuto mbuto ../passt.avx2 + ./mbuto/mbuto -p ./$< -c lz4 -f $@ + nsholder: nsholder.c $(CC) $(CFLAGS) -o $@ $^

diff --git a/test/lib/layout_ugly b/test/lib/layout_ugly index 9397b7d..22f6169 100644 --- a/test/lib/layout_ugly +++ b/test/lib/layout_ugly @@ -81,3 +81,33 @@ layout_pasta_simple() {

sleep 1 } + +# layout_memory() - Screen-scraped panes for memory usage tests, big guest pane +layout_memory() { + sleep 3 + + tmux kill-pane -a -t 0 + cmd_write 0 clear + + tmux split-window -h -l '35%' -t passt_test + + PANE_GUEST=0 + PANE_INFO=1 + + get_info_cols + + tmux send-keys -l -t ${PANE_INFO} 'while cat '"$STATEBASE/log_pipe"'; do :; done' + tmux send-keys -t ${PANE_INFO} -N 100 C-m + tmux select-pane -t ${PANE_INFO} -T "test log" + + if context_exists guest; then + pane_watch_contexts ${PANE_GUEST} guest guest + else + tmux pipe-pane -O -t ${PANE_GUEST} "cat >> ${LOGDIR}/pane_guest.log" + tmux select-pane -t ${PANE_GUEST} -T "guest" + fi + + info_layout "memory usage" + + sleep 1 +} diff --git a/test/lib/setup_ugly b/test/lib/setup_ugly index 764177e..bcf07ef 100755 --- a/test/lib/setup_ugly +++ b/test/lib/setup_ugly @@ -13,6 +13,8 @@ # Copyright (c) 2022 Red Hat GmbH # Author: Stefano Brivio

+INITRAMFS_MEM="${BASEPATH}/mbuto.mem.img" + # setup_distro() - Set up pane layout for distro tests setup_distro() { layout_host @@ -25,6 +27,20 @@ setup_pasta_options() { layout_pasta_simple }

+# setup_memory() - Start qemu in guest pane, and passt in passt context +setup_memory() { + layout_memory + + pane_or_context_run guest 'qemu-system-$(uname -m)' \ + ' -machine accel=kvm' \ + ' -m '${VMEM}' -cpu host -smp '${VCPUS} \ + ' -kernel ' "/boot/vmlinuz-$(uname -r)" \ + ' -initrd '${INITRAMFS_MEM}' -nographic -serial stdio' \ + ' -nodefaults' \ + ' -append "console=ttyS0 mitigations=off apparmor=0"' \ + " -pidfile ${STATESETUP}/qemu.pid" +} + # teardown_distro() - Nothing to do, yet teardown_distro() { : @@ -36,3 +52,7 @@ teardown_pasta_options() { teardown_context_watch ${PANE_PASST} passt }

+# teardown_passt() - Kill qemu with ^C, remove passt PID file +teardown_memory() { + kill $(cat "${STATESETUP}/qemu.pid") +} diff --git a/test/memory/passt b/test/memory/passt new file mode 100644 index 0000000..fa89821 --- /dev/null +++ b/test/memory/passt @@ -0,0 +1,187 @@ +# SPDX-License-Identifier: AGPL-3.0-or-later +# +# PASST - Plug A Simple Socket Transport +# for qemu/UNIX domain socket mode +# +# PASTA - Pack A Subtle Tap Abstraction +# for network namespace/tap device mode +# +# test/memory/passt - Show memory usage of passt in kernel and userspace +# +# Copyright (c) 2022 Red Hat GmbH +# Author: Stefano Brivio + +gtools sed cat diff nm sort kill tee head tail chroot unshare mount mkdir cp + +def meminfo_row +gout DIFF meminfo_diff /tmp/meminfo.before /tmp/meminfo.after __WHAT__ +tl __NAME__ +td __DIFF__ 3 0 0 +endef + +def meminfo_reverse_row +gout DIFF meminfo_diff /tmp/meminfo.after /tmp/meminfo.before __WHAT__ +tl __NAME__ +td __DIFF__ 3 0 0 +endef + +def nm_row +gout SIZE nm_size /tmp/nm.size __WHAT__ +tl __WHAT__ +td __SIZE__ 6 0 0 +endef + +def slab_row +gout COUNT slab_diff_count /tmp/slabinfo.before /tmp/slabinfo.after __WHAT__ +gout SIZE slab_size /tmp/slabinfo.before __WHAT__ +gout DIFF slab_diff_size /tmp/slabinfo.before /tmp/slabinfo.after __WHAT__ +tl __WHAT__ +td __COUNT__ 0 0 0 +td __SIZE__ 0 0 0 +td __DIFF__ 6 0 0 +endef + +def start_stop_diff +guest sed /proc/slabinfo -ne 's/^$[^ ]* *[^ ]* *[^ ]* *[^ ]*$.*/\\\1/p' > /tmp/slabinfo.before +guest cat /proc/meminfo > /tmp/meminfo.before +guest /usr/bin/passt.avx2 -l /tmp/log -s /tmp/sock -P /tmp/pid __OPTS__ --netns-only +sleep 2 +guest cat /proc/meminfo > /tmp/meminfo.after +guest sed /proc/slabinfo -ne 's/^$[^ ]* *[^ ]* *[^ ]* *[^ ]*$.*/\\\1/p' > /tmp/slabinfo.after +guest kill \$(cat /tmp/pid) +guest diff -y --suppress-common-lines /tmp/meminfo.before /tmp/meminfo.after || : +guest nm -td -Sr --size-sort -P /usr/bin/passt.avx2 | head -30 | tee /tmp/nm.size +guest sed /proc/slabinfo -ne 's/$.*<objsize>$.*$/\1/p' | tail -1; (diff -y --suppress-common-lines /tmp/slabinfo.before /tmp/slabinfo.after | sort -grk8) +endef + +def summary +info Memory usage summary +info +th type MiB +set WHAT MemFree +set NAME used +meminfo_reverse_row +set WHAT AnonPages +set NAME userspace +meminfo_row +set WHAT Slab +set NAME kernel +meminfo_row +te +endef + + +guest mkdir /test +guest mount -t tmpfs none /test +guest mkdir /test/proc /test/dev /test/tmp +guest mount -o bind /proc /test/proc +guest mount -o bind /dev /test/dev +guest cp -Lr /bin /lib /lib64 /usr /sbin /test/ + +guest ulimit -Hn 300000 +guest unshare -rUm -R /test +guest chroot . + +guest meminfo_size() { grep "^$2:" $1 | tr -s ' ' | cut -f2 -d ' '; } +guest meminfo_diff() { echo $(( $(meminfo_size $2 $3) - $(meminfo_size $1 $3) )); } + +guest nm_size() { grep -m1 "^$2 " $1 | cut -f4 -d ' '; } + +guest slab_count() { grep "^$2 " $1 | tr -s ' ' | cut -f3 -d ' '; } +guest slab_size() { grep "^$2 " $1 | tr -s ' ' | cut -f4 -d ' '; } +guest slab_diff_count() { echo $(( $(slab_count $2 $3) - $(slab_count $1 $3) )); } +guest slab_diff_size() { echo $(( $(slab_count $2 $3) * $(slab_size $2 $3) - $(slab_count $1 $3) * $(slab_size $1 $3) )); } + + +test Memory usage: all TCP and UDP ports forwarded, IPv4 and IPv6 +set OPTS -t all -u all +start_stop_diff +summary + +info Userspace memory detail +info +th symbol MiB +set WHAT tcp_buf_discard +nm_row +set WHAT tcp6_l2_buf +nm_row +set WHAT tcp4_l2_buf +nm_row +set WHAT tc +nm_row +set WHAT pkt_buf +nm_row +set WHAT udp_splice_map +nm_row +set WHAT udp6_l2_buf +nm_row +set WHAT udp4_l2_buf +nm_row +set WHAT udp_tap_map +nm_row +set WHAT icmp_id_map +nm_row +set WHAT udp_splice_buf +nm_row +set WHAT tc_hash +nm_row +set WHAT pool_tap6_storage +nm_row +set WHAT pool_tap4_storage +nm_row +set WHAT tap6_l4 +nm_row +set WHAT tap4_l4 +nm_row +te + +info Kernel memory detail +info +th objects count size MiB +set WHAT pid +slab_row +set WHAT dentry +slab_row +set WHAT Acpi-Parse +slab_row +set WHAT kmalloc-64 +slab_row +set WHAT kmalloc-32 +slab_row +set WHAT lsm_file_cache +slab_row +set WHAT filp +slab_row +set WHAT anon_vma_chain +slab_row +set WHAT ep_head +slab_row +set WHAT sock_inode_cache +slab_row +set WHAT signal_cache +slab_row +set WHAT TCPv6 +slab_row +set WHAT TCP +slab_row +set WHAT UDPv6 +slab_row +te + + +test Memory usage: all TCP ports forwarded, IPv4 +set OPTS -t all -4 +start_stop_diff +summary + + +test Memory usage: all TCP and UDP ports forwarded, IPv4 +set OPTS -t all -u all -4 +start_stop_diff +summary + + +test Memory usage: no ports forwarded +set OPTS -t none -u none +start_stop_diff +summary diff --git a/test/passt.mem.mbuto b/test/passt.mem.mbuto new file mode 100755 index 0000000..dc06015 --- /dev/null +++ b/test/passt.mem.mbuto @@ -0,0 +1,44 @@ +#!/bin/sh +# +# SPDX-License-Identifier: AGPL-3.0-or-later +# +# PASST - Plug A Simple Socket Transport +# for qemu/UNIX domain socket mode +# +# test/passt.mem.mbuto - mbuto (https://mbuto.sh) profile for memory usage tests +# +# Copyright (c) 2022 Red Hat GmbH +# Author: Stefano Brivio + +PROGS="${PROGS:-ash,dash,bash chmod ip mount insmod mkdir ln cat chmod modprobe + grep mknod sed chown sleep bc ls ps mount unshare chroot cp kill diff + head tail sort tr tee cut nm which}" + +KMODS="${KMODS:- dummy}" + +NODES="${NODES:-console kmsg null ptmx random urandom zero}" + +LINKS="${LINKS:- + ash,dash,bash /init + ash,dash,bash /bin/sh}" + +DIRS="${DIRS} /tmp /sbin" + +COPIES="${COPIES} ../passt.avx2,/bin/passt.avx2" + +FIXUP="${FIXUP}"' +ln -s /bin /usr/bin +chmod 777 /tmp +ip link add eth0 type dummy +ip link set eth0 up +ip address add 192.0.2.2/24 dev eth0 +ip address add 2001:db8::2/64 dev eth0 +ip route add default via 192.0.2.1 +ip -6 route add default via 2001:db8::1 dev eth0 +sleep 2 +sh +m +' + +OUTPUT="KERNEL=__KERNEL__ +INITRD=__INITRD__ +" diff --git a/test/run b/test/run index c3486b9..e07513f 100755 --- a/test/run +++ b/test/run @@ -79,6 +79,10 @@ run() { test pasta_options/log_to_file teardown pasta_options

+ setup memory + test memory/passt + teardown memory + setup passt test passt/ndp test passt/dhcp

-- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson

922

Age (days ago)

923

Last active (days ago)

List overview

Download

6 comments

2 participants

participants (2)

David Gibson
Stefano Brivio