- dev - passt

[PATCH v9 0/2] Add support for SO_PEEK_OFF
by Jon Maloy 15 Jul '24

15 Jul '24

We add support for the SO_PEEK_OFF option, plus work around a kernel bug discovered along the way. Jon Maloy (2): tcp: leverage support of SO_PEEK_OFF socket option when available tcp: handle shrunk window advertisemenst from guest tcp.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++++--- tcp.h | 3 +++ tcp_buf.c | 25 ++++++++++++++++------ 3 files changed, 81 insertions(+), 9 deletions(-) -- 2.45.2

3 7

[PATCH] doc: Trivial fix for reuseaddr-priority
by David Gibson 15 Jul '24

15 Jul '24

This test program checks for particular behaviour regardless of order of operations. So, we step through the test with all possible orders for a number of different of parts. Or at least, we're supposed to, a copy pasta error led to using the same order for two things which should be independent. Fixes: 299c40750 ("doc: Add program to document and test assumptions...") Reported-by: David Taylor <davidt(a)yadt.co.uk> Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au> --- doc/platform-requirements/reuseaddr-priority.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/platform-requirements/reuseaddr-priority.c b/doc/platform-requirements/reuseaddr-priority.c index 644553f8..701b6fff 100644 --- a/doc/platform-requirements/reuseaddr-priority.c +++ b/doc/platform-requirements/reuseaddr-priority.c @@ -222,7 +222,7 @@ static void check_all_orders(void) for (k = 0; k < norders; k++) for (l = 0; l < norders; l++) check_one_order(orders[i], orders[j], - orders[j], orders[l]); + orders[k], orders[l]); } int main(int argc, char *argv[]) -- 2.45.2

2 1

[PATCH/RFC] test: run static checkers with Avocado and JSON definitions
by Cleber Rosa 15 Jul '24

15 Jul '24

This adds a script and configuration to use the Avocado Testing Framework to run, at this time, the static checkers. The actual tests are defined using (JSON based) files, that are known to Avocado as "recipes". The JSON files are parsed and "resolved" into tests by Avocado's "runnables-recipe" resolver. The syntax allows for any kind of test supported by Avocado to be defined there, including a mix of different test types. By the nature of Avocado's default configuration, those will run in parallel in the host system. For more complex tests or different use cases, Avocado could help in future versions by running those in different environments such as containers. The entry point ("test/run_avocado") is intended to be an optional tool at this point, coexisting with the current implementation to run tests. It uses Avocado's Job API to create a job with, at this point, the static checkers suite. The installation of Avocado itself is left to users, given that the details on how to install it (virtual environments and specific tooling) can be a very different and long discussion. Signed-off-by: Cleber Rosa <crosa(a)redhat.com> --- test/avocado/static_checkers.json | 16 ++++++++++ test/run_avocado | 49 +++++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+) create mode 100644 test/avocado/static_checkers.json create mode 100755 test/run_avocado diff --git a/test/avocado/static_checkers.json b/test/avocado/static_checkers.json new file mode 100644 index 0000000..5fae43e --- /dev/null +++ b/test/avocado/static_checkers.json @@ -0,0 +1,16 @@ +[ + { + "kind": "exec-test", + "uri": "make", + "args": [ + "clang-tidy" + ] + }, + { + "kind": "exec-test", + "uri": "make", + "args": [ + "cppcheck" + ] + } +] diff --git a/test/run_avocado b/test/run_avocado new file mode 100755 index 0000000..37db17c --- /dev/null +++ b/test/run_avocado @@ -0,0 +1,49 @@ +#!/usr/bin/env python3 + +import os +import sys + + +def check_avocado_version(): + minimum_version = 106.0 + + def error_out(): + print( + f"Avocado version {minimum_version} or later is required.\n" + f"You may install it with: \n" + f" python3 -m pip install avocado-framework", + file=sys.stderr, + ) + sys.exit(1) + + try: + from avocado import VERSION + + if (float(VERSION)) < minimum_version: + error_out() + except ImportError: + error_out() + + +check_avocado_version() +from avocado.core.job import Job +from avocado.core.suite import TestSuite + + +def main(): + repo_root_path = os.path.abspath( + os.path.dirname(os.path.dirname(os.path.dirname(__file__))) + ) + config = { + "resolver.references": [ + os.path.join(repo_root_path, "test", "avocado", "static_checkers.json") + ], + "runner.identifier_format": "{args[0]}", + } + suite = TestSuite.from_config(config, name="static_checkers") + with Job(config, [suite]) as j: + return j.run() + + +if __name__ == "__main__": + sys.exit(main()) -- 2.45.2

3 4

[PATCH v2 00/11] Preliminaries for UDP flow support
by David Gibson 15 Jul '24

15 Jul '24

The redesign of UDP flows required (or at least, suggested) a new batch of prelininary changes that don't rely on the core of the flow table rework. Changes since v1: * Assorted minor fixes based on Stefano's feedback * Moved test programs from contrib/ to doc/ David Gibson (11): util: sock_l4() determine protocol from epoll type rather than the reverse flow: Add flow_sidx_valid() helper udp: Pass full epoll reference through more of sock handler path udp: Rename IOV and mmsghdr arrays udp: Unify udp[46]_mh_splice udp: Unify udp[46]_l2_iov udp: Don't repeatedly initialise udp[46]_eth_hdr udp: Move some more of sock_handler tasks into sub-functions udp: Consolidate datagram batching doc: Add program to document and test assumptions about SO_REUSEADDR doc: Test behaviour of zero length datagram recv()s doc/platform-requirements/.gitignore | 2 + doc/platform-requirements/Makefile | 45 +++ doc/platform-requirements/README | 18 + doc/platform-requirements/common.c | 66 ++++ doc/platform-requirements/common.h | 47 +++ doc/platform-requirements/recv-zero.c | 74 ++++ .../reuseaddr-priority.c | 240 ++++++++++++ epoll_type.h | 41 ++ flow.h | 11 + flow_table.h | 2 +- icmp.c | 2 +- passt.h | 32 -- tcp.c | 17 +- udp.c | 365 +++++++++--------- util.c | 48 +-- util.h | 3 +- 16 files changed, 756 insertions(+), 257 deletions(-) create mode 100644 doc/platform-requirements/.gitignore create mode 100644 doc/platform-requirements/Makefile create mode 100644 doc/platform-requirements/README create mode 100644 doc/platform-requirements/common.c create mode 100644 doc/platform-requirements/common.h create mode 100644 doc/platform-requirements/recv-zero.c create mode 100644 doc/platform-requirements/reuseaddr-priority.c create mode 100644 epoll_type.h -- 2.45.2

3 14

[PATCH 0/5] Add vhost-user support to passt. (part 3)
by Laurent Vivier 15 Jul '24

15 Jul '24

This series of patches adds vhost-user support to passt and then allows passt to connect to QEMU network backend using virtqueue rather than a socket. With QEMU, rather than using to connect: -netdev stream,id=s,server=off,addr.type=unix,addr.path=/tmp/passt_1.socket we will use: -chardev socket,id=chr0,path=/tmp/passt_1.socket -netdev vhost-user,id=netdev0,chardev=chr0 -device virtio-net,netdev=netdev0 -object memory-backend-memfd,id=memfd0,share=on,size=$RAMSIZE -numa node,memdev=memfd0 The memory backend is needed to share data between passt and QEMU. Performance comparison between "-netdev stream" and "-netdev vhost-user": $ iperf3 -c localhost -p 10001 -t 60 -6 -u -b 50G socket: [ 5] 0.00-60.05 sec 95.6 GBytes 13.7 Gbits/sec 0.017 ms 6998988/10132413 (69%) receiver vhost-user: [ 5] 0.00-60.04 sec 237 GBytes 33.9 Gbits/sec 0.006 ms 53673/7813770 (0.69%) receiver $ iperf3 -c localhost -p 10001 -t 60 -4 -u -b 50G socket: [ 5] 0.00-60.05 sec 98.9 GBytes 14.1 Gbits/sec 0.018 ms 6260735/9501832 (66%) receiver vhost-user: [ 5] 0.00-60.05 sec 235 GBytes 33.7 Gbits/sec 0.008 ms 37581/7752699 (0.48%) receiver $ iperf3 -c localhost -p 10001 -t 60 -6 socket: [ 5] 0.00-60.00 sec 17.3 GBytes 2.48 Gbits/sec 0 sender [ 5] 0.00-60.06 sec 17.3 GBytes 2.48 Gbits/sec receiver vhost-user: [ 5] 0.00-60.00 sec 191 GBytes 27.4 Gbits/sec 0 sender [ 5] 0.00-60.05 sec 191 GBytes 27.3 Gbits/sec receiver $ iperf3 -c localhost -p 10001 -t 60 -4 socket: [ 5] 0.00-60.00 sec 15.6 GBytes 2.24 Gbits/sec 0 sender [ 5] 0.00-60.06 sec 15.6 GBytes 2.24 Gbits/sec receiver vhost-user: [ 5] 0.00-60.00 sec 189 GBytes 27.1 Gbits/sec 0 sender [ 5] 0.00-60.04 sec 189 GBytes 27.0 Gbits/sec receiver Laurent Vivier (5): packet: replace struct desc by struct iovec vhost-user: introduce virtio API vhost-user: introduce vhost-user API iov: add iov_count() vhost-user: add vhost-user Makefile | 4 +- checksum.c | 1 - conf.c | 18 +- iov.c | 35 +- iov.h | 2 + packet.c | 81 ++-- packet.h | 16 +- passt.c | 14 +- passt.h | 10 + pcap.c | 1 - tap.c | 106 ++++- tap.h | 5 +- tcp.c | 17 +- tcp_vu.c | 547 ++++++++++++++++++++++++ tcp_vu.h | 9 + udp.c | 54 ++- udp_internal.h | 39 ++ udp_vu.c | 237 +++++++++++ udp_vu.h | 8 + util.h | 11 + vhost_user.c | 1077 ++++++++++++++++++++++++++++++++++++++++++++++++ vhost_user.h | 137 ++++++ virtio.c | 442 ++++++++++++++++++++ virtio.h | 122 ++++++ 24 files changed, 2886 insertions(+), 107 deletions(-) create mode 100644 tcp_vu.c create mode 100644 tcp_vu.h create mode 100644 udp_internal.h create mode 100644 udp_vu.c create mode 100644 udp_vu.h create mode 100644 vhost_user.c create mode 100644 vhost_user.h create mode 100644 virtio.c create mode 100644 virtio.h -- 2.45.2

2 18

[PATCH v8 0/2] Add support for SO_PEEK_OFF
by Jon Maloy 12 Jul '24

12 Jul '24

We add support for the SO_PEEK_OFF option, plus work around a kernel bug discovered along the way. v8: Added better solution to the work-around. Jon Maloy (2): tcp: leverage support of SO_PEEK_OFF socket option when available tcp: handle shrunk window advertisemenst from guest tcp.c | 40 +++++++++++++++++++++++++++++++++++++++- tcp.h | 3 +++ tcp_buf.c | 22 ++++++++++++++++------ 3 files changed, 58 insertions(+), 7 deletions(-) -- 2.45.2

2 4

[PATCH 0/4] Small, assorted "hardening" fixes
by Stefano Brivio 08 Jul '24

08 Jul '24

All harmless issues as far as I can tell, but nice to fix. Stefano Brivio (4): conf: Copy up to MAXDNSRCH - 1 bytes, not MAXDNSRCH tcp_splice: Check return value of setsockopt() for SO_RCVLOWAT util, lineread, tap: Overflow checks on long signed sums and subtractions tap: Drop frames from guest whose length is more than remaining buffer conf.c | 2 +- lineread.c | 5 +++-- tap.c | 24 ++++++++++++++++++------ tcp_splice.c | 15 ++++++++++----- util.c | 7 +++++-- util.h | 46 +++++++++++++++++++++++++++++++++++++++++++++- 6 files changed, 82 insertions(+), 17 deletions(-) -- 2.43.0

2 20

Avocado, passt, etc.
by David Gibson 05 Jul '24

05 Jul '24

Hi again, Following on on your specific patch, I've done a little bit more thinking on how we might want to use Avocado for more extensive testing. I think I mentioned during some of our calls my idea of having some sort of test side library to automatically turn test code into executables that can be used from Avocado's extended exec stuff. I don't think I explained it very clearly though. I've made a super rudimentary draft of this idea here: https://gitlab.com/dgibson/exeter The "example.py" program, run without arguments lists all the ways you can run it, each of which constitutes a single test case. The idea is that that "list" mode would be extended to generate the json manifest that Avocado can consume. So first you run it to generate the json, then you feed that into avocado which will run each of the cases as a separate test. Ideally avocado could do both steps itself (this is what I was talking about with the "variant 2" of the extended exec stuff). But we can do it externally for the time being. Having things separated this way makes it super easy to manually re-run a specific test case. We can also allow the same library to emit the list of tests in other formats for other runners (being able to do this for meson seems like it might be pretty useful). Obviously, it would need a bunch more functionality to be really useful: ways of automatically generating matrices of tests, composing functions together to make test cases, as well as basic helper functions like assert variants. -- David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson

1 0

[PATCH 00/11] Preliminaries for UDP flow support
by David Gibson 05 Jul '24

05 Jul '24

The redesign of UDP flows required (or at least, suggested) a new batch of prelininary changes that don't rely on the core of the flow table rework. David Gibson (11): util: sock_l4() determine protocol from epoll type rather than the reverse flow: Add flow_sidx_valid() helper udp: Pass full epoll reference through more of sock handler path udp: Rename IOV and mmsghdr arrays udp: Unify udp[46]_mh_splice udp: Unify udp[46]_l2_iov udp: Don't repeatedly initialise udp[46]_eth_hdr udp: Move some more of sock_handler tasks into sub-functions udp: Consolidate datagram batching contrib: Add program to document and test assumptions about SO_REUSEADDR contrib: Test behaviour of zero length datagram recv()s contrib/udp-behaviour/.gitignore | 2 + contrib/udp-behaviour/Makefile | 45 +++ contrib/udp-behaviour/common.c | 66 ++++ contrib/udp-behaviour/common.h | 47 +++ contrib/udp-behaviour/recv-zero.c | 74 +++++ contrib/udp-behaviour/reuseaddr-priority.c | 240 ++++++++++++++ epoll_type.h | 41 +++ flow.h | 11 + flow_table.h | 2 +- icmp.c | 2 +- passt.h | 32 -- tcp.c | 17 +- udp.c | 361 ++++++++++----------- util.c | 48 +-- util.h | 3 +- 15 files changed, 735 insertions(+), 256 deletions(-) create mode 100644 contrib/udp-behaviour/.gitignore create mode 100644 contrib/udp-behaviour/Makefile create mode 100644 contrib/udp-behaviour/common.c create mode 100644 contrib/udp-behaviour/common.h create mode 100644 contrib/udp-behaviour/recv-zero.c create mode 100644 contrib/udp-behaviour/reuseaddr-priority.c create mode 100644 epoll_type.h -- 2.45.2

2 23

[PATCH v2 0/5] Small, assorted "hardening" fixes
by Stefano Brivio 02 Jul '24

02 Jul '24

All harmless issues as far as I can tell, but nice to fix. v2: - in 3/5: - keep 'skip' in write_remainder() unsigned, and check for unsigned overflow instead - refactor sadd_overflow() and ssub_overflow() to use built-ins with automatic types, take ssize_t arguments, and deal with different ssize_t type widths - in 4/5: - switch l2len in tap_handler_passt() to uint32_t, as it really is unsigned and 32-bit wide - return if the length descriptor mismatches, instead of trying to proceed to the next frame - add 5/5 Stefano Brivio (5): conf: Copy up to MAXDNSRCH - 1 bytes, not MAXDNSRCH tcp_splice: Check return value of setsockopt() for SO_RCVLOWAT util, lineread, tap: Overflow checks on long signed sums and subtractions tap: Discard guest data on length descriptor mismatch conf: Use the right maximum buffer size for c->sock_path conf.c | 4 ++-- lineread.c | 5 +++-- tap.c | 31 +++++++++++++++++++---------- tcp_splice.c | 15 +++++++++----- util.c | 5 +++++ util.h | 55 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 96 insertions(+), 19 deletions(-) -- 2.43.0

2 7