June 2024 - dev

[PATCH v2 00/14] Clean up checksum and header generation for inbound packets
by David Gibson 10 Oct '25

10 Oct '25

The main packet "fast paths" for UDP and TCP mostly just forward packets rather than generating them from scratch. However the control paths for ICMP and DHCP sometimes generate packets more or less from scratch. Because these are relatively rare, it's not performance critical. The paths for sending these packets have some duplication of the header generation. There's also some layering violation in tap_ip_send() which both generates IP headers and updates the L4 (UDP or UCMP) checksum. Finally that checksum generation is a little awkward: it temporarily generates the IP pseudo header (or something close enough to serve) in the place of the actual header, generates the checksum, then replaces it with the real IP header. This approach seems to be causing miscompiles with some LTO optimization, because the stores to the pseudo header are being moved or elided across the code calculating the checksum. This series addresses all of these. We consolidate and clarify the packet sending helpers, and use them in some places there was previously duplicated code. In the process we use new checksum generation helpers which take a different approach which should avoid the LTO problems (this aspect I haven't tested yet though). Changes since v1: * Numerous minor style changes * Rename header generation helpers to make their behaviour clearer * Added several missing function doc comments * Corrected some erroneous statements and terms in comments David Gibson (14): Add csum_icmp6() helper for calculating ICMPv6 checksums Add csum_icmp4() helper for calculating ICMP checksums Add csum_udp6() helper for calculating UDP over IPv6 checksums Add csum_udp4() helper for calculating UDP over IPv4 checksums Add csum_ip4_header() helper to calculate IPv4 header checksums Add helpers for normal inbound packet destination addresses Remove support for TCP packets from tap_ip_send() tap: Remove unhelpeful vnet_pre optimization from tap_send() Split tap_ip_send() into IPv4 and IPv6 specific functions tap: Split tap_ip6_send() into UDP and ICMP variants ndp: Remove unneeded eh_source parameter ndp: Use tap_icmp6_send() helper tap: Split tap_ip4_send() into UDP and ICMP variants dhcp: Use tap_udp4_send() helper in dhcp() arp.c | 2 +- checksum.c | 120 ++++++++++++++++----- checksum.h | 15 ++- dhcp.c | 19 +--- dhcpv6.c | 21 +--- icmp.c | 12 +-- ndp.c | 28 +---- ndp.h | 3 +- tap.c | 312 ++++++++++++++++++++++++++++++++++------------------- tap.h | 19 +++- 10 files changed, 345 insertions(+), 206 deletions(-) -- 2.37.3

2 17

[PATCH] pasta.c: modify hostname when detaching new namespace
by Danish Prakash 29 Jul '24

29 Jul '24

When invoking pasta without any arguments, it's difficult to tell whether we are in the new namespace or not leaving users a bit confused. This change modifies the host namespace to add a prefix "pasta-" to make it a bit more obvious. Signed-off-by: Danish Prakash <contact(a)danishpraka.sh> --- pasta.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/pasta.c b/pasta.c index 31e1e00..840a2b1 100644 --- a/pasta.c +++ b/pasta.c @@ -180,6 +180,8 @@ static int pasta_spawn_cmd(void *arg) { const struct pasta_spawn_cmd_arg *a; sigset_t set; + char hostname[HOST_NAME_MAX+1], pasta_hostname[HOST_NAME_MAX+1]; + char *hostname_prefix = "pasta-"; /* We run in a detached PID and mount namespace: mount /proc over */ if (mount("", "/proc", "proc", 0, NULL)) @@ -188,6 +190,17 @@ static int pasta_spawn_cmd(void *arg) if (write_file("/proc/sys/net/ipv4/ping_group_range", "0 0")) warn("Cannot set ping_group_range, ICMP requests might fail"); + if (gethostname(hostname, HOST_NAME_MAX+1) == 0) { + if ((strlen(hostname) + strlen(hostname_prefix)) > HOST_NAME_MAX) { + hostname[strlen(hostname)-strlen(hostname_prefix)] = '\0'; + } + sprintf(pasta_hostname, "%s%s", hostname_prefix, hostname); + + if (sethostname(pasta_hostname, strlen(pasta_hostname)) != 0) { + warn("Unable to set pasta-prefixed hostname"); + } + } + /* Wait for the parent to be ready: see main() */ sigemptyset(&set); sigaddset(&set, SIGUSR1); -- 2.45.1

2 10

[PATCH/RFC] test: run static checkers with Avocado and JSON definitions
by Cleber Rosa 15 Jul '24

15 Jul '24

This adds a script and configuration to use the Avocado Testing Framework to run, at this time, the static checkers. The actual tests are defined using (JSON based) files, that are known to Avocado as "recipes". The JSON files are parsed and "resolved" into tests by Avocado's "runnables-recipe" resolver. The syntax allows for any kind of test supported by Avocado to be defined there, including a mix of different test types. By the nature of Avocado's default configuration, those will run in parallel in the host system. For more complex tests or different use cases, Avocado could help in future versions by running those in different environments such as containers. The entry point ("test/run_avocado") is intended to be an optional tool at this point, coexisting with the current implementation to run tests. It uses Avocado's Job API to create a job with, at this point, the static checkers suite. The installation of Avocado itself is left to users, given that the details on how to install it (virtual environments and specific tooling) can be a very different and long discussion. Signed-off-by: Cleber Rosa <crosa(a)redhat.com> --- test/avocado/static_checkers.json | 16 ++++++++++ test/run_avocado | 49 +++++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+) create mode 100644 test/avocado/static_checkers.json create mode 100755 test/run_avocado diff --git a/test/avocado/static_checkers.json b/test/avocado/static_checkers.json new file mode 100644 index 0000000..5fae43e --- /dev/null +++ b/test/avocado/static_checkers.json @@ -0,0 +1,16 @@ +[ + { + "kind": "exec-test", + "uri": "make", + "args": [ + "clang-tidy" + ] + }, + { + "kind": "exec-test", + "uri": "make", + "args": [ + "cppcheck" + ] + } +] diff --git a/test/run_avocado b/test/run_avocado new file mode 100755 index 0000000..37db17c --- /dev/null +++ b/test/run_avocado @@ -0,0 +1,49 @@ +#!/usr/bin/env python3 + +import os +import sys + + +def check_avocado_version(): + minimum_version = 106.0 + + def error_out(): + print( + f"Avocado version {minimum_version} or later is required.\n" + f"You may install it with: \n" + f" python3 -m pip install avocado-framework", + file=sys.stderr, + ) + sys.exit(1) + + try: + from avocado import VERSION + + if (float(VERSION)) < minimum_version: + error_out() + except ImportError: + error_out() + + +check_avocado_version() +from avocado.core.job import Job +from avocado.core.suite import TestSuite + + +def main(): + repo_root_path = os.path.abspath( + os.path.dirname(os.path.dirname(os.path.dirname(__file__))) + ) + config = { + "resolver.references": [ + os.path.join(repo_root_path, "test", "avocado", "static_checkers.json") + ], + "runner.identifier_format": "{args[0]}", + } + suite = TestSuite.from_config(config, name="static_checkers") + with Job(config, [suite]) as j: + return j.run() + + +if __name__ == "__main__": + sys.exit(main()) -- 2.45.2

3 4

[PATCH 0/5] Add vhost-user support to passt. (part 3)
by Laurent Vivier 15 Jul '24

15 Jul '24

This series of patches adds vhost-user support to passt and then allows passt to connect to QEMU network backend using virtqueue rather than a socket. With QEMU, rather than using to connect: -netdev stream,id=s,server=off,addr.type=unix,addr.path=/tmp/passt_1.socket we will use: -chardev socket,id=chr0,path=/tmp/passt_1.socket -netdev vhost-user,id=netdev0,chardev=chr0 -device virtio-net,netdev=netdev0 -object memory-backend-memfd,id=memfd0,share=on,size=$RAMSIZE -numa node,memdev=memfd0 The memory backend is needed to share data between passt and QEMU. Performance comparison between "-netdev stream" and "-netdev vhost-user": $ iperf3 -c localhost -p 10001 -t 60 -6 -u -b 50G socket: [ 5] 0.00-60.05 sec 95.6 GBytes 13.7 Gbits/sec 0.017 ms 6998988/10132413 (69%) receiver vhost-user: [ 5] 0.00-60.04 sec 237 GBytes 33.9 Gbits/sec 0.006 ms 53673/7813770 (0.69%) receiver $ iperf3 -c localhost -p 10001 -t 60 -4 -u -b 50G socket: [ 5] 0.00-60.05 sec 98.9 GBytes 14.1 Gbits/sec 0.018 ms 6260735/9501832 (66%) receiver vhost-user: [ 5] 0.00-60.05 sec 235 GBytes 33.7 Gbits/sec 0.008 ms 37581/7752699 (0.48%) receiver $ iperf3 -c localhost -p 10001 -t 60 -6 socket: [ 5] 0.00-60.00 sec 17.3 GBytes 2.48 Gbits/sec 0 sender [ 5] 0.00-60.06 sec 17.3 GBytes 2.48 Gbits/sec receiver vhost-user: [ 5] 0.00-60.00 sec 191 GBytes 27.4 Gbits/sec 0 sender [ 5] 0.00-60.05 sec 191 GBytes 27.3 Gbits/sec receiver $ iperf3 -c localhost -p 10001 -t 60 -4 socket: [ 5] 0.00-60.00 sec 15.6 GBytes 2.24 Gbits/sec 0 sender [ 5] 0.00-60.06 sec 15.6 GBytes 2.24 Gbits/sec receiver vhost-user: [ 5] 0.00-60.00 sec 189 GBytes 27.1 Gbits/sec 0 sender [ 5] 0.00-60.04 sec 189 GBytes 27.0 Gbits/sec receiver Laurent Vivier (5): packet: replace struct desc by struct iovec vhost-user: introduce virtio API vhost-user: introduce vhost-user API iov: add iov_count() vhost-user: add vhost-user Makefile | 4 +- checksum.c | 1 - conf.c | 18 +- iov.c | 35 +- iov.h | 2 + packet.c | 81 ++-- packet.h | 16 +- passt.c | 14 +- passt.h | 10 + pcap.c | 1 - tap.c | 106 ++++- tap.h | 5 +- tcp.c | 17 +- tcp_vu.c | 547 ++++++++++++++++++++++++ tcp_vu.h | 9 + udp.c | 54 ++- udp_internal.h | 39 ++ udp_vu.c | 237 +++++++++++ udp_vu.h | 8 + util.h | 11 + vhost_user.c | 1077 ++++++++++++++++++++++++++++++++++++++++++++++++ vhost_user.h | 137 ++++++ virtio.c | 442 ++++++++++++++++++++ virtio.h | 122 ++++++ 24 files changed, 2886 insertions(+), 107 deletions(-) create mode 100644 tcp_vu.c create mode 100644 tcp_vu.h create mode 100644 udp_internal.h create mode 100644 udp_vu.c create mode 100644 udp_vu.h create mode 100644 vhost_user.c create mode 100644 vhost_user.h create mode 100644 virtio.c create mode 100644 virtio.h -- 2.45.2

2 18

[PATCH 0/4] Small, assorted "hardening" fixes
by Stefano Brivio 08 Jul '24

08 Jul '24

All harmless issues as far as I can tell, but nice to fix. Stefano Brivio (4): conf: Copy up to MAXDNSRCH - 1 bytes, not MAXDNSRCH tcp_splice: Check return value of setsockopt() for SO_RCVLOWAT util, lineread, tap: Overflow checks on long signed sums and subtractions tap: Drop frames from guest whose length is more than remaining buffer conf.c | 2 +- lineread.c | 5 +++-- tap.c | 24 ++++++++++++++++++------ tcp_splice.c | 15 ++++++++++----- util.c | 7 +++++-- util.h | 46 +++++++++++++++++++++++++++++++++++++++++++++- 6 files changed, 82 insertions(+), 17 deletions(-) -- 2.43.0

2 20

[PATCH v2 0/5] Small, assorted "hardening" fixes
by Stefano Brivio 02 Jul '24

02 Jul '24

All harmless issues as far as I can tell, but nice to fix. v2: - in 3/5: - keep 'skip' in write_remainder() unsigned, and check for unsigned overflow instead - refactor sadd_overflow() and ssub_overflow() to use built-ins with automatic types, take ssize_t arguments, and deal with different ssize_t type widths - in 4/5: - switch l2len in tap_handler_passt() to uint32_t, as it really is unsigned and 32-bit wide - return if the length descriptor mismatches, instead of trying to proceed to the next frame - add 5/5 Stefano Brivio (5): conf: Copy up to MAXDNSRCH - 1 bytes, not MAXDNSRCH tcp_splice: Check return value of setsockopt() for SO_RCVLOWAT util, lineread, tap: Overflow checks on long signed sums and subtractions tap: Discard guest data on length descriptor mismatch conf: Use the right maximum buffer size for c->sock_path conf.c | 4 ++-- lineread.c | 5 +++-- tap.c | 31 +++++++++++++++++++---------- tcp_splice.c | 15 +++++++++----- util.c | 5 +++++ util.h | 55 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 96 insertions(+), 19 deletions(-) -- 2.43.0

2 7

[PATCH v6 00/26] RFC: Unified flow table
by David Gibson 27 Jun '24

27 Jun '24

This is a sixth draft of an implementation of more general "connection" tracking, as described at: https://pad.passt.top/p/NewForwardingModel This series changes the TCP connection table and hash table into a more general flow table that can track other protocols as well. Each flow uniformly keeps track of all the relevant addresses and ports, which will allow for more robust control of NAT and port forwarding. ICMP and UDP are converted to use the new flow table. This is based on the most recent series of flowtable preliminaries. NOTE: this fails the pasta UDP perf tests. This is one aspect of a curly issue I'm not sure how to deail with yet. So, this isn't ready for merge yet, but I'm posting because there have been a *lot* of changes since v5, and it could do with another round of review, Other caveats: * We roughly double the size of a connection/flow entry * UDP still has a number of forwarding bugs because we don't consider the local address when looking for a socket to use. There are at least some FIXMEs noted for this. Changes since v5: * flowside_from_af() is now static * Small fixes to state verification * Pass protocol specific types into deferred/timer callbacks * No longer require complete forwarding address info for the hash table (we won't have it for UDP) * Fix bugs with logging of flow addresses * Make sure to initialise sin_zero field sockaddr_from_inany * Added patch better typing parameters to flow type specific callbacks * Terminology change "forwarded side" to "target side" * Assorted wording and style tweaks based on Stefano's review * Fold introduction of struct flowside and populating the initiating side together * Manage outbound addresses via the flow table as well * Support for UDP * Correct type of 'b' in flowside_lookup() (was a signed int) Changes since v4: * flowside_from_af() no longer fills in unspecified addresses when passed NULL * Split and rename flow hash lookup function * Clarified flow state transitions, and enforced where practical * Made side 0 always the initiating side of a flow, rather than letting the protocol specific code decide * Separated pifs from flowside addresses to allow better structure packing Changes since v3: * Complex rebase on top of the many things that have happened upstream since v2. * Assorted other changes. * Replace TAPFSIDE() and SOCKFSIDE() macros with local variables. Changes since v2: * Cosmetic fixes based on review * Extra doc comments for enum flow_type * Rename flowside to flowaddrs which turns out to make more sense in light of future changes * Fix bug where the socket flowaddrs for tap initiated connections wasn't initialised to match the socket address we were using in the case of map-gw NAT * New flowaddrs_from_sock() helper used in most cases which is cleaner and should avoid bugs like the above * Using newer centralised workarounds for clang-tidy issue 58992 * Remove duplicate definition of FLOW_MAX as maximum flow type and maximum number of tracked flows * Rebased on newer versions of preliminary work (ICMP, flow based dispatch and allocation, bind/address cleanups) * Unified hash table as well as base flow table * Integrated ICMP Changes since v1: * Terminology changes - "Endpoint" address/port instead of "correspondent" address/port - "flowside" instead of "demiflow" * Actually move the connection table to a new flow table structure in new files * Significant rearrangement of earlier patchs on top of that new table, to reduce churn David Gibson (26): flow: Common address information for initiating side flow: Common address information for target side tcp, flow: Remove redundant information, repack connection structures tcp: Obtain guest address from flowside tcp: Manage outbound address via flow table tcp: Simplify endpoint validation using flowside information tcp_splice: Eliminate SPLICE_V6 flag tcp, flow: Replace TCP specific hash function with general flow hash flow, tcp: Generalise TCP hash table to general flow hash table tcp: Re-use flow hash for initial sequence number generation icmp: Remove redundant id field from flow table entry icmp: Obtain destination addresses from the flowsides icmp: Look up ping flows using flow hash icmp: Eliminate icmp_id_map icmp: Manage outbound socket address via flow table flow, tcp: Flow based NAT and port forwarding for TCP flow, icmp: Use general flow forwarding rules for ICMP fwd: Update flow forwarding logic for UDP udp: Create flow table entries for UDP udp: Direct traffic from tap according to flow table udp: Direct traffic from host to guest tap according to flow table udp: Direct spliced traffic according to flow table udp: Remove 'splicesrc' tracking udp: Remove tap port flags field udp: Remove rdelta port forwarding maps udp: Eliminate 'splice' flag from epoll reference Makefile | 2 +- conf.c | 14 +- flow.c | 402 +++++++++++++++++++++++++- flow.h | 45 +++ flow_table.h | 45 ++- fwd.c | 177 +++++++++++- fwd.h | 9 + icmp.c | 105 +++---- icmp_flow.h | 2 - inany.h | 31 +- passt.h | 3 + tap.c | 11 - tap.h | 1 - tcp.c | 524 +++++++++------------------------- tcp_buf.c | 6 +- tcp_conn.h | 51 ++-- tcp_internal.h | 10 +- tcp_splice.c | 98 +------ tcp_splice.h | 5 +- udp.c | 757 +++++++++++++++++++++++-------------------------- udp.h | 22 +- udp_flow.h | 25 ++ util.c | 6 +- util.h | 3 + 24 files changed, 1328 insertions(+), 1026 deletions(-) create mode 100644 udp_flow.h -- 2.45.2

2 34

[PATCH v5 0/9] Fixes for early logging/prints and related cleanups
by Stefano Brivio 26 Jun '24

26 Jun '24

The most apparent issue fixed by this series is the one from 4/6: with a log file configured, we wouldn't print to standard error anymore, during initialisation, which means that users such as libvirt lost the ability to report meaningful error messages that occurred during initialisation, in that case. v5: - in 4/8, rename the new flag once more to 'log_runtime': we don't want to log to standard error if we're running in foreground, a log file is given, and initialisation is done, otherwise debugging pasta when it spawns its own shell becomes rather impractical - add 9/8, taking care of a direct stderr print left-over v4: - in 4/8, name the new flag 'log_daemonised' instead of 'log_daemon_ready' v3: - add 2/8: we don't really need --stderr anymore - in 5/8, save errno at the beginning of the _perror() helper - in 7/8, avoid assigning errno to whatever return code we have just for the sake of using the new helpers: strerror() is actually less convoluted than that - add 8/8: there's no need to call __openlog() with a log file v2: - turn flag bitmap into simple, separate boolean flags - move errno description after message in _perror() functions - make some of the old perror() messages more descriptive *** BLURB HERE *** Stefano Brivio (9): conf, passt: Don't try to log to stderr after we close it conf, passt: Make --stderr do nothing, and deprecate it conf, log: Instead of abusing log levels, add log_conf_parsed flag log, passt: Always print to stderr before initialisation is complete log: Add _perror() logging function variants treewide: Replace perror() calls with calls to logging functions treewide: Replace strerror() calls conf, passt: Don't call __openlog() if a log file is used log: Don't report syslog failures to stderr after initialisation arch.c | 10 ++++----- conf.c | 41 ++++++++++++--------------------- fwd.c | 2 +- isolation.c | 46 ++++++++++++++++--------------------- log.c | 65 ++++++++++++++++++++++++++++++++--------------------- log.h | 25 +++++++++++++++++---- netlink.c | 4 ++-- passt.1 | 9 ++++---- passt.c | 64 +++++++++++++++++++--------------------------------- passt.h | 2 -- pasta.c | 27 +++++++++------------- pcap.c | 8 +++---- tap.c | 14 ++++++------ tcp.c | 24 +++++++------------- util.c | 12 +++++----- 15 files changed, 164 insertions(+), 189 deletions(-) -- 2.43.0

2 16

[PATCH 0/2] Fix bug #80 again
by Stefano Brivio 24 Jun '24

24 Jun '24

Oops. Stefano Brivio (2): Revert "udp: Make rport calculation more local" udp: Reduce scope of rport in udp_invert_portmap() udp.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) -- 2.43.0

2 5

passt: new version 2024_06_24.1ee2eca available
by Stefano Brivio 24 Jun '24

24 Jun '24

The new version with tag 2024_06_24.1ee2eca includes the following changes: 1ee2eca udp: Reduce scope of rport in udp_invert_portmap() 0546975 Revert "udp: Make rport calculation more local" c66f034 log: Don't report syslog failures to stderr after initialisation e7323e5 conf, passt: Don't call __openlog() if a log file is used dba7f0f treewide: Replace strerror() calls 92a22fe treewide: Replace perror() calls with calls to logging functions c1140df log: Add _perror() logging function variants afd9cdc log, passt: Always print to stderr before initialisation is complete 8c2f24a conf, log: Instead of abusing log levels, add log_conf_parsed flag bca0fef conf, passt: Make --stderr do nothing, and deprecate it b748016 conf, passt: Don't try to log to stderr after we close it 65923ba conf: Accept duplicate and conflicting options, the last one wins 62de614 netlink: Strip nexthop identifiers when duplicating routes 1544a43 passt.1, qrap.1: align license description with SPDX identifier f301bb1 netlink: Ignore EHOSTUNREACH failures when duplicating routes 450a613 netlink: With no default route, pick the first interface with a route 54a9d38 tcp: Don't rely on bind() to fail to decide that connection target is valid 020ff7a siphash: Remove stale prototypes 7e87bd9 udp: Move management of udp[46]_localname into udp_splice_send() ff57f8d udp: Rework how we divide queued datagrams between sending methods 63db7dc udp: Fold checking of splice flag into udp_mmh_splice_port() 523fbc5 util: Split construction of bind socket address from the rest of sock_l4() 4070bac tap: use in->buf_size rather than sizeof(pkt_buf) 7290335 iov: remove iov_copy() 0c335d7 vhost-user: compare mode MODE_PASTA and not MODE_PASST 377b666 udp: rename udp_sock_handler() to udp_buf_sock_handler() e7ac995 udp: refactor UDP header update functions 9ecf7fe tap: refactor packets handling functions fba2b54 tcp: move buffers management functions to their own file ec26fa0 tcp: extract buffer management from tcp_send_flag() d949667 cppcheck: Suppress constParameterCallback errors https://passt.top/passt/log/?qt=range&q=2024_06_07.8a83b53..2024_06_24.1ee2… Packages: - Alpine Linux: https://pkgs.alpinelinux.org/packages?name=passt - Arch Linux: https://www.archlinux.org/packages/extra/x86_64/passt/ https://archlinuxarm.org/packages/aarch64/passt https://archlinuxarm.org/packages/armv7h/passt - Chimera: https://pkgs.chimera-linux.org/packages?name=passt - Debian tracker: https://tracker.debian.org/pkg/passt - Copr (CentOS Stream, EPEL, Fedora, Mageia): https://copr.fedorainfracloud.org/coprs/sbrivio/passt/build/7661289/ permanent mirror: https://passt.top/builds/copr/0^20240624.g1ee2eca/ - Fedora updates: https://bodhi.fedoraproject.org/updates/?packages=passt - Gentoo versions: https://packages.gentoo.org/packages/net-misc/passt - GNU Guix: https://packages.guix.gnu.org/packages/passt/ - openSUSE: https://software.opensuse.org/package/passt - PLD Linux: https://git.pld-linux.org/cgi-bin/gitweb.cgi?p=packages/passt.git - Solus: https://github.com/getsolus/packages/tree/main/packages/p/passt - Ubuntu tracker: https://launchpad.net/ubuntu/+source/passt - Void Linux: https://voidlinux.org/packages/?q=passt - Static builds: - Package for other RPM-based distributions, x86_64 only: https://passt.top/builds/latest/x86_64/passt-g1ee2eca-1.x86_64.rpm - x86_64 static binaries: https://passt.top/builds/latest/x86_64/ - Debian package, from x86_64 static build: https://passt.top/builds/latest/x86_64/passt_1ee2eca-1_all.deb -- Stefano

1 0