August 2026 - dev

[PATCH 0/7] multithreading: Add worker threads for queue pair processing
by Laurent Vivier 14 Aug '26

14 Aug '26

NOTE: This series has known stability issues (hangs and crashes under load) and is submitted for reference and testing purposes. This series adds pthread-based worker threads to enable parallel processing of queue pairs in vhost-user mode. Each queue pair gets its own worker thread with a dedicated epollfd. Thread 0 handles control events (listen sockets, vhost-user commands, repair, netlink) plus queue pair 0. Threads 1-15 handle only their assigned queue pair events. Threads start/stop dynamically via SET_VRING_ENABLE. Known issues include system hangs under load and crashes when virtqueues are exhausted. A temporary workaround in virtio.c masks underlying coordination problems between worker threads that need proper fixes. Based-on: 20260731162329.3552800-1-lvivier(a)redhat.com Laurent Vivier (7): vhost_user: Reset vq enable flag in vu_cleanup() threading: Add basic threading infrastructure passt: Integrate main event loop with threading infrastructure flow: Delegate epoll file descriptor management to threading subsystem ctx: Remove epollfd from context structure vhost-user: Add per-qpair worker threads virtio: Prevent crash on virtqueue exhaustion with temporary workaround Makefile | 8 +- conf.c | 10 ++- flow.c | 12 +-- flow.h | 4 +- fwd.c | 4 +- isolation.c | 6 +- netlink.c | 4 +- passt.c | 151 ++++++++++++++++++++++++-------------- passt.h | 7 +- pasta.c | 3 +- pif.c | 3 +- repair.c | 12 ++- tap.c | 9 ++- tcp.c | 3 +- tcp_splice.c | 1 + threading.c | 202 +++++++++++++++++++++++++++++++++++++++++++++++++++ threading.h | 22 ++++++ vhost_user.c | 34 +++++++-- vhost_user.h | 1 + virtio.c | 8 +- vu_common.c | 13 ++-- 21 files changed, 417 insertions(+), 100 deletions(-) create mode 100644 threading.c create mode 100644 threading.h -- 2.54.0

2 13

Patch for the close_range() musl build failure
by Huang Jing 13 Aug '26

13 Aug '26

Please see attached patch.

3 6

[PATCH v2 00/10] multithreading: Prepare data structures for concurrent queue pair workers
by Laurent Vivier 05 Aug '26

05 Aug '26

This series makes the passt data structures safe for concurrent access by multiple worker threads, each handling a different queue pair. It builds on the multiqueue series which threads the queue pair parameter through the network stack. The changes fall into four categories: 1. Per-queue-pair data isolation: - Convert packet pools (pool_tap4, pool_tap6) to per-queue-pair arrays - Convert L4 sequence batching arrays (tap4_l4, tap6_l4) similarly - Move flow and TCP timer state out of global/context structures into per-caller parameters 2. Stack-local buffers: - Move static iovec and virtqueue buffers onto the stack in tcp.c, tcp_vu.c, and udp_vu.c so each thread operates on its own copy 3. Locking for remaining shared state: - pthread_mutex on pre-opened socket pools (init_sock_pool4/6) - pthread_rwlock on the flow table, hash table, and free list, with read locks for lookups and write locks for mutations - _Thread_local for flow_new_entry so each thread independently tracks its in-progress allocation - Per-qpair filtering in flow_defer_handler() so each worker only processes its own flows 4. Lock-free flow migration between queue pairs: - When guest steers traffic to a different TX queue, flows migrate lazily using a to_migrate[] array - flow_migrate_mark() records target qpair from tap path (write-once, no lock needed) - flow_migrate_epollfd() completes migration on next socket/timer event by moving fd and updating qpair on the old thread - TCP timers migrate the same way using new tcp_timer_epoll_add() helper Global operations that don't need per-queue-pair handling (tcp_payload_flush, socket pool refills) are guarded to run only on queue pair 0. v2: - Rebase on latest master and multiqueue v6 - Split v1's "flow: Add mutex and per-qpair filtering..." into two patches: - "flow: Add locking, per-qpair filtering, and intermediate state handling" (locking only) - "flow: Add lazy, lock-free flow migration between queue pairs" (migration logic separated) - Changed from simple mutex to pthread_rwlock for flow table (read locks for lookups, write locks for mutations and free-list rebuild) - Updated flow_foreach() and flow_defer_handler() to handle intermediate flow states (NEW, INI, TGT, TYPED) that can be observed between flow_alloc() and FLOW_ACTIVATE() when lock is released - New patch: "tcp: Extract tcp_timer_epoll_add() helper" - factors out epoll registration for timer migration support - flow_migrate_epollfd() now uses tcp_timer_epoll_add() for timer migration v1: - Initial submission Based-on: 20260616125130.1324274-1-lvivier(a)redhat.com Laurent Vivier (10): tap: Convert packet pools to per-queue-pair arrays for multiqueue tap: Make L4 sequence pools per-qpair for thread safety tcp: Make static buffers stack-local for thread safety udp_vu: Make virtqueue buffers stack-local for thread safety flow: Make flow timer per-caller for thread safety tcp: Make TCP timer state per-caller and guard global tasks tcp: Protect init socket pools with mutex for thread safety tcp: Extract tcp_timer_epoll_add() helper flow: Add locking, per-qpair filtering, and intermediate state handling flow: Add lazy, lock-free flow migration between queue pairs flow.c | 222 ++++++++++++++++++++++++++++++++++++++++++++++----- flow.h | 5 +- flow_table.h | 2 +- icmp.c | 10 ++- passt.c | 37 +++++---- tap.c | 109 ++++++++++++++----------- tap.h | 2 +- tcp.c | 108 ++++++++++++++++++------- tcp.h | 9 +-- tcp_vu.c | 33 +++++--- udp.c | 3 + udp_flow.c | 4 + udp_vu.c | 4 +- vu_common.c | 2 +- 14 files changed, 409 insertions(+), 141 deletions(-) -- 2.54.0

2 22

[PATCH v6 00/12] vhost-user: Add multiqueue support
by Laurent Vivier 03 Aug '26

03 Aug '26

This series implements multiqueue support for vhost-user mode, allowing passt to utilize multiple queue pairs for improved network throughput when used with multi-CPU guest VMs. While this version uses a single thread for packet processing, it enables the guest kernel to distribute network traffic across multiple queues and vCPUs. The implementation advertises support for up to 16 queue pairs (32 virtqueues) by setting VIRTIO_NET_F_MQ and VHOST_USER_PROTOCOL_F_MQ feature flags. Packets are routed to the appropriate RX queue based on which TX queue they originated from, following the virtio specification's automatic receive steering requirements. This series adds: - Multiqueue capability advertisement (VIRTIO_NET_F_MQ and VHOST_USER_PROTOCOL_F_MQ features) - Queue pair parameter throughout the network stack, propagated through all protocol handlers (TCP, UDP, ICMP, ARP, DHCP, DHCPv6, NDP) - Flow-aware queue routing that tracks the originating TX queue for each flow and routes return packets to the corresponding RX queue - Epoll fd derived directly from queue pair, removing the separate epollid field and simplifying flow-to-queue association - Test coverage with VHOST_USER_MQ environment variable to validate multiqueue functionality across all protocols (TCP, UDP, ICMP) and services (DHCP, NDP) Current behavior: - TX queue selection is controlled by the guest kernel's networking stack - RX packets are routed to queues based on their associated flows, with the queue assignment updated on each packet from TX to maintain affinity - Host-initiated flows (e.g., from socket-side connections) currently default to queue pair 0 The changes are transparent to single-queue operation - passt/pasta modes and single-queue vhost-user configurations continue to work unchanged, always using queue pair 0. v6: - Rebase on latest master - Moved flow_migrate()/FLOW_MIGRATE() to a following series - this series now focuses solely on basic multiqueue support without dynamic flow migration between queue pairs. - qpair is now a parameter of flow_alloc(). - Made flow_setqp() static (only called during flow allocation) - Updated for upstream changes to flow logging infrastructure (flow_log__(), new logging macros with improved perror and details handling) - Updated for upstream migration handler signature changes (added 'now' parameter to flow_migrate_source_pre(), flow_migrate_source(), and flow_migrate_target()) v5: - Rebase - Dropped "tap: Convert packet pools to per-queue-pair arrays for multiqueue" - Split "vhost-user: Add queue pair parameter throughout the network stack" into separate per-protocol patches (tap, arp, tcp, udp, dhcp/dhcpv6, icmp, ndp) for easier review - New patch: "flow: Derive epoll fd from queue pair, removing epollid field" derives epoll fd directly from queue pair, removing the redundant epollid field and adding flow_migrate()/FLOW_MIGRATE() for existing flows - tcp_keepalive() and tcp_inactivity() now filter flows by queue pair - Renamed "vhost-user: Enable multiqueue" to "vhost-user: Advertise multiqueue support" v4: - Rebase - Added QPAIR_DEFAULT, QPAIR_FROMGUEST_QUEUE(), QPAIR_TOGUEST_QUEUE(), and QPAIR_FROM_QUEUE() macros to centralize queue pair calculations - Replaced hardcoded 0 with QPAIR_DEFAULT throughout for clarity - Set queue pair at flow creation time in icmp_ping_new() and udp_flow_new() rather than after creation - Removed vhost-user-specific queue macros (VHOST_USER_TX_QUEUE, VHOST_USER_IS_QUEUE_TX, VHOST_USER_IS_QUEUE_RX) in favor of generic QPAIR_* macros for better consistency v3: - Removed --max-qpairs configuration option - multiqueue support is now always enabled up to 16 queue pairs without requiring explicit configuration - Replaced "tap: Add queue pair parameter throughout the packet processing path" with "tap: Convert packet pools to per-queue-pair arrays for multiqueue" - simplified implementation by converting global pools to arrays rather than passing pool parameters throughout - Changed qpair parameter type from int to unsigned int throughout the codebase - Simplified test infrastructure - queues parameter is always set on netdev, mq=true added to virtio-net only when VHOST_USER_MQ > 1 - Updated QEMU usage hints to always show multiqueue-capable command line v2: - New patch: "tap: Remove pool parameter from tap4_handler() and tap6_handler()" to clean up unused parameters before adding queue pair parameter - Changed to one packet pool per queue pair instead of shared pools across all queue pairs - Split "multiqueue: Add queue-aware flow management..." into two patches: - "tap: Add queue pair parameter throughout the packet processing path" - "flow: Add queue pair tracking to flow management" - Updated test infrastructure patch with refined implementation Laurent Vivier (12): tap: Remove pool parameter from tap4_handler() and tap6_handler() vhost-user: Advertise multiqueue support test: Add multiqueue support to vhost-user test infrastructure tap: Thread queue pair through all remaining tap paths arp: Pass queue pair explicitly through ARP send path tcp: Pass queue pair explicitly through TCP send path udp: Pass queue pair explicitly through UDP send path dhcp/dhcpv6: Pass queue pair explicitly through DHCP send path icmp: Pass queue pair explicitly through ICMP send path ndp: Pass queue pair explicitly through NDP send path flow: Add queue pair tracking to flow management flow: Derive epoll fd from queue pair, removing epollid field arp.c | 9 ++- arp.h | 2 +- dhcp.c | 5 +- dhcp.h | 2 +- dhcpv6.c | 13 +-- dhcpv6.h | 2 +- flow.c | 45 +++++------ flow.h | 20 ++--- flow_table.h | 2 +- icmp.c | 24 +++--- icmp.h | 7 +- ndp.c | 33 ++++---- ndp.h | 2 +- passt.c | 25 +++--- passt.h | 10 +++ tap.c | 118 ++++++++++++++------------- tap.h | 19 ++--- tcp.c | 211 ++++++++++++++++++++++++++++--------------------- tcp.h | 15 ++-- tcp_buf.c | 8 +- tcp_internal.h | 6 +- tcp_splice.c | 1 - tcp_vu.c | 19 +++-- tcp_vu.h | 5 +- test/lib/setup | 21 +++-- test/run | 23 ++++++ udp.c | 48 ++++++----- udp.h | 9 ++- udp_flow.c | 27 ++++--- udp_flow.h | 13 +-- udp_internal.h | 3 +- udp_vu.c | 7 +- udp_vu.h | 3 +- vhost_user.c | 12 +-- vhost_user.h | 9 --- virtio.h | 2 +- vu_common.c | 22 +++--- vu_common.h | 3 +- 38 files changed, 462 insertions(+), 343 deletions(-) -- 2.54.0

2 22