- dev - passt

[PATCH] netlink: Allow NULL to be passed as addr parameter to nl_addr_get (again)
by David Gibson 11 Mar '26

11 Mar '26

A recent fix to nl_addr_get() unfortunately introduced a different problem. 251e676bd0bf removed a test if @addr was non-NULL. conf_ip6() relied on being able to pass NULL in order to update the link-local address without updating the main address (because it was already set by the user). This means that 'pasta -a 2001:db8::1' will SEGV. Reverse the incorrect part of the change to fix this. Fixes: 251e676bd0bf ("netlink: Return prefix length for IPv6 addresses in nl_addr_get()") Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au> --- netlink.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/netlink.c b/netlink.c index ee9a6b16..e07b47f4 100644 --- a/netlink.c +++ b/netlink.c @@ -788,7 +788,7 @@ int nl_addr_get(int s, unsigned int ifi, sa_family_t af, (af == AF_INET6 && rta->rta_type != IFA_ADDRESS)) continue; - if (ifa->ifa_prefixlen > prefix_max && + if (ifa->ifa_prefixlen > prefix_max && addr && (af == AF_INET || ifa->ifa_scope < RT_SCOPE_LINK)) { memcpy(addr, RTA_DATA(rta), RTA_PAYLOAD(rta)); -- 2.53.0

1 0

[PATCH 0/6] Unify TCP and UDP forwarding tables
by David Gibson 11 Mar '26

11 Mar '26

This will make the dynamic update protocol and future forwarding extensions easier. Along the way I spotted a number of other minor faults, and have fixed those too. David Gibson (6): conf, fwd: Make overall forwarding mode local to conf path tcp: Remove stale description of port_to_tap field fwd: Don't initialise unused port bitmaps Fix misnamed field in struct ctx comments fwd: Split forwarding table from port scanning state fwd: Unify TCP and UDP forwarding tables conf.c | 129 ++++++++++++++++------------- flow.c | 23 ++---- fwd.c | 247 ++++++++++++++++++++++++++++++++++---------------------- fwd.h | 58 ++++++------- passt.c | 3 + passt.h | 7 +- tcp.c | 8 +- tcp.h | 11 +-- udp.c | 9 +-- udp.h | 8 +- 10 files changed, 282 insertions(+), 221 deletions(-) -- 2.53.0

2 9

[PATCH v4] netlink: Return prefix length for IPv6 addresses in nl_addr_get()
by Jon Maloy 10 Mar '26

10 Mar '26

nl_addr_get() was not setting the prefix_len output parameter for IPv6 addresses, only for IPv4. This meant callers always got 0 for IPv6, forcing them to use a hardcoded default (64). Fix by assigning *prefix_len even in the IPv6 case. Signed-off-by: Jon Maloy <jmaloy(a)redhat.com> --- v2: - Simplified according to feedback from S. Brivio - Added test for AF_INET link local property v3: - Corrected claim about convention for IPv4 link local address scope in commit log. v4: - Reverted from the IPv4 link local exclusion change --- netlink.c | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/netlink.c b/netlink.c index 82a2f0c..ee9a6b1 100644 --- a/netlink.c +++ b/netlink.c @@ -752,7 +752,7 @@ int nl_addr_set_ll_nodad(int s, unsigned int ifi) * @ifi: Interface index in outer network namespace * @af: Address family * @addr: Global address to fill - * @prefix_len: Mask or prefix length, to fill (for IPv4) + * @prefix_len: Mask or prefix length, to fill * @addr_l: Link-scoped address to fill (for IPv6) * * Return: 0 on success, negative error code on failure @@ -788,16 +788,11 @@ int nl_addr_get(int s, unsigned int ifi, sa_family_t af, (af == AF_INET6 && rta->rta_type != IFA_ADDRESS)) continue; - if (af == AF_INET && ifa->ifa_prefixlen > prefix_max) { + if (ifa->ifa_prefixlen > prefix_max && + (af == AF_INET || ifa->ifa_scope < RT_SCOPE_LINK)) { memcpy(addr, RTA_DATA(rta), RTA_PAYLOAD(rta)); prefix_max = *prefix_len = ifa->ifa_prefixlen; - } else if (af == AF_INET6 && addr && - ifa->ifa_scope < RT_SCOPE_LINK && - ifa->ifa_prefixlen > prefix_max) { - memcpy(addr, RTA_DATA(rta), RTA_PAYLOAD(rta)); - - prefix_max = ifa->ifa_prefixlen; } if (addr_l && -- 2.52.0

3 2

[PATCH v4] iov: Add iov_truncate() helper and use it in vu handlers
by Laurent Vivier 10 Mar '26

10 Mar '26

Add a generic iov_truncate() function that truncates an IO vector to a given number of bytes, returning the number of iov entries that contain data after truncation. Use it in udp_vu_sock_recv() and tcp_vu_sock_recv() to replace the open-coded truncation logic that adjusted iov entries after recvmsg(). Also convert the direct iov_len assignment in tcp_vu_send_flag() to use iov_truncate() for consistency. Add an ASSERT() in tcp_vu_data_from_sock() to quiet the Coverity error: passt/tcp_vu.c:457:3: 19. overflow_const: Expression "dlen + hdrlen", where "dlen" is known to be equal to -86, and "hdrlen" is known to be equal to 86, underflows the type of "dlen + hdrlen", which is type "unsigned long". Signed-off-by: Laurent Vivier <lvivier(a)redhat.com> --- Notes: v4: add ASSERT() to quiet the Coverity error v3: use in tcp_vu_send_flag() too v2: use iov_truncate() in udp_vu_sock_recv() too iov.c | 22 ++++++++++++++++++++++ iov.h | 1 + tcp_vu.c | 20 ++++++++------------ udp_vu.c | 12 +++--------- 4 files changed, 34 insertions(+), 21 deletions(-) diff --git a/iov.c b/iov.c index ad726daa4cd8..31a3f5bc29e5 100644 --- a/iov.c +++ b/iov.c @@ -147,6 +147,28 @@ size_t iov_size(const struct iovec *iov, size_t iov_cnt) return len; } +/** + * iov_truncate() - Truncate an IO vector to a given number of bytes + * @iov: IO vector (modified) + * @iov_cnt: Number of entries in @iov + * @size: Total number of bytes to keep + * + * Return: number of iov entries that contain data after truncation + */ +size_t iov_truncate(struct iovec *iov, size_t iov_cnt, size_t size) +{ + size_t i, offset; + + i = iov_skip_bytes(iov, iov_cnt, size, &offset); + + if (i < iov_cnt) { + iov[i].iov_len = offset; + i += !!offset; + } + + return i; +} + /** * iov_tail_prune() - Remove any unneeded buffers from an IOV tail * @tail: IO vector tail (modified) diff --git a/iov.h b/iov.h index d1ab91a94e22..b4e50b0fca5a 100644 --- a/iov.h +++ b/iov.h @@ -29,6 +29,7 @@ size_t iov_from_buf(const struct iovec *iov, size_t iov_cnt, size_t iov_to_buf(const struct iovec *iov, size_t iov_cnt, size_t offset, void *buf, size_t bytes); size_t iov_size(const struct iovec *iov, size_t iov_cnt); +size_t iov_truncate(struct iovec *iov, size_t iov_cnt, size_t size); /* * DOC: Theory of Operation, struct iov_tail diff --git a/tcp_vu.c b/tcp_vu.c index 88be232dca66..fd734e857b3b 100644 --- a/tcp_vu.c +++ b/tcp_vu.c @@ -131,7 +131,7 @@ int tcp_vu_send_flag(const struct ctx *c, struct tcp_tap_conn *conn, int flags) return ret; } - flags_elem[0].in_sg[0].iov_len = hdrlen + optlen; + iov_truncate(&flags_iov[0], 1, hdrlen + optlen); payload = IOV_TAIL(flags_elem[0].in_sg, 1, hdrlen); if (flags & KEEPALIVE) @@ -192,9 +192,9 @@ static ssize_t tcp_vu_sock_recv(const struct ctx *c, struct vu_virtq *vq, struct msghdr mh_sock = { 0 }; uint16_t mss = MSS_GET(conn); int s = conn->sock; - ssize_t ret, len; size_t hdrlen; int elem_cnt; + ssize_t ret; int i; *iov_cnt = 0; @@ -247,15 +247,7 @@ static ssize_t tcp_vu_sock_recv(const struct ctx *c, struct vu_virtq *vq, ret -= already_sent; /* adjust iov number and length of the last iov */ - len = ret; - for (i = 0; len && i < elem_cnt; i++) { - struct iovec *iov = &elem[i].in_sg[0]; - - if (iov->iov_len > (size_t)len) - iov->iov_len = len; - - len -= iov->iov_len; - } + i = iov_truncate(&iov_vu[DISCARD_IOV_NUM], elem_cnt, ret); /* adjust head count */ while (*head_cnt > 0 && head[*head_cnt - 1] >= i) @@ -448,10 +440,14 @@ int tcp_vu_data_from_sock(const struct ctx *c, struct tcp_tap_conn *conn) for (i = 0, previous_dlen = -1, check = NULL; i < head_cnt; i++) { struct iovec *iov = &elem[head[i]].in_sg[0]; int buf_cnt = head[i + 1] - head[i]; - ssize_t dlen = iov_size(iov, buf_cnt) - hdrlen; + size_t frame_size = iov_size(iov, buf_cnt); bool push = i == head_cnt - 1; + ssize_t dlen; size_t l2len; + ASSERT(frame_size >= hdrlen); + + dlen = frame_size - hdrlen; vu_set_vnethdr(iov->iov_base, buf_cnt); /* The IPv4 header checksum varies only with dlen */ diff --git a/udp_vu.c b/udp_vu.c index 3520f89e5671..5effca777e0a 100644 --- a/udp_vu.c +++ b/udp_vu.c @@ -71,9 +71,9 @@ static int udp_vu_sock_recv(const struct ctx *c, struct vu_virtq *vq, int s, bool v6, ssize_t *dlen) { const struct vu_dev *vdev = c->vdev; - int iov_cnt, idx, iov_used; - size_t off, hdrlen, l2len; struct msghdr msg = { 0 }; + int iov_cnt, iov_used; + size_t hdrlen, l2len; ASSERT(!c->no_udp); @@ -115,13 +115,7 @@ static int udp_vu_sock_recv(const struct ctx *c, struct vu_virtq *vq, int s, iov_vu[0].iov_base = (char *)iov_vu[0].iov_base - hdrlen; iov_vu[0].iov_len += hdrlen; - /* count the numbers of buffer filled by recvmsg() */ - idx = iov_skip_bytes(iov_vu, iov_cnt, *dlen + hdrlen, &off); - - /* adjust last iov length */ - if (idx < iov_cnt) - iov_vu[idx].iov_len = off; - iov_used = idx + !!off; + iov_used = iov_truncate(iov_vu, iov_cnt, *dlen + hdrlen); /* pad frame to 60 bytes: first buffer is at least ETH_ZLEN long */ l2len = *dlen + hdrlen - VNET_HLEN; -- 2.53.0

3 2

[PATCH 3/6] fwd: Don't initialise unused port bitmaps
by David Gibson 10 Mar '26

10 Mar '26

Since b223bec48213 ("fwd, tcp, udp: Set up listening sockets based on forward table"), the 'map' field of struct fwd_ports has only been used for forwarding rules with the FWD_SCAN bit, where it is initialised from a port scan. However, we still pointlessly initialise it for rules with !FWD_SCAN. Remove the unneeded code. Fixes: b223bec48213 ("fwd, tcp, udp: Set up listening sockets based on forward table") Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au> --- fwd.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/fwd.c b/fwd.c index c5090fb1..9e455094 100644 --- a/fwd.c +++ b/fwd.c @@ -405,13 +405,8 @@ void fwd_rule_add(struct fwd_ports *fwd, uint8_t flags, new->socks = &fwd->socks[fwd->sock_count]; fwd->sock_count += num; - for (port = new->first; port <= new->last; port++) { + for (port = new->first; port <= new->last; port++) new->socks[port - new->first] = -1; - - /* Fill in the legacy forwarding data structures to match the table */ - if (!(new->flags & FWD_SCAN)) - bitmap_set(fwd->map, port); - } } /** -- 2.53.0

1 0

[PATCH v5 00/13] Introduce multiple addresses and late binding
by Jon Maloy 10 Mar '26

10 Mar '26

This series adds handling of multiple addresses into a unified address array, so that a guest can see the same addresses on his own interface. o All addresses are stored as union inany_addr o User configured addresses are marked with a USER flag. o Host provided addresses are marked with a HOST flag. o Link local addresses are also marked with a LINKLOCAL flag. o Addresses the guest is actually using are marked with an OBSERVED flag. o The address selected for IPv4 DHCP offers is marked with a DHCP flag. We also include a commit to support late binding of interfaces, addresses, and routes. v2: - Added the earlier standalone CIDR commit to the head of the series. - Replaced the guest namespace interface subscriptions with just an address observation feature, so that it works with both PASTA and PASST. - Unified 'no_copy_addrs' and 'copy_addrs' code paths, as suggested by David G. - Multiple other changes, also based on feedback from David. - Removed the host interface subscription patches, -for now. I intend to re-add them once this series is applied. - Outstanding question: When do we add an IPv4 link local address to the guest? Only in local/opaque mode? Only when explicitly requested? Always? v3: - Unified the IPv4 and IPv6 arrays into one array - Changed prefix_len to always be in IPv6/IpV4 mapped format - Updated migration protocol to v3, handling multiple addresses - Many other smaller changes, based on feedback from the PASST team v4: - Numerous changes based on feedback - Added several new commits, mostly broken out of the pre-existing ones. v5: - Re-introduced multiple OBSERVED addresses. This actually turned out to be cleaner and with more predictable behaviour than allowing only one. - Included the DHCP and NDP patches from previous versions, improved and updated according to feedback from the team. - Likewise re-included the host-side netlink commit to support late binding. Jon Maloy (13): ip: Introduce unified multi-address data structures ip: Introduce for_each_addr() macro for address iteration fwd: Unify guest accessibility checks with unified address array arp: Check all configured addresses in ARP filtering netlink: Return prefix length for IPv6 addresses in nl_addr_get() conf: Allow multiple -a/--address options per address family ip: Track observed guest IPv4 addresses in unified address array ip: Track observed guest IPv6 addresses in unified address array migrate: Rename v1 address functions to v2 for clarity migrate: Update protocol to v3 for multi-address support dhcp, dhcpv6: Select addresses for DHCP distribution ndp: Support advertising multiple prefixes in Router Advertisement netlink: Add host-side monitoring for late template interface binding arp.c | 15 +- conf.c | 200 +++++++++++++++---------- conf.h | 9 ++ dhcp.c | 30 ++-- dhcp.h | 2 +- dhcpv6.c | 78 +++++++--- dhcpv6.h | 2 +- epoll_type.h | 2 + fwd.c | 263 +++++++++++++++++++++++++-------- fwd.h | 9 ++ inany.h | 3 + ip.h | 5 + isolation.c | 5 + migrate.c | 183 ++++++++++++++++++++--- ndp.c | 134 ++++++++++++----- netlink.c | 401 ++++++++++++++++++++++++++++++++++++++++++++++++++- netlink.h | 3 + passt.c | 5 + passt.h | 104 +++++++++++-- pasta.c | 39 +++-- tap.c | 67 ++++++--- 21 files changed, 1282 insertions(+), 277 deletions(-) -- 2.52.0

3 29

[PATCH v3] netlink: Return prefix length for IPv6 addresses in nl_addr_get()
by Jon Maloy 10 Mar '26

10 Mar '26

nl_addr_get() was not setting the prefix_len output parameter for IPv6 addresses, only for IPv4. This meant callers always got 0 for IPv6, forcing them to use a hardcoded default (64). Fix by assigning *prefix_len even in the IPv6 case. We also add another functional change. We now check for if an AF_INET address is link local, in which case we have to skip it. Although it is conventional to set the scope of such addresses to RT_SCOPE_LINK, this is not stated in any RFC, and we cannot trust it to have been set correctly by the user, just as we cannot trust it to have been set correctly for any other AF_INET address. We therefore add this check explicitly on the address itself. Signed-off-by: Jon Maloy <jmaloy(a)redhat.com> --- v2: - Simplified according to feedback from S. Brivio - Added test for AF_INET link local property v3: - Corrected claim about convention for IPv4 link local address scope in commit log. --- netlink.c | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/netlink.c b/netlink.c index 82a2f0c..f0e8fa7 100644 --- a/netlink.c +++ b/netlink.c @@ -752,7 +752,7 @@ int nl_addr_set_ll_nodad(int s, unsigned int ifi) * @ifi: Interface index in outer network namespace * @af: Address family * @addr: Global address to fill - * @prefix_len: Mask or prefix length, to fill (for IPv4) + * @prefix_len: Mask or prefix length, to fill * @addr_l: Link-scoped address to fill (for IPv6) * * Return: 0 on success, negative error code on failure @@ -777,6 +777,7 @@ int nl_addr_get(int s, unsigned int ifi, sa_family_t af, nl_foreach_oftype(nh, status, s, buf, seq, RTM_NEWADDR) { struct ifaddrmsg *ifa = (struct ifaddrmsg *)NLMSG_DATA(nh); struct rtattr *rta; + bool link_local; size_t na; if (ifa->ifa_index != ifi || ifa->ifa_flags & IFA_F_DEPRECATED) @@ -788,18 +789,14 @@ int nl_addr_get(int s, unsigned int ifi, sa_family_t af, (af == AF_INET6 && rta->rta_type != IFA_ADDRESS)) continue; - if (af == AF_INET && ifa->ifa_prefixlen > prefix_max) { - memcpy(addr, RTA_DATA(rta), RTA_PAYLOAD(rta)); + link_local = (af == AF_INET6) ? + ifa->ifa_scope >= RT_SCOPE_LINK : + IN4_IS_ADDR_LINKLOCAL(RTA_DATA(rta)); - prefix_max = *prefix_len = ifa->ifa_prefixlen; - } else if (af == AF_INET6 && addr && - ifa->ifa_scope < RT_SCOPE_LINK && - ifa->ifa_prefixlen > prefix_max) { + if (ifa->ifa_prefixlen > prefix_max && !link_local) { memcpy(addr, RTA_DATA(rta), RTA_PAYLOAD(rta)); - - prefix_max = ifa->ifa_prefixlen; + prefix_max = *prefix_len = ifa->ifa_prefixlen; } - if (addr_l && af == AF_INET6 && ifa->ifa_scope == RT_SCOPE_LINK && ifa->ifa_prefixlen > prefix_max_ll) { -- 2.52.0

3 6

[PATCH v2 00/13] vhost-user,udp: Handle multiple iovec entries per virtqueue element
by Laurent Vivier 09 Mar '26

09 Mar '26

Some virtio-net drivers (notably iPXE) provide descriptors where the vnet header and the frame payload are in separate buffers, resulting in two iovec entries per virtqueue element. Currently, the RX (host to guest) path assumes a single iovec per element, which triggers: ASSERTION FAILED in virtqueue_map_desc (virtio.c:403): num_sg < max_num_sg This series reworks the UDP vhost-user receive path to support multiple iovec entries per element, fixing the iPXE crash. This series only addresses the UDP path. TCP vhost-user will be updated to use multi-iov elements in a subsequent series. v2: - add iov_truncate(), iov_memset() - remove iov_tail_truncate() and iov_tail_zero_end() - manage 802.3 minimum frame size Laurent Vivier (13): iov: Add iov_truncate() helper and use it in vu handlers vhost-user: Centralise 802.3 frame padding in vu_collect() and vu_flush() vhost-user: Use ARRAY_SIZE(elem) instead of VIRTQUEUE_MAX_SIZE udp_vu: Use iov_tail to manage virtqueue buffers udp_vu: Move virtqueue management from udp_vu_sock_recv() to its caller iov: Add IOV_PUT_HEADER() to write header data back to iov_tail udp: Pass iov_tail to udp_update_hdr4()/udp_update_hdr6() udp_vu: Use iov_tail in udp_vu_prepare() vu_common: Pass iov_tail to vu_set_vnethdr() vu_common: Accept explicit iovec counts in vu_set_element() vu_common: Accept explicit iovec count per element in vu_init_elem() vu_common: Prepare to use multibuffer with guest RX vhost-user,udp: Use 2 iovec entries per element iov.c | 69 +++++++++++++++++ iov.h | 17 ++++- tcp_vu.c | 73 +++++++++--------- udp.c | 72 +++++++++--------- udp_internal.h | 10 ++- udp_vu.c | 197 +++++++++++++++++++++++++------------------------ vu_common.c | 127 +++++++++++++++++++------------ vu_common.h | 20 +++-- 8 files changed, 356 insertions(+), 229 deletions(-) -- 2.53.0

1 13

[PATCH v2] netlink: Return prefix length for IPv6 addresses in nl_addr_get()
by Jon Maloy 06 Mar '26

06 Mar '26

nl_addr_get() was not setting the prefix_len output parameter for IPv6 addresses, only for IPv4. This meant callers always got 0 for IPv6, forcing them to use a hardcoded default (64). Fix by assigning *prefix_len even in the IPv6 case. We also add another functional change. We now check for if an AF_INET address is link local, in which case we have to skip it. Although it is conventional to set the scope of such addresses to RT_SCOPE_HOST, this is not stated in any RFC, and we cannot trust it to have been set correctly by the user, just as we cannot trust it to have been set correctly for any other AF_INET address. We therefore add this check explicitly on the address itself. Signed-off-by: Jon Maloy <jmaloy(a)redhat.com> --- v2: - Simplified according to feedback from S. Brivio - Added test fo AF_INET link local property --- netlink.c | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/netlink.c b/netlink.c index 82a2f0c..f0e8fa7 100644 --- a/netlink.c +++ b/netlink.c @@ -752,7 +752,7 @@ int nl_addr_set_ll_nodad(int s, unsigned int ifi) * @ifi: Interface index in outer network namespace * @af: Address family * @addr: Global address to fill - * @prefix_len: Mask or prefix length, to fill (for IPv4) + * @prefix_len: Mask or prefix length, to fill * @addr_l: Link-scoped address to fill (for IPv6) * * Return: 0 on success, negative error code on failure @@ -777,6 +777,7 @@ int nl_addr_get(int s, unsigned int ifi, sa_family_t af, nl_foreach_oftype(nh, status, s, buf, seq, RTM_NEWADDR) { struct ifaddrmsg *ifa = (struct ifaddrmsg *)NLMSG_DATA(nh); struct rtattr *rta; + bool link_local; size_t na; if (ifa->ifa_index != ifi || ifa->ifa_flags & IFA_F_DEPRECATED) @@ -788,18 +789,14 @@ int nl_addr_get(int s, unsigned int ifi, sa_family_t af, (af == AF_INET6 && rta->rta_type != IFA_ADDRESS)) continue; - if (af == AF_INET && ifa->ifa_prefixlen > prefix_max) { - memcpy(addr, RTA_DATA(rta), RTA_PAYLOAD(rta)); + link_local = (af == AF_INET6) ? + ifa->ifa_scope >= RT_SCOPE_LINK : + IN4_IS_ADDR_LINKLOCAL(RTA_DATA(rta)); - prefix_max = *prefix_len = ifa->ifa_prefixlen; - } else if (af == AF_INET6 && addr && - ifa->ifa_scope < RT_SCOPE_LINK && - ifa->ifa_prefixlen > prefix_max) { + if (ifa->ifa_prefixlen > prefix_max && !link_local) { memcpy(addr, RTA_DATA(rta), RTA_PAYLOAD(rta)); - - prefix_max = ifa->ifa_prefixlen; + prefix_max = *prefix_len = ifa->ifa_prefixlen; } - if (addr_l && af == AF_INET6 && ifa->ifa_scope == RT_SCOPE_LINK && ifa->ifa_prefixlen > prefix_max_ll) { -- 2.52.0

1 0

Pesto Protocol Proposals, imProved
by David Gibson 06 Mar '26

06 Mar '26

Stefano convinced me that my earlier proposal for the dynamic update protocol was unnecessarily complex. Plus, I saw a much better way of handling socket continuity in the context of a "whole table" replacement. So here's an entirely revised protocol suggestion. # Outline I suggest that each connection to the control socket handles a single transaction. 1. Server hello - Server sends magic number, version - Possibly feature flags / limits (e.g. max number of rules allowed) 2. Client hello - Client sends magic number - Do we need anything else? 3. Server lists pifs - Server sends the number of pifs, their indices and names 4. Server lists rules - Server sends the list of rules, one pif at a time 5. Client gives new rules - Client sends the new list of rules, one pif at a time - Server loads them into the shadow table, and validates (no socket operations) 6. Server acknowledges - Either reports an error and disconnects, or acks waiting for client 7. Client signals apply - Server swaps shadow and active tables, and syncs sockets with new active table 8. Server gives error summary - Server reports bind/listen/whatever errors 9a. Client signals commit - Shadow table (now the old table) discarded or 9b. Client signals rollback - Shadow and active tables swapped back, syncs sockets - Discard shadow table (now the "new" table again) - New bind error report? 10. Server closes control connection # Client disconnects A client disconnect before step (7) is straightforward: discard the shadow table, nothing has changed. A client disconnect between (7) and (9) triggers a rollback, same as (9b). # Error reporting Error reporting at step (6) is fairly straightforward: we can send an error code and/or an error message. Error reporting at (8) is trickier. As a first cut, we could just report "yes" or "no" - taking into account the FWD_WEAK flag. But the client might be able to make better decisions or at least better messages to the user if we report more detailed information. Exactly how detailed is an open question: number of bind failures? number of failures per rule? specific ports which failed? # Interim steps I propose these steps toward implementing this: i. Merge TCP and UDP rule tables. The protocol above assumes a single rule table per-pif, which I think is an easier model to understand and more extensible for future protocol support. ii. Read-only client. Implement steps (1) to (4). Client can query and list the current rules, but not change them. iii. Rule updates. Implement remaining protocol steps, but with a "close and re-open" approach on the server, so unaltered listening sockets might briefly disappear iv. Socket continuity. Have the socket sync "steal" sockets from the old table in preference to re-opening them. If you have any time to work on (ii) while I work on (i), those should be parallelizable. # Concurrent updates Server guarantees that a single transaction as above is atomic in the sense that nothing else is allowed to change the rules between (4) and (9). The easiest way to do that initially is probably to only allow a single client connection at a time. If there's a reason to, we could alter that so that concurrent connections are allowed, but if another client changed anything after step (4), then we give an error on the next op (or maybe just close the control socket from the server side). # Tweaks / variants - I'm not sure that step (2) is necessary - I'm not certain that step (7) is necessary, although I do kind of prefer the client getting a chance to see a "so far, so good" before any socket operations happen. -- David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson

2 3