Re: [PATCH 5/5] fuzz: Add test server for bidirectional protocol fuzzing

On Wed, Aug 12, 2026 at 12:56:28PM +0530, Anshu Kumari wrote:

Add fuzz-server that acts as passt's network peer during fuzzing. It connects to passt's UNIX socket and listens on 127.0.0.1:9999 for TCP connections.

As discussed in my other reply, I'm not really convinced a test server makes sense at all, at least in this form. So, I won't have much to say on the details here.

UNIX socket path: responds to ARP requests and TCP SYNs with stateless replies (swapped addresses, fixed ISN). Responses are XOR'd with AFL++ shared memory data so the fuzzer can mutate server behavior.

What's the reason for having a fixed response that's then mutated, rather than just generating the response directly from AFL's data?

TCP loopback path: accepts connections on port 9999, reads data, XOR-mutates with AFL++ shared memory, and echoes it back. Uses SO_LINGER(0) for immediate RST on close to avoid TIME_WAIT port exhaustion.

Turn-based synchronization via mmap'd flag in /dev/shm coordinates frame exchange between passt and the server.

Also adds: - fuzzing/testcase_dir/empty.bin - fuzzing/README.fuzzing.md

Signed-off-by: Anshu Kumari --- Makefile | 8 +- fuzz-server.c | 490 +++++++++++++++++++++++++++++++++ fuzzing/README.fuzzing.md | 95 +++++++ fuzzing/testcase_dir/empty.bin | Bin 0 -> 12 bytes 4 files changed, 591 insertions(+), 2 deletions(-) create mode 100644 fuzz-server.c create mode 100644 fuzzing/README.fuzzing.md create mode 100644 fuzzing/testcase_dir/empty.bin

diff --git a/Makefile b/Makefile index 8e4121e..4517bc7 100644 --- a/Makefile +++ b/Makefile @@ -125,17 +125,21 @@ valgrind: all

FUZZ_CC ?= afl-clang-fast

-.PHONY: fuzz +.PHONY: fuzz fuzz-server

fuzz: $(MAKE) clean $(MAKE) CC="$(FUZZ_CC)" CPPFLAGS="-DFUZZING -DNDEBUG" CFLAGS="-g -fsanitize=address" passt

Oh, sorry, forgot to mention this on the earlier patch: compiling with -DNDEBUG for fuzzing seems undesirable. Generally if fuzzed input can cause an assert() that's a bug, which we'd like to catch. -NDEBUG will suppress the assert()s, so we won't see it.

+fuzz-server: + $(CC) -D_GNU_SOURCE -O2 -o fuzz-server fuzz-server.c + .PHONY: clean clean: $(RM) $(BIN) *~ *.o seccomp.h seccomp_repair.h seccomp_pesto.h pasta.1 \ passt.tar passt.tar.gz *.deb *.rpm \ - passt.pid README.plain.md + passt.pid README.plain.md \ + fuzz-server

install: $(BIN) $(MANPAGES) docs mkdir -p $(DESTDIR)$(bindir) $(DESTDIR)$(man1dir) diff --git a/fuzz-server.c b/fuzz-server.c new file mode 100644 index 0000000..c9b4101 --- /dev/null +++ b/fuzz-server.c @@ -0,0 +1,490 @@ +// SPDX-License-Identifier: GPL-2.0-or-later + +/* fuzz-server.c - Test server for bidirectional fuzz testing of passt + * + * Connects to passt's UNIX socket as a client. + * Reads outbound frames, generates protocol responses, XORs them + * with AFL++ shared memory data, and writes them back. + * + * Build: make fuzz-server + * Run: ./fuzz-server (after passt.fuzz is listening) + * + * Copyright Red Hat + * Author: Anshu Kumari + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define FUZZ_SOCK_PATH "/tmp/passt_fuzz_1.socket" +#define FUZZ_TURN_PATH "/dev/shm/passt_fuzz_turn" +#define FUZZ_XOR_OFF (12 + 64 * 1024) +#define MAX_FRAME 1600 +#define TCP_LISTEN_PORT 9999 +#define TCP_MAX_EVENTS 16 + +/** + * struct fuzz_turn - Turn-based synchronization flag (mmap'd shared memory) + * @turn: 0 = passt's turn, 1 = server's turn + */ +struct fuzz_turn { + uint32_t turn; +}; + +/** + * swap_eth_ip() - Swap Ethernet MACs and IPv4 addresses + * @pkt: Pointer to start of Ethernet frame + */ +static void swap_eth_ip(uint8_t *pkt) +{ + struct ethhdr *eth = (void *)pkt; + struct iphdr *iph = (void *)(eth + 1); + uint8_t tmp_mac[ETH_ALEN]; + uint8_t tmp_ip[4]; + + memcpy(tmp_mac, eth->h_source, ETH_ALEN); + memcpy(eth->h_source, eth->h_dest, ETH_ALEN); + memcpy(eth->h_dest, tmp_mac, ETH_ALEN); + + memcpy(tmp_ip, &iph->saddr, 4); + memcpy(&iph->saddr, &iph->daddr, 4); + memcpy(&iph->daddr, tmp_ip, 4); +} + +/** + * respond_arp() - Generate ARP reply for an ARP request + * @in: Incoming frame + * @in_len: Incoming frame length + * @out: Output buffer for response + * + * Swaps sender hardware and protocol addresses to form a + * valid ARP reply. Only responds to ARP requests (op=1). + * + * Return: response length, or 0 if not an ARP request + */ +static int respond_arp(const uint8_t *in, uint32_t in_len, uint8_t *out) +{ + /* Min ARP Frame size: eth(14) + ARP Header(8) + Payload(20) */ + if (in_len < 42) + return 0; + + const struct ethhdr *eth = (const void *)in; + const struct arphdr *arp = (const void *)(eth + 1); + + if (eth->h_proto != htons(ETH_P_ARP) || + arp->ar_op != htons(ARPOP_REQUEST)) + return 0; + + memcpy(out, in, in_len); + + struct ethhdr *out_eth = (void *)out; + struct arphdr *out_arp = (void *)(out_eth + 1); + + /* set MAC addresses for output buffer */ + memcpy(out_eth->h_dest, eth->h_source, ETH_ALEN); + memcpy(out_eth->h_source, eth->h_dest, ETH_ALEN); + + /* set ARP operation to REPLY */ + out_arp->ar_op = htons(ARPOP_REPLY); + + /* Swap ARP Sender and Target fields */ + uint8_t *sender = (uint8_t *)(out_arp + 1); + uint8_t *target = sender + 10; + uint8_t tmp[10]; + + memcpy(tmp, sender, 10); + memcpy(sender, target, 10); + memcpy(target, tmp, 10); + + return in_len; +} + +/** + * respond_tcp_syn() - Generate SYN-ACK for a TCP SYN + * @in: Incoming frame + * @in_len: Incoming frame length + * @out: Output buffer for response + * + * Swaps MAC/IP/port addresses and crafts a stateless SYN-ACK with + * a fixed ISN of 1000. Only responds to pure SYN (no ACK set). + * + * Return: response length, or 0 if not a TCP SYN + */ +static int respond_tcp_syn(const uint8_t *in, uint32_t in_len, uint8_t *out) +{ + const struct ethhdr *eth = (const void *)in; + const struct iphdr *iph; + const struct tcphdr *th; + struct tcphdr *out_th; + uint32_t ihl, hdr_len; + + /* Combined minimum length check for Eth + IP + TCP */ + if (in_len < sizeof(*eth) + sizeof(*iph) + sizeof(*th)) + return 0; + + if (eth->h_proto != htons(ETH_P_IP)) + return 0; + + iph = (const void *)(eth + 1); + ihl = iph->ihl * 4; + + if (iph->protocol != IPPROTO_TCP || + in_len < sizeof(*eth) + ihl + sizeof(*th)) + return 0; + + th = (const void *)((const uint8_t *)iph + ihl); + + /* Only respond to pure SYN packets (no ACK set) */ + if (!th->syn || th->ack) + return 0; + + hdr_len = sizeof(*eth) + ihl + sizeof(*th); + memcpy(out, in, hdr_len); + swap_eth_ip(out); + + /* Craft SYN-ACK response header */ + out_th = (struct tcphdr *)(out + sizeof(*eth) + ihl); + out_th->source = th->dest; + out_th->dest = th->source; + out_th->seq = htonl(1000); + out_th->ack_seq = htonl(ntohl(th->seq) + 1); + out_th->syn = 1; + out_th->ack = 1; + out_th->doff = 5; + out_th->check = 0; + + return hdr_len; +} + +/** + * generate_response() - Try all protocol responders on a frame + * @in: Incoming frame + * @in_len: Incoming frame length + * @out: Output buffer for response + * + * Tries ARP then TCP SYN responders in order. Returns the first + * successful response. + * + * Return: response length, or 0 if no responder matched + */ +static int generate_response(const uint8_t *in, uint32_t in_len, uint8_t *out) +{ + int len; + + if (in_len < sizeof(struct ethhdr)) + return 0; + + if ((len = respond_arp(in, in_len, out)) > 0) + return len; + if ((len = respond_tcp_syn(in, in_len, out)) > 0) + return len; + + return 0; +} + +/** + * map_afl_shm() - Map AFL++ shared memory for XOR mutation + * + * Reads __AFL_SHM_FUZZ_ID from the AFL++ environment + * variable and attaches the corresponding shared memory + * segment read-only. + * + * Return: pointer to mapped SHM, or NULL if not available + */ +static uint8_t *map_afl_shm(void) +{ + const char *id_str = getenv("__AFL_SHM_FUZZ_ID"); + int shm_id; + uint8_t *map; + + if (!id_str) + return NULL; + + shm_id = atoi(id_str); + map = shmat(shm_id, NULL, SHM_RDONLY); + if (map == (void *)-1) + return NULL; + + return map; +} + +/** + * apply_xor_mask() - XOR buffer with AFL++ shared memory data + * @buf: Buffer to mutate in place + * @len: Buffer length + * @afl_buf: AFL++ shared memory + * + * XORs each byte of @buf with the corresponding byte from the + * AFL++ buffer starting at offset FUZZ_XOR_OFF (65548). This + * lets AFL++ mutate server responses. + */ +static void apply_xor_mask(uint8_t *buf, int len, const uint8_t *afl_buf) +{ + int i; + + if (!afl_buf) + return; + + for (i = 0; i < len; i++) + buf[i] ^= afl_buf[FUZZ_XOR_OFF + i]; +} + +/* ---- TCP Host Listener (Loopback) ---- */ + +static int tcp_listen_fd = -1; +static int tcp_epfd = -1; + +/** + * tcp_listener_init() - Set up TCP listener on loopback port 9999 + * + * Creates an epoll instance and a non-blocking TCP listener on + * 127.0.0.1:9999 with SO_REUSEADDR. + * + * Return: 0 on success, -1 on failure + */ +static int tcp_listener_init(void) +{ + struct sockaddr_in sa = { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + .sin_port = htons(TCP_LISTEN_PORT), + }; + struct epoll_event ev; + int opt = 1; + + tcp_epfd = epoll_create1(EPOLL_CLOEXEC); + tcp_listen_fd = socket(AF_INET, SOCK_STREAM | SOCK_NONBLOCK, 0); + if (tcp_epfd < 0 || tcp_listen_fd < 0) + return -1; + + setsockopt(tcp_listen_fd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt)); + + if (bind(tcp_listen_fd, (struct sockaddr *)&sa, sizeof(sa)) < 0 || + listen(tcp_listen_fd, 128) < 0) + return -1; + + ev.events = EPOLLIN; + ev.data.fd = tcp_listen_fd; + epoll_ctl(tcp_epfd, EPOLL_CTL_ADD, tcp_listen_fd, &ev); + + return 0; +} + +/** + * tcp_handle_accept() - Drain all pending TCP connections + * + * Sets SO_LINGER with l_linger=0 on every accepted connection, + * which does the immediate RST on connection close instead of + * waiting for connection to close on it's own. + */ +static void tcp_handle_accept(void) +{ + struct linger lg = { .l_onoff = 1, .l_linger = 0 }; + struct epoll_event ev; + int fd; + + /* Drain ALL pending connections in the backlog */ + while ((fd = accept4(tcp_listen_fd, NULL, NULL, + SOCK_NONBLOCK)) >= 0) { + setsockopt(fd, SOL_SOCKET, SO_LINGER, &lg, sizeof(lg)); + + ev.events = EPOLLIN | EPOLLRDHUP | EPOLLHUP | EPOLLERR; + ev.data.fd = fd; + epoll_ctl(tcp_epfd, EPOLL_CTL_ADD, fd, &ev); + } +} + +/** + * tcp_handle_data() - Read TCP data, XOR-mutate, echo back + * @fd: Connected TCP socket + * @afl_buf: AFL++ shared memory for XOR mask + */ +static void tcp_handle_data(int fd, const uint8_t *afl_buf) +{ + uint8_t buf[MAX_FRAME]; + ssize_t n, written, ret; + + n = read(fd, buf, sizeof(buf)); + if (n <= 0) { + epoll_ctl(tcp_epfd, EPOLL_CTL_DEL, fd, NULL); + close(fd); + return; + } + + apply_xor_mask(buf, (uint32_t)n, afl_buf); + + /* Ensure all n bytes are written, handling short writes */ + written = 0; + while (written < n) { + ret = write(fd, buf + written, (size_t)(n - written)); + if (ret <= 0) { + if (ret < 0 && (errno == EAGAIN || + errno == EWOULDBLOCK)) + continue; + + epoll_ctl(tcp_epfd, EPOLL_CTL_DEL, fd, NULL); + close(fd); + return; + } + written += ret; + } +} + +/** + * tcp_process_events() - Non-blocking poll for TCP events + * @afl_buf: AFL++ shared memory + */ +static void tcp_process_events(const uint8_t *afl_buf) +{ + struct epoll_event events[TCP_MAX_EVENTS]; + int nfds, i; + + nfds = epoll_wait(tcp_epfd, events, TCP_MAX_EVENTS, 0); + + for (i = 0; i < nfds; i++) { + int fd = events[i].data.fd; + + if (fd == tcp_listen_fd) { + tcp_handle_accept(); + } else if (events[i].events & (EPOLLIN | EPOLLRDHUP)) { + tcp_handle_data(fd, afl_buf); + } else if (events[i].events & (EPOLLHUP | EPOLLERR)) { + epoll_ctl(tcp_epfd, EPOLL_CTL_DEL, fd, NULL); + close(fd); + } + } +} + +/** + * create_turn_flag() - Create and mmap the turn synchronization flag + * + * Creates FUZZ_TURN_PATH in /dev/shm and maps it + * as shared memory. Both passt and fuzz-server map this file to + * coordinate turn-based frame exchange via atomic load/store. + * + * Return: pointer to mapped turn struct, or NULL on failure + */ +static struct fuzz_turn *create_turn_flag(void) +{ + struct fuzz_turn *t; + int fd; + + fd = open(FUZZ_TURN_PATH, O_RDWR | O_CREAT | O_TRUNC, 0644); + if (fd < 0) + return NULL; + + if (ftruncate(fd, sizeof(struct fuzz_turn)) < 0) { + close(fd); + return NULL; + } + + t = mmap(NULL, sizeof(*t), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); + close(fd); + + if (t == MAP_FAILED) + return NULL; + + t->turn = 0; + return t; +} + +/** + * main() - Server entry point + * + * Outer loop reconnects to passt's UNIX socket on each AFL++ fork + * server restart. Inner loop handles turn-based TAP frame exchange + * and TCP events concurrently. + * + * Return: 0 on success, 1 on initialization failure + */ +int main(void) +{ + struct sockaddr_un addr = { .sun_family = AF_UNIX }; + uint8_t frame[MAX_FRAME], response[MAX_FRAME]; + struct fuzz_turn *turn; + uint8_t *afl_buf; + + strncpy(addr.sun_path, FUZZ_SOCK_PATH, sizeof(addr.sun_path) - 1); + + turn = create_turn_flag(); + if (!turn || tcp_listener_init() < 0) { + (void)fprintf(stderr, + "fuzz-server: tcp initialization failed\n"); + return 1; + } + + afl_buf = map_afl_shm(); + (void)fprintf(stderr, + "fuzz-server: ready, TCP listening on 127.0.0.1:%d\n", + TCP_LISTEN_PORT); + + /* + * Reconnect UNIX socket on passt restart — AFL++'s fork server + * restarts passt on each iteration. + */ + while (1) { + ssize_t n; + int sock; + + sock = socket(AF_UNIX, SOCK_SEQPACKET, 0); + if (sock < 0) + return 1; + + while (connect(sock, (struct sockaddr *)&addr, + sizeof(addr)) < 0) + usleep(10000); + + while (1) { + tcp_process_events(afl_buf); + + if (__atomic_load_n(&turn->turn, + __ATOMIC_ACQUIRE) == 1) { + n = recv(sock, frame, MAX_FRAME, + MSG_DONTWAIT); + if (n > 0) { + int resp_len; + + resp_len = generate_response( + frame, n, response); + if (resp_len > 0) { + apply_xor_mask(response, + resp_len, + afl_buf); + send(sock, response, + resp_len, MSG_NOSIGNAL); + } + } else if (n == 0 || + (n < 0 && errno != EAGAIN && + errno != EWOULDBLOCK)) { + __atomic_store_n(&turn->turn, 0, + __ATOMIC_RELEASE); + break; + } + + __atomic_store_n(&turn->turn, 0, + __ATOMIC_RELEASE); + } + } + + close(sock); + } + + return 0; +} diff --git a/fuzzing/README.fuzzing.md b/fuzzing/README.fuzzing.md new file mode 100644 index 0000000..f23e265 --- /dev/null +++ b/fuzzing/README.fuzzing.md @@ -0,0 +1,95 @@ +## Fuzzing passt with AFL++ + +### Prerequisites + +- AFL++ (afl-clang-fast, afl-fuzz) + +### Build + +``` +make fuzz +cp passt passt.fuzz +make fuzz-server +``` + +This produces: +- `passt.fuzz` -- instrumented with AFL++ and AddressSanitizer +- `fuzz-server` -- test server for bidirectional protocol fuzzing + +To use a specific AFL++ installation: + +``` +make FUZZ_CC=/path/to/afl-clang-fast fuzz +``` + +### Run + +Start the test server first, then the fuzzer: + +``` +# Terminal 1 -- test server: +./fuzz-server + +# Terminal 2 -- fuzzer: +afl-fuzz -i fuzzing/testcase_dir -o fuzzing/sync_dir \ + -- ./passt.fuzz --foreground +``` + +Multi-core (secondary instances share the corpus): + +``` +# Terminal 1 -- main instance: +afl-fuzz -M main -i fuzzing/testcase_dir -o fuzzing/sync_dir \ + -- ./passt.fuzz --foreground + +# Terminal 2 -- secondary with different power schedule: +afl-fuzz -S variant1 -p rare -i fuzzing/testcase_dir \ + -o fuzzing/sync_dir -- ./passt.fuzz --foreground +``` + +### Architecture + +The fuzzer uses AFL++ persistent mode with shared memory fuzzing. +Each iteration: + +1. Resets deterministic state (clock, flow table, epoll) +2. Reads an epoll event and packet data from AFL++ shared memory +3. For TAP events: injects a packet with fixed L2/L3/L4 headers + into the tap pipeline via tap_add_packet() + tap_handler(). + The TCP destination port is fixed to 9999 (the test server's + listening port). +4. Exchanges a turn flag with fuzz-server for bidirectional flow. +5. Calls passt_worker() to process the epoll event +6. Polls for host-side TCP events (connect completion, server data) + +The test server connects to passt's UNIX socket (SOCK_SEQPACKET) +and listens on 127.0.0.1:9999 for TCP connections. It responds to +ARP requests and TCP SYNs on the UNIX socket, and echoes TCP data +(XOR'd with AFL++ shared memory) on the loopback side. + +Deterministic wrappers in fuzz.c replace clock_gettime(), +getrandom(), getsockopt(TCP_INFO), and recv/recvmsg/recvfrom/ +recvmmsg to eliminate non-determinism from kernel state. The recv +wrappers return AFL++ buffer data for non-TAP file descriptors, +giving the fuzzer control over what passt "receives" from host +sockets. + +### Seed inputs + +`testcase_dir/empty.bin` is a 12-byte zero file +(sizeof(struct epoll_event)). AFL++ discovers packet formats +through mutation from this minimal seed. + +### Reproducing crashes + +``` +./passt.fuzz --foreground < \ + fuzzing/sync_dir/default/crashes/id:000000,... +``` + +Minimize a crash input: + +``` +afl-tmin -i fuzzing/sync_dir/default/crashes/id:000000,... \ + -o crash_minimized -- ./passt.fuzz --foreground +``` diff --git a/fuzzing/testcase_dir/empty.bin b/fuzzing/testcase_dir/empty.bin new file mode 100644 index 0000000000000000000000000000000000000000..ce58bc9f84b9623e708de4eb8427a57d9f9a160f GIT binary patch literal 12 KcmZQzKmY&$3;+QD

literal 0 HcmV?d00001

-- 2.55.0

-- David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson