[PATCH v4 00/13] Rework option parsing in preparation for destination remapping
This was... a bit of a nightmare. It took way too long and went down a bunch of blind alleys. I'm not sure how much I like the final result: it's pleasingly terse in some cases, but in others it feels dangerously subtle, requiring a pretty careful understanding of the sequencing rules of C's &&, || and , operators. That said, while I was at many points ready to pack it in and hack my around the problems limiting the parsing for destination remapping, I couldn't really think of feasible way to do that either. So, here we are. v4: * Update manpages and --help to reflect changes in 12/13 * No other changes v3: * Fixed validation of ifnames in 13/13. * No other changes. v2: * Numerous minor tweaks based on Stefano's feedback. * The "minor issues" aren't actually minor - they're hard to solve in C, remain. David Gibson (13): Makefile: Add missing PESTO_HEADERS variable conf: Use parameter instead of global in conf_nat() parse: Start splitting out parsing helpers conf: Remove duplicate parsing of -F option conf: Clean up conf_ip4_prefix() parse: Add helper to parse unsigned integer values parse: Move parse_port_range() to new parsing framework parse: Add helpers for parsing IP addresses conf: Move address configuration into helper function conf: Remove unnecessary mode checks from conf_addr() conf: Use new parsing tools to handle -a option fwd_rule: Allow "all" port specs to be combined with other options fwd_rule: Rewrite forward rule parsing using parse.c helpers Makefile | 22 +-- common.h | 3 + conf.c | 220 ++++++++++++++++------------- conf.h | 1 + fwd_rule.c | 400 ++++++++++++++++++++++++----------------------------- fwd_rule.h | 2 - inany.c | 70 +--------- inany.h | 3 - parse.c | 244 ++++++++++++++++++++++++++++++++ parse.h | 37 +++++ passt.1 | 33 +++-- util.c | 12 +- 12 files changed, 621 insertions(+), 426 deletions(-) create mode 100644 parse.c create mode 100644 parse.h -- 2.55.0
As we add more complexity to what forwarding rules are allowed, our
existing ad-hoc C parsing is starting to become quite awkward. We already
have some parts that resemble a very simple recursive[0] descent parser,
with composable subfunctions that consume as much input as they need,
rather than pre-splitting based on known delimiters.
Start extending this approach, by creating parse.[ch] with parsing helpers
with a uniform interface. Initially we start with very simple cases: the
parse_keyword() function from fwd_rule.c and another helper to check that
we've reached the end of input.
Rename parse_keyword() to parse_literal(), because it's not just useful
for "keywords" as such. We can use it in a bunch of additional places for
parsing delimiters and other symbols. This doesn't gain us a lot now but
will make things clearer as we use more such parser helpers.
[0] Except that the grammars we have aren't actually recursive, so neither
is the code.
Signed-off-by: David Gibson
parse_ipv[46]() are wrappers around inet_pton() that are more
convenient for use when the IP is part of a longer string, rather than
the entire string. parse_inany() replaces inany_pton() which again
will become more convenient for strings including IPs that aren't just
an IP.
For now we only have some simple and sometimes awkward use cases,
we'll replace these with more natural uses in future.
Cc: Jon Maloy
The -F option is parsed in conf() along with everything else. However,
because it informs what fds we can close at startup, we also have a special
case parse of it in close_open_files().
At present we duplicate the parsing and validation code, which is a bit
risky. Avoid that with a conf_tap_fd() helper.
Signed-off-by: David Gibson
The -a command line option can take either an address prefix, or a bare
address. Current parsing of this is pretty awkward, using the special
purpose helper inany_prefix_pton(). With the new incremental parsing
helpers this can be done more naturally. Rework it to use them.
This does requiring extending parse_inany() to parse_inany_() which also
reports the format of the address as parse, as opposed to the family of
the resulting address. This is so that ::ffff:192.0.1.1/112 will be
correctly interpreted the same as 192.0.1.1/16, rather than the
nonsensical 192.0.0.1/112.
Cc: Jon Maloy
conf_addr() sets c->ip[46]no_copy_addrs conditional upon being in pasta
mode. That sort of makes sense, since address copying is only a thing for
pasta mode. However, setting the variables anyway is harmless, and
arguably more logically consistent. If we had a way of copying addresses
for passt mode (or some future mode), it would still be incorrect to do so
in these circumstances.
So, make the assignments unconditional.
Signed-off-by: David Gibson
Restructure conf_ip4_prefix() so that success is the early exit path and
failure is the "default" path. Also move the error handling (die()) into
it from the caller.
This saves a handful of lines for now, and will make integration with
upcoming parsing changes nicer.
Signed-off-by: David Gibson
Currently we handle -t all and the like as a special case, it can't be
combined with other port specifier options. Remove that restriction,
allowing combined options like:
-t all,~9999 # Forward everything non-ephemeral except 9999
-t all,auto # Equivalent to -t auto
-t all,33000 # Forward non-ephemeral plus port 33,000
This isn't particularly useful immediately, but will become important for
destination address specification - it provides a place to attach the
target address for "all" or exclude only mappings. It will also work
better with some parsing reworks we want to make.
Signed-off-by: David Gibson
The handling of the -a option is getting complex enough that it's pretty
bulky inside its switch label. Move it into a new conf_addr() function.
We also rename the bulky addr_has_prefix_len and prefix_len_from_opt
variables to the terser 'opt_a_is_prefix' and 'opt_n', since they are
specifically about those command line options.
Cc: Jon Maloy
parse_port_range() in fwd_rule.c takes an approach similar to that used
by the new parsing helpers in parse.c (and is partially inspiration for
it), but isn't quite the same. Move it into parse.c, and bring it into
line with that file's conventions.
Signed-off-by: David Gibson
Most places we need to parse an integer encoded as a string, we use
strtol() or strtoul(). These already work a bit like descent parser
helpers, in that they consume as much as they can, but don't require the
number parsed to be the whole of the einput string. Make a wrapper to
parse unsigned integers as part of our parsing helper. This wrapper
handles the mildly fiddly error checking requirements for strtoul().
We replace a number, though not all, of our existing strtoul() uses with
the new parse_unsigned(). We also replace a number of strtol() use cases,
because, despite using that instead of strtoul() they are only used for
non-negative values. In some cases this makes the logic a little more
straightforward. In some other cases it will catch some error cases we
previously could have missed.
Signed-off-by: David Gibson
Forwarding rules for the -[tTuU] options are parsed in fwd_rule_parse()
and fwd_rule_parse_ports(). Currently those use a mix of loosely
recursive descent parsing helpers, consuming input from left to right,
and ad-hoc C parsing - searching for delimiters with strchr() or the like
and breaking down on that basis.
As well as being a slightly awkward mix, the current approach will not work
for adding target addresses to the rules: we can't easily split the
listening from target parts first, because the ':' delimiter could appear
multiple times for multiple port ranges, or in IPv6 addresses. However,
without splitting the target part first, we can't first split off the
listening address and interface because the '/' delimiter could appear in
the target portion, but not the listening portion, e.g.:
-t 12345:192.0.1.1/12345
To address this, rewrite the entirety of the parsing so we consume the
input left to right in a more-or-less recursive descent manner. This means
we no longer rely on certain delimiters having a single meaning over the
whole input, just the next part of it.
Because of the semantics of port specs, we need to make several passes
over the list of comma separated chunks. Previously, some of the logic was
duplicated between the two passes, making it fragile. Now, we introduce
a parse_port_chunk() helper which we re-use in each pass to make it clearer
we parse exactly the same way on each pass.
Signed-off-by: David Gibson
On Thu, 2 Jul 2026 16:31:42 +1000
David Gibson
Currently we handle -t all and the like as a special case, it can't be combined with other port specifier options. Remove that restriction, allowing combined options like: -t all,~9999 # Forward everything non-ephemeral except 9999 -t all,auto # Equivalent to -t auto -t all,33000 # Forward non-ephemeral plus port 33,000
This isn't particularly useful immediately, but will become important for destination address specification - it provides a place to attach the target address for "all" or exclude only mappings. It will also work better with some parsing reworks we want to make.
Signed-off-by: David Gibson
--- conf.c | 11 +++++------ fwd_rule.c | 39 ++++++++++++++++++++------------------- passt.1 | 33 ++++++++++++++++----------------- 3 files changed, 41 insertions(+), 42 deletions(-) diff --git a/conf.c b/conf.c index c4a36dee..a610c0c6 100644 --- a/conf.c +++ b/conf.c @@ -660,11 +660,9 @@ static void usage(const char *name, FILE *f, int status) " SPEC can be:\n" " 'none': don't forward any ports\n" " [ADDR[%%IFACE]/]PORTS: forward specific ports\n" - " PORTS is either 'all' (forward all unbound, non-ephemeral\n" - " ports), or a comma-separated list of ports, optionally\n" - " ranged with '-' and optional target ports after ':'.\n" - " Ranges can be reduced by excluding ports or ranges\n" - " prefixed by '~'.\n" + " PORTS is comma-separated list of ports, either\n"
I didn't really consider this change as worth updating usage and man page (the previous version wouldn't be entirely accurate anymore but practically speaking rather clear, I thought). If it is: - PORTS is _a_ comma-separated ... - I think we should maintain the description for 'all' (forward all unbound, non-ephemeral ports), because otherwise just "Forward all ports" below becomes particularly misleading
+ " 'all', a port number or range. Ranges can be reduced\n" + " by excluding ports or ranges prefixed by '~'.\n" "%s" " Examples:\n" " -t all Forward all ports\n" @@ -677,7 +675,8 @@ static void usage(const char *name, FILE *f, int status) " corresponding port numbers plus 10\n" " -t 192.0.2.1/5 Bind port 5 of 192.0.2.1 to %s\n" " -t 5-25,~10-20 Forward ports 5 to 9, and 21 to 25\n" - " -t ~25 Forward all ports except for 25\n" + " -t ~25,all\n" + " -t 25 Forward all ports except for 25\n"
I think the previous version makes more sense. This isn't an exhaustive description, it just shows how to quickly do things. This is missing a ~ by the way.
"%s" " default: %s\n" " -u, --udp-ports SPEC UDP port forwarding to %s\n" diff --git a/fwd_rule.c b/fwd_rule.c index 6d7ec2c5..b14df340 100644 --- a/fwd_rule.c +++ b/fwd_rule.c @@ -471,20 +471,13 @@ static void fwd_rule_parse_ports(struct fwd_table *fwd, bool del, uint8_t proto, uint8_t flags = 0; unsigned i;
- if (!strcmp(spec, "all")) { - /* Treat "all" as equivalent to "": all non-ephemeral ports */ - spec = ""; - } - /* Parse excluded ranges and "auto" in the first pass */ for_each_chunk(p, ep, spec, ",") { struct port_range xrange;
- if (isdigit(*p)) { - /* Include range, parse later */ - exclude_only = false; + /* Include range, parse later */ + if (parse_literal(&p, "all") || isdigit(*p)) continue; - }
if (parse_literal(&p, "auto")) { if (p != ep) /* Garbage after the keyword */ @@ -512,20 +505,18 @@ static void fwd_rule_parse_ports(struct fwd_table *fwd, bool del, uint8_t proto, bitmap_set(exclude, i); }
- if (exclude_only) { - /* Exclude ephemeral ports */ - fwd_port_map_ephemeral(exclude); - - fwd_rule_range_except(fwd, del, proto, addr, ifname, - 1, NUM_PORTS - 1, exclude, - 1, flags | FWD_WEAK); - return; - } - /* Now process base ranges, skipping exclusions */ for_each_chunk(p, ep, spec, ",") { struct port_range orig_range, mapped_range;
+ /* Handle "all" like exclude only */ + if (parse_literal(&p, "all")) { + if (p != ep) /* Garbage after the keyword */ + goto bad; + + continue; + } + if (!isdigit(*p)) /* Already parsed */ continue; @@ -533,6 +524,8 @@ static void fwd_rule_parse_ports(struct fwd_table *fwd, bool del, uint8_t proto, if (!parse_port_range(&p, &orig_range)) goto bad;
+ exclude_only = false; + if (parse_literal(&p, ":")) { /* There's a range to map to as well */ if (!parse_port_range(&p, &mapped_range)) @@ -553,6 +546,14 @@ static void fwd_rule_parse_ports(struct fwd_table *fwd, bool del, uint8_t proto, mapped_range.first, flags); }
+ /* Finally handle "all" and exclude only specs */ + if (exclude_only) { + fwd_port_map_ephemeral(exclude); + + fwd_rule_range_except(fwd, del, proto, addr, ifname, + 1, NUM_PORTS - 1, exclude, + 1, flags | FWD_WEAK); + } return; bad: die("Invalid port specifier '%s'", spec); diff --git a/passt.1 b/passt.1 index 908fd4a4..c3722ef9 100644 --- a/passt.1 +++ b/passt.1 @@ -432,29 +432,22 @@ Send \fIname\fR as Client FQDN: DHCP option 81 and DHCPv6 option 39.
.TP .BR \-t ", " \-\-tcp-ports " " \fIspec -Configure TCP port forwarding to guest or namespace. \fIspec\fR can be one of: +Configure TCP port forwarding to guest or namespace. \fIspec\fR can be either: .RS
.TP .BR none Don't forward any ports
+or .TP [\fIaddress\fR[\fB%\fR\fIinterface\fR]\fB/\fR]\fIports\fR ... -Specific ports to forward. Optionally, a specific listening address -and interface name (since Linux 5.7) can be specified. \fIports\fR -may be either: -.RS -.TP -\fBall\fR -Forward all unbound, non-ephemeral ports, as permitted by current -capabilities. For low (< 1024) ports, see \fBNOTES\fR. No failures -are reported for unavailable ports, unless no ports could be forwarded -at all. + +Ports to forward. Optionally, a specific listening address and +interface name (since Linux 5.7) can be specified. .RE
-.RS -or a comma-separated list of entries which may be any of: +\fIports\fR is a comma-separated list of entries which may be any of: .TP \fIfirst\fR[\fB-\fR\fIlast\fR][\fB:\fR\fItofirst\fR[\fB-\fR\fItolast\fR]] Include range. Forward port numbers between \fIfirst\fR and \fIlast\fR @@ -468,6 +461,13 @@ as \fIfirst\fR. Exclude range. Don't forward port numbers between \fIfirst\fR and \fIlast\fR. This takes precedences over include ranges.
+.TP +.BR all +Forward all unbound, non-ephemeral ports, not covered by exclude +ranges above, as permitted by current capabilities. For low (< 1024) +ports, see \fBNOTES\fR. No failures are reported for unavailable +ports, unless no ports could be forwarded at all. + .TP .BR auto \fBpasta\fR only. Only forward ports in the specified set if the @@ -477,10 +477,9 @@ periodically derived (every second) from listening sockets reported by .RE
Specifying excluded ranges only implies that all other non-ephemeral -ports are forwarded. Specifying no ranges at all implies forwarding -all non-ephemeral ports permitted by current capabilities. In this -case, no failures are reported for unavailable ports, unless no ports -could be forwarded at all. +ports are forwarded. Specifying no ranges is equivalent +to '\fBall\fR'. In this case, no failures are reported for +unavailable ports, unless no ports could be forwarded at all.
Nit: this could use a few more columns (I think it's slightly more readable as source), say: ports are forwarded. Specifying no ranges is equivalent to '\fBall\fR'. In this case, no failures are reported for unavailable ports, unless no ports could be forwarded at all. -- Stefano
On Thu, Jul 02, 2026 at 09:14:17AM +0200, Stefano Brivio wrote:
On Thu, 2 Jul 2026 16:31:42 +1000 David Gibson
wrote: Currently we handle -t all and the like as a special case, it can't be combined with other port specifier options. Remove that restriction, allowing combined options like: -t all,~9999 # Forward everything non-ephemeral except 9999 -t all,auto # Equivalent to -t auto -t all,33000 # Forward non-ephemeral plus port 33,000
This isn't particularly useful immediately, but will become important for destination address specification - it provides a place to attach the target address for "all" or exclude only mappings. It will also work better with some parsing reworks we want to make.
Signed-off-by: David Gibson
--- conf.c | 11 +++++------ fwd_rule.c | 39 ++++++++++++++++++++------------------- passt.1 | 33 ++++++++++++++++----------------- 3 files changed, 41 insertions(+), 42 deletions(-) diff --git a/conf.c b/conf.c index c4a36dee..a610c0c6 100644 --- a/conf.c +++ b/conf.c @@ -660,11 +660,9 @@ static void usage(const char *name, FILE *f, int status) " SPEC can be:\n" " 'none': don't forward any ports\n" " [ADDR[%%IFACE]/]PORTS: forward specific ports\n" - " PORTS is either 'all' (forward all unbound, non-ephemeral\n" - " ports), or a comma-separated list of ports, optionally\n" - " ranged with '-' and optional target ports after ':'.\n" - " Ranges can be reduced by excluding ports or ranges\n" - " prefixed by '~'.\n" + " PORTS is comma-separated list of ports, either\n"
I didn't really consider this change as worth updating usage and man page (the previous version wouldn't be entirely accurate anymore but practically speaking rather clear, I thought).
I think we should update the manpage, since it is pretty detailed (and now wrong). usage() I'll grant is borderline at best.
If it is:
- PORTS is _a_ comma-separated ...
Fixed.
- I think we should maintain the description for 'all' (forward all unbound, non-ephemeral ports), because otherwise just "Forward all ports" below becomes particularly misleading
Good point. I've reworded again, I think it's better.
+ " 'all', a port number or range. Ranges can be reduced\n" + " by excluding ports or ranges prefixed by '~'.\n" "%s" " Examples:\n" " -t all Forward all ports\n" @@ -677,7 +675,8 @@ static void usage(const char *name, FILE *f, int status) " corresponding port numbers plus 10\n" " -t 192.0.2.1/5 Bind port 5 of 192.0.2.1 to %s\n" " -t 5-25,~10-20 Forward ports 5 to 9, and 21 to 25\n" - " -t ~25 Forward all ports except for 25\n" + " -t ~25,all\n" + " -t 25 Forward all ports except for 25\n"
I think the previous version makes more sense. This isn't an exhaustive description, it just shows how to quickly do things. This is missing a ~ by the way.
Oops, yes. I dropped these lines.
"%s" " default: %s\n" " -u, --udp-ports SPEC UDP port forwarding to %s\n" diff --git a/fwd_rule.c b/fwd_rule.c index 6d7ec2c5..b14df340 100644 --- a/fwd_rule.c +++ b/fwd_rule.c @@ -471,20 +471,13 @@ static void fwd_rule_parse_ports(struct fwd_table *fwd, bool del, uint8_t proto, uint8_t flags = 0; unsigned i;
- if (!strcmp(spec, "all")) { - /* Treat "all" as equivalent to "": all non-ephemeral ports */ - spec = ""; - } - /* Parse excluded ranges and "auto" in the first pass */ for_each_chunk(p, ep, spec, ",") { struct port_range xrange;
- if (isdigit(*p)) { - /* Include range, parse later */ - exclude_only = false; + /* Include range, parse later */ + if (parse_literal(&p, "all") || isdigit(*p)) continue; - }
if (parse_literal(&p, "auto")) { if (p != ep) /* Garbage after the keyword */ @@ -512,20 +505,18 @@ static void fwd_rule_parse_ports(struct fwd_table *fwd, bool del, uint8_t proto, bitmap_set(exclude, i); }
- if (exclude_only) { - /* Exclude ephemeral ports */ - fwd_port_map_ephemeral(exclude); - - fwd_rule_range_except(fwd, del, proto, addr, ifname, - 1, NUM_PORTS - 1, exclude, - 1, flags | FWD_WEAK); - return; - } - /* Now process base ranges, skipping exclusions */ for_each_chunk(p, ep, spec, ",") { struct port_range orig_range, mapped_range;
+ /* Handle "all" like exclude only */ + if (parse_literal(&p, "all")) { + if (p != ep) /* Garbage after the keyword */ + goto bad; + + continue; + } + if (!isdigit(*p)) /* Already parsed */ continue; @@ -533,6 +524,8 @@ static void fwd_rule_parse_ports(struct fwd_table *fwd, bool del, uint8_t proto, if (!parse_port_range(&p, &orig_range)) goto bad;
+ exclude_only = false; + if (parse_literal(&p, ":")) { /* There's a range to map to as well */ if (!parse_port_range(&p, &mapped_range)) @@ -553,6 +546,14 @@ static void fwd_rule_parse_ports(struct fwd_table *fwd, bool del, uint8_t proto, mapped_range.first, flags); }
+ /* Finally handle "all" and exclude only specs */ + if (exclude_only) { + fwd_port_map_ephemeral(exclude); + + fwd_rule_range_except(fwd, del, proto, addr, ifname, + 1, NUM_PORTS - 1, exclude, + 1, flags | FWD_WEAK); + } return; bad: die("Invalid port specifier '%s'", spec); diff --git a/passt.1 b/passt.1 index 908fd4a4..c3722ef9 100644 --- a/passt.1 +++ b/passt.1 @@ -432,29 +432,22 @@ Send \fIname\fR as Client FQDN: DHCP option 81 and DHCPv6 option 39.
.TP .BR \-t ", " \-\-tcp-ports " " \fIspec -Configure TCP port forwarding to guest or namespace. \fIspec\fR can be one of: +Configure TCP port forwarding to guest or namespace. \fIspec\fR can be either: .RS
.TP .BR none Don't forward any ports
+or .TP [\fIaddress\fR[\fB%\fR\fIinterface\fR]\fB/\fR]\fIports\fR ... -Specific ports to forward. Optionally, a specific listening address -and interface name (since Linux 5.7) can be specified. \fIports\fR -may be either: -.RS -.TP -\fBall\fR -Forward all unbound, non-ephemeral ports, as permitted by current -capabilities. For low (< 1024) ports, see \fBNOTES\fR. No failures -are reported for unavailable ports, unless no ports could be forwarded -at all. + +Ports to forward. Optionally, a specific listening address and +interface name (since Linux 5.7) can be specified. .RE
-.RS -or a comma-separated list of entries which may be any of: +\fIports\fR is a comma-separated list of entries which may be any of: .TP \fIfirst\fR[\fB-\fR\fIlast\fR][\fB:\fR\fItofirst\fR[\fB-\fR\fItolast\fR]] Include range. Forward port numbers between \fIfirst\fR and \fIlast\fR @@ -468,6 +461,13 @@ as \fIfirst\fR. Exclude range. Don't forward port numbers between \fIfirst\fR and \fIlast\fR. This takes precedences over include ranges.
+.TP +.BR all +Forward all unbound, non-ephemeral ports, not covered by exclude +ranges above, as permitted by current capabilities. For low (< 1024) +ports, see \fBNOTES\fR. No failures are reported for unavailable +ports, unless no ports could be forwarded at all. + .TP .BR auto \fBpasta\fR only. Only forward ports in the specified set if the @@ -477,10 +477,9 @@ periodically derived (every second) from listening sockets reported by .RE
Specifying excluded ranges only implies that all other non-ephemeral -ports are forwarded. Specifying no ranges at all implies forwarding -all non-ephemeral ports permitted by current capabilities. In this -case, no failures are reported for unavailable ports, unless no ports -could be forwarded at all. +ports are forwarded. Specifying no ranges is equivalent +to '\fBall\fR'. In this case, no failures are reported for +unavailable ports, unless no ports could be forwarded at all.
Nit: this could use a few more columns (I think it's slightly more readable as source), say:
Huh, odd. For some reason emacs M-q was making it narrower. Fixed manually.
ports are forwarded. Specifying no ranges is equivalent to '\fBall\fR'. In this case, no failures are reported for unavailable ports, unless no ports could be forwarded at all.
-- Stefano
-- David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson
participants (2)
-
David Gibson
-
Stefano Brivio