[PATCH v5 00/18] RFC: Dynamic configuration update implementation
Here's the next draft of dynamic configuration updates. This now can successfully update rules, though I've not tested it very extensively. Patches 1..8/18 are preliminary reworks that make sense even without pesto - feel free to apply if you're happy with them. I don't think the rest should be applied yet; we need to at least harden it so passt can't be blocked indefinitely by a client which sends a partial update then waits. Based on my earlier series reworking static checking invocation. TODO: - Don't allow a client which sends a partial configuration then blocks also block passt - Allow pesto to clear existing configuration, not just add - Allow pesto selectively delete existing rules, not just add Changes in v5: * If multiple clients connect at once, they're now blocked until the first one finishes, instead of later ones being discarded Changes in v4: * Merged with remainder of forward rule parsing rework series * Fix some bugs in rule checking pointed out by Laurent * Significantly cleaned up option parsing code * Changed from replacing all existing rules to adding new rules (clear and remove still TBD) * Somewhat simplified protocol (pif names and rules sent in a single pass) * pesto is now allocation free * Fixed commit message and style nits pointed out by Stefano Changes in v3: * Removed already applied ASSERT() rename * Renamed serialisation functions * Incorporated Stefano's extensions, reworked and fixed * Several additional cleanups / preliminary reworks Changes in v2: * Removed already applied cleanups * Reworked assert() patch to handle -DNDEBUG properly * Numerous extra patches: * Factored out serialisation helpers and use them for migration as well * Reworked to allow ip.[ch] and inany.[ch] to be shared with pesto * Reworks to share some forwarding rule datatypes with pesto * Implemented sending pif names and current ruleset to pesto David Gibson (18): conf, fwd: Stricter rule checking in fwd_rule_add() fwd_rule: Move ephemeral port probing to fwd_rule.c fwd, conf: Move rule parsing code to fwd_rule.[ch] fwd_rule: Move conflict checking back within fwd_rule_add() fwd: Generalise fwd_rules_info() pif: Limit pif names to 128 bytes fwd_rule: Fix some format specifiers tap, repair: Use SOCK_NONBLOCK and SOCK_CLOEXEC on Unix sockets pesto: Introduce stub configuration tool pesto, log: Share log.h (but not log.c) with pesto tool pesto, conf: Have pesto connect to passt and check versions pesto: Expose list of pifs to pesto and optionally display ip: Prepare ip.[ch] for sharing with pesto tool inany: Prepare inany.[ch] for sharing with pesto tool pesto: Read current ruleset from passt/pasta and optionally display it pesto: Parse and add new rules from command line pesto, conf: Send updated rules from pesto back to passt/pasta conf, fwd: Allow switching to new rules received from pesto .gitignore | 2 + Makefile | 54 ++-- common.h | 122 +++++++++ conf.c | 686 ++++++++++++++++++++++----------------------------- conf.h | 2 + epoll_type.h | 4 + flow.c | 4 +- fwd.c | 169 ++++--------- fwd.h | 41 +-- fwd_rule.c | 603 ++++++++++++++++++++++++++++++++++++++++++-- fwd_rule.h | 66 ++++- inany.c | 19 +- inany.h | 17 +- ip.c | 56 +---- ip.h | 4 +- lineread.c | 2 +- log.h | 59 ++++- passt.1 | 5 + passt.c | 8 + passt.h | 8 + pesto.1 | 46 ++++ pesto.c | 470 +++++++++++++++++++++++++++++++++++ pesto.h | 55 +++++ pif.c | 2 +- pif.h | 8 +- repair.c | 9 +- serialise.c | 7 + serialise.h | 1 + siphash.h | 13 + tap.c | 64 ++++- util.c | 2 +- util.h | 110 +-------- 32 files changed, 1921 insertions(+), 797 deletions(-) create mode 100644 common.h create mode 100644 pesto.1 create mode 100644 pesto.c create mode 100644 pesto.h -- 2.53.0
All current pif names are quite short, and we expect them to remain short
when/if we allow arbitrary pifs. However, because of the structure of
the current code we don't enforce any limit on the length.
This will become more important with dynamic configuration updates, so
start enforcing a length limit. Specifically we allow pif names to be up
to 128 bytes (PIF_NAME_SIZE), including the terminating \0. This is
more or less arbitrary, but seems like it should be comfortably enough for
all the cases we have in mind.
Signed-off-by: David Gibson
sock_unix(), which creates a listening Unix socket, doesn't set the
SOCK_NONBLOCK flag. Generally, this doesn't matter because we only
accept() once we've received an epoll event awaiting a connection. However
we will need non-blocking accept() for the upcoming control/configuration
socket. Always add SOCK_NONBLOCK, which is more robust and in keeping with
the normal non-blocking style of passt.
In tap.c, always set SOCK_NONBLOCK and SOCK_CLOEXEC on the accept()ed
sockets as well, which we weren't doing in all cases before. According to
accept(2), in Linux accepted sockets do *not* inherit these flags from the
listening socket. Also check for failures of accept, discarding EAGAIN
silently (a spurious epoll event) and warning for other errors.
In repair.c, similarly always add CLOEXEC. Use NONBLOCK for discard
sockets, but *not* for the final repair socket, since we want blocking
transactions during migration.
Signed-off-by: David Gibson
The new warnings in fwd_rule_add() have some not quite technically correct
format specifiers. For some weird reason these don't trip warnings on
passt itself, but do when we re-use this code in the upcoming configuration
client. Fix them.
Signed-off-by: David Gibson
In pesto we're going to want several levels of error/warning messages, much
like passt itself. Particularly as we start to share mode code between
passt and pesto, we want to use a similar interface to emit those. However
we don't want to use the same implementation - logging to a file or syslog
doesn't make sense for the command line tool.
To accomplish this loosely share log.h, but not log.c between pesto and
passt. In fact, an #ifdef means even most of log.h isn't actually shared,
but we do provide similar warn(), die() etc. macros.
This includes the *_perror() variants, which need strerror(). However,
we want to avoid allocations for pesto as we do for passt, and strerror()
allocates in some libc versions. Therefore, also move our workaround for
this to be shared with pesto.
Signed-off-by: Stefano Brivio
fwd_rules_info() is used to print a full table of forwarding rules for
debugging or the like. Currently it has one caller, and uses info() to
dump the messages. However for the upcoming configuration client, we're
going to want to dump the rules in some similar, but not quite identical
ways. For example, at different severity levels, or to stdout instead of
stderr / system log / logfile.
So, generalise fwd_rules_info() to fwd_rules_dump() which takes a printing
function as a parameter. Because we want this to work with "functions"
like info, which is actually a macro, we have to convert fwd_rules_dump()
to a macro as well. We also allow the prefix and suffix for each rule /
line to be provided as a parameter.
Signed-off-by: David Gibson
Start implementing pesto in earnest. Create a control/configuration
socket in passt. Have pesto connect to it and retrieve a server greeting
Perform some basic version checking.
Signed-off-by: David Gibson
Extend the dynamic update protocol to expose the pif indices and names
from a running passt/pasta to the pesto tool. pesto records that data
and, if requested with a new --show flag, prints it out.
Signed-off-by: David Gibson
Build a new "pesto" binary, which will become the tool to update a running
passt/pasta's configuration. For now, we just build a stub binary which
sets up a basic environment, parses trivial command line options but does
nothing else.
Signed-off-by: David Gibson
Most things in ip.[ch] related purely to IP addresses and headers with
no dependency on other passt/pasta internals. A number of these will be
useful to re-use in pesto. The exception is ipv6_l4hdr() which uses
iov_tail.
The only caller of this is in tap.c, so move the function there. Along
with moving the constant byteswapping functions to common.h, that lets
ip.[ch] to be linked into pesto as well as passt/pasta.
Signed-off-by: David Gibson
inany contains a number of helpful functions for dealing with addresses
which might be IPv4 or IPv6. We're going to want to use that in pesto.
For the most part inany doesn't depend on other passt/pasta internals,
however it does depend on siphash.h, which pesto doesn't need.
Move the single dependent function, inany_siphash_feed() to siphash.h,
renaming to match. Use that include inany.[ch] into pesto as well as
passt/pasta. While we're there reformat pesto.c's header comment to match
the convention used in most other files.
Signed-off-by: David Gibson
Implement serialisation of our current forwarding rules in conf.c,
deserialising it to display in the pesto client. Doing this requires
adding ip.c, inany.c, bitmap.c, lineread.c and fwd_rule.c to the pesto
build. With previous preparations that now requires only a trivial change
to lineread.c.
Signed-off-by: David Gibson
Extend pesto to send the updated rule configuration back to passt/pasta.
Extend passt/pasta to read the new configuration and store the new rules in
a "pending" table. We don't yet attempt to activate them.
Signed-off-by: Stefano Brivio
This adds parsing of options using fwd_rule_parse(), validates them and
adds them to the existing rules. It doesn't yet send those rules back to
passt or pasta.
Signed-off-by: Stefano Brivio
We can now receive updates to the forwarding rules from the pesto client
and store them in a "pending" copy of the forwarding tables. Implement
switching to using the new rules.
The logic is in a new fwd_listen_switch(). For now this closes all
listening sockets related to the old tables, swaps the active and pending
tables, then listens based on the new tables. In future we look to improve
this so that we don't temporarily stop listening on ports that both the
old and new tables specify.
Signed-off-by: David Gibson
participants (1)
-
David Gibson