Here's another draft of my work on testing passt with Avocado and the
exeter library I recently created. It includes Cleber's patch adding
some basic Avocado tests and builds on that.
This draft is IMO significantly less janky than the last one, but
certainly not jank-free:
* We create the Avocado job files from the exeter sources in the
Makefile. Ideally Avocado would eventually be extended to handle
this itself
* The names that Avocado sees for each test aren't great
Stefano,
If you could look particularly at 7/15 and 15/15 which add the real
tests for passt/pasta, that would be great. The more specific you can
be about what you find ugly about how the tests are written, then
better I can try to address that.
I suspect it will be easier to actually apply the series, then look at
the new test files (test/build/build.py, and test/pasta/pasta.py
particularly). From there you can look at as much of the support
library as you need to, rather than digging through the actual patches
to look for that.
Cleber,
If you could look at 4..7/22 particularly to review how I'm connecting
the actual tests to the Avocado runner, that would be helpful.
Changes since v2:
* Added mypy type checking throughout
* Use exeter "scenarios" to reduce boilerplate
* Folded together a number of closely related patches (from 22
patches down to 15)
* Entirely avoid open-coded Python context managers in favour of
contextlib.contextmanager
* No longer accidentally run a lot of the "meta" tests in the "real"
testsuite
* So, so many assorted cleanups
Cleber Rosa (1):
test: run static checkers with Avocado and JSON definitions
David Gibson (14):
test: Adjust how we invoke tests with run_avocado
test: Extend make targets to run Avocado tests
test: Exeter based static tests
tasst: Support library and linters for tests in Python
tasst/cmdsite: Base helpers for running shell commands in various
places
test: Add exeter+Avocado based build tests
tasst/unshare: Add helpers to run commands in Linux unshared
namespaces
tasst/ip: Helpers for configuring IPv4 and IPv6
tasst/veth: Helpers for constructing veth devices between namespaces
tasst: Helpers to test transferring data between sites
tasst: Helpers for testing NDP behaviour
tasst: Helpers for testing DHCP & DHCPv6 behaviour
tasst: Helpers to construct a simple network environment for tests
avocado: Convert basic pasta tests
test/.gitignore | 2 +
test/Makefile | 59 +++++++-
test/avocado/static_checkers.json | 12 ++
test/build/.gitignore | 2 +
test/build/build.py | 86 +++++++++++
test/build/static_checkers.sh | 28 ++++
test/meta/.gitignore | 1 +
test/meta/lint.sh | 28 ++++
test/pasta/.gitignore | 1 +
test/pasta/pasta.py | 130 +++++++++++++++++
test/run_avocado | 52 +++++++
test/tasst/.gitignore | 1 +
test/tasst/__init__.py | 11 ++
test/tasst/__main__.py | 40 +++++
test/tasst/cmdsite.py | 193 ++++++++++++++++++++++++
test/tasst/dhcp.py | 190 ++++++++++++++++++++++++
test/tasst/ip.py | 234 ++++++++++++++++++++++++++++++
test/tasst/ndp.py | 106 ++++++++++++++
test/tasst/pasta.py | 48 ++++++
test/tasst/scenario/__init__.py | 12 ++
test/tasst/scenario/simple.py | 108 ++++++++++++++
test/tasst/selftest/__init__.py | 16 ++
test/tasst/transfer.py | 193 ++++++++++++++++++++++++
test/tasst/unshare.py | 166 +++++++++++++++++++++
test/tasst/veth.py | 104 +++++++++++++
25 files changed, 1822 insertions(+), 1 deletion(-)
create mode 100644 test/avocado/static_checkers.json
create mode 100644 test/build/.gitignore
create mode 100644 test/build/build.py
create mode 100644 test/build/static_checkers.sh
create mode 100644 test/meta/.gitignore
create mode 100644 test/meta/lint.sh
create mode 100644 test/pasta/.gitignore
create mode 100644 test/pasta/pasta.py
create mode 100755 test/run_avocado
create mode 100644 test/tasst/.gitignore
create mode 100644 test/tasst/__init__.py
create mode 100644 test/tasst/__main__.py
create mode 100644 test/tasst/cmdsite.py
create mode 100644 test/tasst/dhcp.py
create mode 100644 test/tasst/ip.py
create mode 100644 test/tasst/ndp.py
create mode 100644 test/tasst/pasta.py
create mode 100644 test/tasst/scenario/__init__.py
create mode 100644 test/tasst/scenario/simple.py
create mode 100644 test/tasst/selftest/__init__.py
create mode 100644 test/tasst/transfer.py
create mode 100644 test/tasst/unshare.py
create mode 100644 test/tasst/veth.py
--
2.46.0