iwd/tools/test-runner

#!/usr/bin/python3

from runner import Runner

Runner().start()
auto-t: introduce pure python test-runner re-write This patch completely re-writes test-runner in Python. This was done because the existing C test-runner had some clunky work arounds and maintaining or adding new features was starting to become a huge pain. There were a few aspects of test-runner which continually had to be dealt with when adding any new functionality: * Argument parsing: Adding new arguments to test-runner wasn't so bad, but if you wanted those arguments passed into the VM it became a huge pain. Arguments needed to be parsed, then re-formatted into the qemu command line, then re-parsed in a special order (backwards) once in the VM. The burden for adding new arguments was quite high so it was avoided (at least by me) at all costs. * The separation between C and Python: The tests are all written in python, but the executables, radios, and interfaces were all created from C. The way we solved this was by encoding the require info as environment variables, then parsing those from Python. It worked, but it was, again, a huge pain. * Process management: It started with all processes being launched from C, but eventually tests required the ability to start IWD, or kill hostapd ungracefully in order to test certain functionality. Since the processes were tracked in C, Python had no way of signalling that it killed a process and when it started one C had no idea. This was mitigated (basically by killall), but it was no where close to an elegant solution. Re-writing test-runner in python solves all these problems and will be much easier to maintain. * Argument parsing: Now all arguments are forwarded automatically to the VM. The ArgParse library takes care of parsing and each argument is stored in a dictionary. * Separation between C and Python: No more C, so no more separation. * Process management: Python will now manage all processes. This allows a test to kill, restart, or start a new process and not have to remember the PID or to kill it after the test. There are a few more important aspects of the python implementation that should now be considered when writing new tests: * The IWD constructor now has different default arugments. IWD will always be started unless specified and the configuration directory will always be /tmp * Any non .py file in the test directory will be copied to /tmp. This avoids the need for 'tmpfs_extra_stuff' completely. ctrl_interface will automatically be appended to every hostapd config. There is no need to include this in a config file from now on. * Test cleanup is extremely important. All tests get run in the same interpreter now and the tests themselves are actually loaded as python modules. This means e.g. if you somehow kept a reference to IWD() any subsequent tests would not start since IWD is still running. * For debugging, the test context can be printed which shows running processes, radios, and interfaces. Three non-native python modules were used: PrettyTable, colored, and pyroute2 $ pip3 install prettytable $ pip3 install termcolor $ pip3 install pyroute2 2020-09-11 01:12:23 +02:00			`#!/usr/bin/python3`

test-runner: remove environment specific code This removes all the Qemu/environment related code as this has been moved into runner.py. 2022-04-01 01:16:29 +02:00			`from runner import Runner`

test-runner: rename test-runner, add run-tests In order to keep all test-runner dev scripts working and to work with the new runner.py system some file renaming was required. test-runner was renamed to run-tests A new test-runner was added which only creates the Runner() class. 2022-04-01 01:16:30 +02:00			`Runner().start()`