3
0
mirror of https://git.kernel.org/pub/scm/network/wireless/iwd.git synced 2024-12-23 06:02:37 +01:00
Commit Graph

435 Commits

Author SHA1 Message Date
James Prestwood
7123f47f84 test-runner: remove special case for "root" namespace
This was a placeholder at one point but modules grew to depend on it
being a string. Fix these dependencies and set the root namespace
name to None so there is no more special case needed to handle both
a named namespace and the original 'root' namespace.
2021-09-07 12:45:26 -05:00
James Prestwood
1b44527511 test-runner: fix --monitor option
Two issues:
 - log-gid/uid was not being set
 - the log file was being appended if it already existed
2021-09-03 13:19:46 -05:00
Andrew Zaborowski
c79f583e0c test-runner: Fix verbose check for processes in namespaces
Check whether verbose output is enabled for process name arg[0] before
prepending the "ip netns exec" part to arg since arg[0] is going to be
"ip" after that.
2021-08-31 10:11:01 -05:00
James Prestwood
5de6af9f4a test-runner: only import Hwsim/HostapdCLI once
These modules only needed to be imported a single time for the entire
run of tests. This is significantly cheaper in terms of memory and
should prevent random OOM exceptions.
2021-08-27 20:07:26 -05:00
James Prestwood
cb7e9b5885 auto-t: hostapd.py: properly implement singleton (per-config)
This shouldn't change any functionality but it is much more convenient
in the cleanup path i.e. nothing special needs to be done.
2021-08-27 20:07:26 -05:00
James Prestwood
7d94aee5b6 test-runner: inherit Popen by Process class
The Procss class was doing quite a bit of what Popen already does like
storing the return code and process arguments. In addition the Process
class ended up storing a Popen object which was frequently accessed.

For both simplicity and memory savings have Process inherit Popen and
add the additional functionality test-runner needs like stdout
processing to output files and the console.

To do this Popen.wait() needed to be overridden to to prevent blocking
as well as wait for the HUP signal so we are sure all the process
output was written. kill() was also overritten to perform cleanup.

The most intrusive change was removing wait as a kwarg, and instead
requiring the caller to call wait(). This doesn't change much in
terms of complexity to the caller, but simplifies the __init__
routine of Process.

Some convenient improvements:
 - Separate multiple process instance output (Terminate: <args> will
   be written to outfiles each time a process dies.)
 - Append to outfile if the same process is started again
 - Wait for HUP before returning from wait(). This allows any remaining
   output to be written without the need to manually call process_io.
 - Store ctx as a class variable so callers don't need to pass it in
   (e.g. when using Process directly rather than start_process)
2021-08-26 08:52:48 -05:00
James Prestwood
382bbf1861 hwsim: remove unconditional packet delay
This was initially put in to solve an issue that was specific to
mac80211_hwsim where the connect callback would get queued and
delayed until after the connect event. This caused IWD to get very
confused.

Later it was found that "real" drivers can sometimes do this so
some code was added to IWD core to handle it.

Now there isn't much point to delay all frames unless a rule specifies
so change the behavior back to sending out frames immediately.
2021-08-18 19:52:46 -05:00
James Prestwood
a4ee9c8152 hwsim: require enabling rule before use
The hwsim Rule API was structured as properties so once a rule is
created it automatically starts being applied to frames. This happens
before anything has time to actually define the rule (source, destination
etc). This leads to every single frame being matched to the rule until
these other properties are added, which can result in unexpected behavior.

To fix this an "Enabled" property has been added and the rule will not
be applied until this is true.
2021-08-18 19:52:23 -05:00
James Prestwood
6f9dd97f50 test-runner: fix subtest option
This was not working correctly and would run all subtests when only a single
was requested
2021-08-18 16:57:38 -05:00
James Prestwood
81cf11b0f5 test-runner: fix exception in Hostapd __del__
If Hostapd did not start there was no CLI instance created and
that member should be be accessed.
2021-08-17 11:44:36 -05:00
James Prestwood
fd43a3938f test-runner: extend -S option
The -S/--sub-tests option allows the user to specify a test file
from inside an autotest. Inside this file there may also be many
test functions. This option is being extended to allow running
a single test function inside a test file. For example:

* Runs all test functions inside connection_test.py *
./test-runner -A some_test -S connection_test

* Runs only connection_test.py test_connect_success() *
./test-runner -A some_test -S connection_test.test_connect_success
2021-08-17 11:44:36 -05:00
James Prestwood
7b98a6ed9c test-runner: only remove GLib timeout if it exists
There was a race condition here where the GLib timeout could have
fired but the test function returned successfully prior to the
end of the while loop. This would end up causing source_remove to
print a warning that the source did not exist.

Instead check if the timeout fired prior to removing it.
2021-08-13 20:39:13 -05:00
James Prestwood
973ee269d3 auto-t: return existing instance from HostapdCLI
This addresses the TODO where HostapdCLI was creating separate
objects each time HostapdCLI was called. This was worked around
by manually setting the important members but instead the class
can be re-worked to act as somewhat of a singleton, per-config
at least.

If there is no HostapdCLI instance for a given config one is
created and initialized. Subsequent HostapdCLI calls (for the
same config) will be returned the same object rather than a
new one.
2021-08-13 20:39:13 -05:00
James Prestwood
dbd8842d51 test-runner: fix test skipping
Tests that called skipTest would result in an exception which would
hault execution as it was uncaught. In addition this wouldn't result
in an skipped test.

Now the actual test run is surrounded in a try/except block, skipped
exceptions are handled specifically, and a stack trace is printed if
some other exception occurs.
2021-08-13 16:44:14 -05:00
Denis Kenzior
6fee25af9b test-runner: Make run 2021-08-13 15:19:28 -05:00
James Prestwood
790bb44785 test-runner: don't always print failure to remove ctrl interface
This can happen, particularly with SAQuery when hostapd is forcefully
killed. Better to only print when running in debug mode.
2021-08-13 15:18:40 -05:00
James Prestwood
c03afda96f test-runner: start dmesg early with --follow
dmesg was being called at the very end of testing and dumped into
a log file. If many tests were run this could take quite a long
time and was timing out the default process wait. Instead --follow
can be used (basically like 'tail') which prints messages as they
come and avoids the time consuming full dump at the end.
2021-08-13 15:18:40 -05:00
James Prestwood
4e3cedc901 test-runner: wait for individual hostapd control sockets 2021-08-13 11:16:19 -05:00
James Prestwood
23cf6cb2e4 test-runner: implement non_block_wait
There was a common bit of code all over test-runner and utilities
which would wait for 'something' in a loop. At best these loops
would do the right thing and use the GLib.iteration call as to not
block the main loop, and at worst would not use it and just busy
wait.

Namespace.non_block_wait unifies all these into a single API to
a) do the wait correctly and b) prevent duplicate code.
2021-08-13 11:16:19 -05:00
James Prestwood
57933425ca test-runner: start HostapdCLI from test-runner
This gives the benefit of test-runner itself having access to
the CLI, e.g. for getting status information.
2021-08-13 10:46:13 -05:00
James Prestwood
ba6c8174d2 test-runner: run individual test functions manually
While losing the convenience of unittest this patch breaks out
each individual test function in order to run it manually and
get results. This vastly improves the user experience by seeing
which test file and function is being executed rather than simply
seeing "PASSED" for the entire test set.

In addition exceptions/failures are printed out as they happen
rather than at the end.
2021-08-13 10:43:47 -05:00
James Prestwood
89bddf551a test-runner: pass **kwargs in dbg() 2021-08-13 10:43:47 -05:00
James Prestwood
c23c37c07c test-runner: print uncaught test exceptions always 2021-08-13 10:43:47 -05:00
James Prestwood
89c1ec3d79 auto-t: make hwsim registration off by default
With the addition of connect_bssid/roam very few tests actually
require hwsim. Since hwsim can lead to problems with scan results
its best to have it off by default and have each test that needs
it explicitly turn it on.

Tests which previously turned it off have had that option removed.

Tests that do require hwsim still are vulnerable to scan result
problems, so for these tests beacon_int was added to the hostapd
config which seems to help with reliability somewhat.
2021-08-12 16:59:01 -05:00
James Prestwood
30158fe89c test-runner: don't run duplicate tests
If the user provides duplicate tests in the list only append one
of them to the list. This can happen accidentally when using
glob matches.
2021-08-12 15:14:07 -05:00
James Prestwood
9a15a46df5 test-runner: don't fatally exit on bad test configuration
If a test has no hw.conf file test-runner was fully exiting and not
running any additional tests. This shouldn't happen in practice
since all upstreamed tests should run, but if any locally created
tests existed like this, it would cause the entire test run to exit
early.

Instead raise an exception which bails out of only that test, and
allows the rest to continue.
2021-08-12 15:14:07 -05:00
James Prestwood
8203b1d542 test-runner: start IWD in developer mode
This is being done to utilize the Roam() developer method
2021-08-12 15:14:07 -05:00
James Prestwood
e2e625fa18 test-runner: refactor process output code
The Process class requires the ability to write out any processes
output to stdout, logging, or an explicit file, as well as store
it inside python for processing by test utilities. To accomplish
this each process was given a temporary file to write to, and that
file had an IO watch set on it. Any data that was written was then
read, and re-written out to where it needed to go. This ended up
being very buggy and quite complex due to needing to mess with
read/write pointers inside the file.

Popen already creates pipes to stdout if told, and they are accessable
via the p.stdout. Its then as simple as setting an IO watch on that
pipe and keeping the same code for reading out new data and writing
it to any files we want. This greatly reduces the complexity.
2021-07-29 17:51:25 -05:00
James Prestwood
52189d99f6 test-runner: don't reset dbus object on reset()
Doing this is redundant since the dbus object is initialized
anyways when a new test starts.
2021-07-29 17:51:25 -05:00
James Prestwood
b5c3fc1222 test-runner: increase RAM to 384MB
Occationally python will fatally terminate trying to load a test
using importlib with an out of memory exception. Increasing RAM
allows reliable exection of all tests.
2021-07-29 17:51:25 -05:00
James Prestwood
75b20ff1d0 test-runner: set msize for --monitor
This was added for logging but --monitor suffers the same warning
without setting msize as well.
2021-07-29 17:51:25 -05:00
James Prestwood
ba5bc1f5b3 test-runner: remove TLS debugging artifact
When logging is enabled TLS debugging is turned on which creates
a PEM file during runtime. There is no way for IWD itself to clean
this up since its meant to be there for debugging.
2021-07-29 17:51:25 -05:00
James Prestwood
8d40c37469 test-runner: set msize to remove runtime warning
Newer QEMU version warn that msize is set too low and may result
in poor IO performance. The default is 8KiB which QEMU claims is
too low. Explicitly setting to 10KiB removes the warning:

qemu-system-x86_64: warning: 9p: degraded performance: a
reasonable high msize should be chosen on client/guest side
(chosen msize is <= 8192).
See https://wiki.qemu.org/Documentation/9psetup#msize for details.
2021-07-28 09:32:42 -05:00
Denis Kenzior
074ed35b3d test-runner: Quiet a warning about msize
We seem to be not specifying the msize for the root filesystem, which
results in this warning being printed:

emu-system-x86_64: warning: 9p: degraded performance: a reasonable high msize should be chosen on client/guest side (chosen msize is <= 8192). See https://wiki.qemu.org/Documentation/9psetup#msize for details.

There doesn't seem to be much performance difference in the end since
iwd does not process large files.
2021-05-25 18:42:23 -05:00
Denis Kenzior
8e68e73c43 auto-t: Do not remove valgrind.log
Right now the --valgrind option logs to a static file named
'valgrind.log'.  This means that for any test that run multiple
instances of iwd, output is lost for all invocations except the last.
Fix that by using a per-process log file and making sure that all log
files are printed to stdout when the test ends.

This approach isn't perfect since it is possible for the pid to be
reused, but better than the current behavior.
2021-05-24 14:31:04 -05:00
James Prestwood
f9c2fa7bb2 test-runner: remove stale file after test
test-runner will print out if files were left behind after a
test which lets the developer know something was not cleaned
up. But in this case test-runner should also remove these files
so they are not left, and printed, for each subsequent test.
2021-05-10 10:12:48 -05:00
James Prestwood
751db56045 test-runner: increase RAM for valgrind
Certain tests like testAP spawn two IWD process in separate
namespaces. When --valrind is used this eats up quite a bit
of RAM and causes the VM to run out of memory and start
killing off processes.
2021-05-04 10:30:55 -05:00
James Prestwood
4085aceb4a test-runner: fix verbose output with --valgrind
Since using --valgrind actually runs IWD using the valgrind
process the --verbose flag would only work if 'valgrind' was
also specified. This was taken into account with is_verbose
but the actual logic enabling stdout did not use that helper.
This was due, in part, to logging since is_verbose will always
return true if --log is used. To fix this a new flag was added
to is_verbose which omits the --log check to handle this
specific case.
2021-04-29 12:56:51 -05:00
James Prestwood
8aac527e29 test-runner: fix process output truncation
There was a bug with process output where the last bit of data would
never make it into stdout or log files. This was due to the IO watch
being cleaned up when the process was killed and never allowing it
to finish writing any pending data.

Now the IO watch implementation has been moved out into its own
function (io_process) which is now used to write the final bits of
data out on process exit.
2021-04-22 13:25:23 -05:00
James Prestwood
d11974c717 test-runner: add timeout for waiting for process to finish
If a process hangs on exit test-runner would sit indefinitely
waiting.
2021-04-22 13:25:23 -05:00
James Prestwood
709c86086f test-runner: fix process cleanup loop
The processes in the list ultimately get removed for each
kill() call. This causes strange behavior since the list is
being iterated and each iteration is removing items. Instead
iterate over a new temporary list so the actual process list
can be cleaned up.
2021-04-22 13:25:23 -05:00
Denis Kenzior
38a7c4a8dc test-runner: Don't bother enabling IWD_GENL_DEBUG
IWD_GENL_DEBUG is not generally useful anymore as it just prints a
hexdump of the raw data on the socket.  The messages are quite verbose
and spam test-runner logs for little utility.
2021-04-20 10:48:50 -05:00
James Prestwood
0756c301f3 hwsim: add Prefix match rule support
Hwsim rules now have a 'Prefix' property which will allow
matching frames based on their payload data.
2021-04-05 17:47:42 -05:00
James Prestwood
66b4723216 test-runner: fix duplicate process output
Process output was being duplicated when -v was used. This was
due to both stderr and stdout being appended to the write_fd list
as well as stderr being set to stdout in the Popen call.

To fix this only stdout should be appended to the write_fd list,
but then there comes a problem with closing the streams. stdout
cannot be closed, so instead it is special cased. A new
verbose boolean was added to Process which, if True, will
cause any output to be written to stdout explicitly.
2021-03-02 10:37:22 -06:00
James Prestwood
c660feadf1 test-runner: stop/warn on processes running after test
This gives the dev some inidication their test did not clean up
correctly and running processes remain.
2021-02-26 10:09:56 -06:00
James Prestwood
670f063a30 test-runner: warn on files left after test completes 2021-02-26 10:09:56 -06:00
James Prestwood
2e55e00662 test-runner: start haveged in foreground
haveged was forking itself which then prevented test-runner from
cleaning it up.
2021-02-26 10:09:56 -06:00
James Prestwood
73b89555fa test-runner: fix Namespace class ref problems
The Namespace class was never being removed when tests finished.
This is fixed by unreffing the hwsim internal _radio object which
both cleans up the radio and allows the Namespace to be removed.
2021-02-26 10:09:56 -06:00
James Prestwood
d8abc88e4c test-runner: move process cleanup into kill()
This moves all the de-init code into kill(), which fixes a few
reference issues causing processes to hang around longer than
desired. If the process terminates on its own and/or the last
reference is lost __del__ will kill the process and clean up.
2021-02-26 10:09:56 -06:00
James Prestwood
48d27c982d test-runner: reset namespaces on test completion 2021-02-26 10:09:56 -06:00
James Prestwood
8f33926a75 test-runner: use cleanup() for start_dbus
Use a cleanup routine to remove the dbus_address/dbus_cfg files
2021-02-26 10:09:56 -06:00
James Prestwood
11f0abebe6 test-runner: add better Hostapd cleanup
There were a few issues with the cleanup of Hostapd. First the
process was only being killed, which did not actually remove
the process from the list.

In addition, with EAP-SIM/AKA tests, hostapd created sim_db
unix socket files which it does not clean up.
2021-02-26 10:09:36 -06:00
James Prestwood
bf9d2b6c52 test-runner: allow radio remove failure
This is somewhat of an open issue/TODO but for now this avoids
the exception caused by trying to remove a radio that has been
moved to a namespace. Once a radio is moved hwsim loses that phy
and can no longer interact with it. This causes the Destroy()
method call to fail.
2021-02-26 10:09:36 -06:00
James Prestwood
5e79cda7c6 test-runner: cleanup output files/others in Process
A cleanup parameter was added to __init__ which can be used
by processes which create any additional files or require more
a custom cleanup routine. Some additional house keeping was
done to make Process cleanup more robust.
2021-02-26 10:09:36 -06:00
James Prestwood
697b4ce82f test-runner: remove concept of 'multi_test'
Though multi-test processes seemed like a good idea in terms of
efficiency, the additional code/special cases was not worth it
for the only two multi-test processes (dbus/haveged). Intead this
concept was removed completely and TestContext/Namespaces will
now start all processes for each individual test. This also is
fair to all tests as a previous failed test could end up bleeding
into future tests.
2021-02-26 10:09:36 -06:00
James Prestwood
90b695cfdf test-runner: fix start_dbus and clean up config
start_dbus was not returning the Process class which was
expected by the caller. This resulted in the config not
being cleaned up.
2021-02-19 10:16:37 -06:00
James Prestwood
4682c22ece test-runner: add __str__ method to Process
Printing out processes was done manually but instead we can
make Process printing extendable by adding its own __str__
method. This now will print if the process is a multi-test
process as well.
2021-02-19 10:16:37 -06:00
James Prestwood
c10bd14cde test-runner: use unique name for namespace output files
Output files in namespaces were not handled differently and would
end up overwriting/duplicating files from the root namespace. These
are now named /tmp/<process>-<namespace>-out.
2021-02-19 10:16:37 -06:00
James Prestwood
1791ef1dc7 test-runner: refactor Process class
The process class was quite hard to understand, and somewhat
fragile when multiple output options were needed like verbose
and logging, and in some cases even an additional output file.

To make things simpler we can have all processes output to a
temporary file (/tmp/<name>-out) and set a GLib IO watch on
that file. When the IO watch callback fires any additional
files (stdout, log files, output files) can be written to.

For wait=True processes we do not use an IO watch, but do
the same thing once the process exits, write to any additional
output files using the process output we already have.
2021-02-19 10:16:37 -06:00
James Prestwood
272a60a478 test-runner: require root user to run with --monitor
Since the monitor file requires touching the host file
system, root access is required.
2021-02-19 10:16:37 -06:00
James Prestwood
c2d71fdaa5 test-runner: clear log dir before new log test run
The log dir was never being cleaned out prior to a new logging
test run. This could leave old stale files around. Note that this
will remove any past log files so if you need them, you want to
make a copy before running test-runner with --log again.
2021-02-19 10:16:37 -06:00
James Prestwood
a0865c9010 test-runner: start dmesg process with start_process
This avoids the need to pass in the context explicitly.
2021-02-19 10:16:37 -06:00
Denis Kenzior
897ef661fe hwsim: Don't use l_genl_msg_new_sized with a conditional
l_genl API will now automatically grow message buffers as needed, so
there's no need to make the logic over-complicated
2021-02-08 15:37:21 -06:00
James Prestwood
af8438878f test-runner: clean up dbus path file properly
Dbus should be started as a multi-test process from the
TestContext, which leaves the dbus address file around for
the full test run. For Namespaces dbus-daemon should be
closed when the Namespace closes.
2021-02-08 11:56:42 -06:00
James Prestwood
a8768e354d test-runner: print error if kernel/qemu path is not found 2021-02-05 18:01:49 -06:00
James Prestwood
f21e79c81c test-runner: add monitor to path 2021-02-05 18:01:49 -06:00
James Prestwood
ed80dc68aa test-runner: clean up temporary files 2021-02-05 18:01:43 -06:00
James Prestwood
b60f564bed test-runner: print Radio class path 2021-02-05 14:43:17 -06:00
James Prestwood
b0e970ae38 test-runner: fix logging, verbose, and process output
There were some major problems related to logging and process
output. Tests which required output from start_process would
break if used with '--log/--verbose'. This is because we relied
on 'communicate' to retrieve the process output, but Popen does
not store process output when stdout/stderr are anything other
than PIPE.

Intead, in the case of logging or outfiles, we can simply read
from the file we just wrote to.

For an explicit --verbose application we must handle things
slightly different. A keyword argument was added to Process,
'need_out' which will ensure the process output is kept
regardless of --log or --verbose.

Now a user should be able to use --log/--verbose without any
tests failing.
2021-01-26 13:41:23 -06:00
James Prestwood
cc345582ad test-runner: don't always print "verbose on for ..."
This shouldn't have been a dbg print, but rather a normal print
which will only be printed when '-d' is used.
2021-01-26 13:41:01 -06:00
James Prestwood
296f1d0d1c test-runner: fix verbose arguments as single string
The verbose arguments come in from the QEMU command line as a
single string. This should have been split into an array immediately
but was not. This led to issues like hostapd debug being enabled
when "-v hostapd_cli" was passed in.
2021-01-26 13:40:49 -06:00
Andrew Zaborowski
a55f8864d2 testrunner: Fix /tmp files cleanup on error
Since the list of files copied to /tmp was part of the return value from
pre_test(), if an exception occurred inside pre_test(), "copied" would
be undefined and the post_test(ctx, copied) call in the finally clause
cause another exception:

raceback (most recent call last):
  File "/home/balrog/repos/iwd/tools/test-runner", line 1508, in <module>
    run_tests()
  File "/home/balrog/repos/iwd/tools/test-runner", line 1242, in run_tests
    run_auto_tests(config.ctx, args)
  File "/home/balrog/repos/iwd/tools/test-runner", line 1166, in run_auto_tests
    post_test(ctx, copied)
UnboundLocalError: local variable 'copied' referenced before assignment

(apart from not being able to clean up the files).  Pass "copied" as a
paremeter to pre_test instead.
2021-01-25 14:01:58 -06:00
James Prestwood
01c108938e test-runner: Use DBus for hwsim radios
Use the hwsim DBus API rather than command line. This both is
faster and more dynamic than doing so with the command line.
This also avoids tracking the radio ID since we can just hang
on to the radio Dbus object directly.
2020-12-17 20:13:56 -06:00
James Prestwood
5567caf64a hwsim: add iftype/cipher disabling through DBus
Update the Dbus API to allow disabling iftypes and ciphers
just as you can with the command line.
2020-12-17 20:11:25 -06:00
James Prestwood
fe0879f7d1 test-runner: start hwsim always (optionally --no-register)
Always start hwsim but if tests do not need radio rules start
with the --no-register option.
2020-12-17 20:11:13 -06:00
James Prestwood
e8031bab2a test-runner: add wait_for_dbus_service
Common API to wait for a DBus service to appear on the bus
2020-12-17 20:10:57 -06:00
James Prestwood
6a1853b01a hwsim: change radio Create() to take a dictionary
The Create() API was limited to only taking a Name and boolean
(for p2p enabling). The actual hwsim nl80211 API can take more
attributes than this (which are actually utilized when creating
from the command line). To get the DBus API up to the same
functionality the two arguments in Create were replaced with
a single dictionary. This allows for extending later if more
arguments are needed.
2020-12-17 20:10:33 -06:00
James Prestwood
799ab03f59 hwsim: check pending_create_msg before replying
In the NEW_RADIO callback hwsim was assuming that DBus had no
yet replied to the Create() method. In some cases the NEW_RADIO
event fires before the actual callback which will respond to
DBus. This causes a crash in the create callback.
2020-12-17 20:08:54 -06:00
James Prestwood
554a78fbf0 hwsim: add --no-register option
Starts hwsim but does not register to mac80211_hwsim. This is to
allow autotests to disable hwsim, while still having the ability
to create/destroy radios over DBus.
2020-12-17 20:08:49 -06:00
James Prestwood
4e4ba6769d test-runner: add iwd-acd to verbose options
This will turn on IWD_ACD_DEBUG
2020-12-08 15:16:28 -06:00
James Prestwood
ada46fb2db test-runner: set processor count for VM
For better reliability the processor count is now set to qemu.
In cases of low CPU count (< 2) hosts the processor count is
limited to 1. Otherwise half of the host cores will be used for
the VM.
2020-12-08 15:15:51 -06:00
James Prestwood
41b8a32dfc test-runner: have start_iwd take a storage dir
Allow the storage directory (default /tmp/iwd) to be configured
just like the state directory. This is in order to support multiple
IWD instances which require separate storage directories for network
provisioning files.
2020-12-08 15:15:33 -06:00
James Prestwood
e1e1c4edd1 test-runner: introduce network namespaces
Our simulated environment was really only meant to test air-to-air
communication by using mac80211_hwsim. Protocols like DHCP use IP
communication which starts to fall apart when using hwsim radios.
Mainly unicast sockets do not work since there is no underlying
network infrastructure.

In order to simulate a more realistic environment network namespaces
are introduced in this patch. This allows wireless phy's to be added
to a network namespace and unique IWD instances manage those phys.
This is done automatically when 'NameSpaces' entries are configured
in hw.conf:

[SETUP]
num_radios=2

[NameSpaces]
ns0=rad1,...

This will create a namespace named ns0, and add rad1 to that
namespace. rad1 will not appear as a phy in what's being called the
'root' namespace (the default namespace).

As far as a test is concerned you can create a new IWD() class and
pass the namespace in. This will start a new IWD instance in that
namespace:

ns0 = ctx.get_namespace('ns0')
wd_ns0 = IWD(start_iwd=True, namespace=ns0)

'wd_ns0' can now be used to interact with IWD in that namespace, just
like any other IWD class object.
2020-11-18 11:01:11 -06:00
James Prestwood
a6808aa55a auto-t: introduce new autotest syntax
Sometimes improperly written tests can end up causing future tests
to fail. For faster debugging you can now add a '+' after a given
autotest which will start that test and run all tests which come
alphabetically after it (as if you are running a full autotest suite).

Example:

./test-runner -A testWPA+

This will run testWPA, testWPA2, testWPA2-no-CCMP, testWPA2-SHA256,
and testWPA2withMFP.
2020-11-16 17:05:45 -06:00
James Prestwood
423f9a4e60 auto-t: do away with -1 test results
This can result in strange test results since there was no less
than zero checks before subtracting the total tests from failed
tests. In case of an internal exception we can just set all values
to zero. This will be handled specially as we do for timeout
errors.
2020-11-16 17:05:45 -06:00
James Prestwood
1eb82d661f auto-t: catch exceptions in post_test 2020-11-16 17:05:45 -06:00
James Prestwood
f88d45c9d3 test-runner: delay starting the shell until after pre_test
This makes --shell somewhat more useful by copying all the test
files and starting test processes before dumping the user into
a shell.
2020-11-16 17:05:45 -06:00
James Prestwood
6bf514c4c3 test-runner: store process stdout
If a blocking process is started store the output in case
the caller needs it. Output will be stored in Process.out.
2020-11-16 17:05:45 -06:00
James Prestwood
4bf8bf2396 test-runner: add option to specify subtests to run
You can now specify a limited list of subtests to run out of a
full auto-test using --sub-tests,-S. This option is limited in
that it is only meant to be used with a single autotest (since
it doesn't make much sense otherwise).

The subtest can be specified both with or without the file
extension.

Example usage:

./test-runner -A testAP -S failure_test,dhcp_test.py

This will only run the two subtests and exclude any other *.py
tests present in the test directory.
2020-11-16 17:05:45 -06:00
James Prestwood
c098db1837 test-runner: fix Radio/Interface circular reference
Code was added with commit 04487f575b which passes a radio object
to the Interface class constructor and stores it in the Interface
object. The radio class also stores each Interface object which
creates a circular reference and causes the Radio to stick around
long after the tests finishes.

I cannot see why the Interface needs to keep track of the Radio
object. None of the wpa_supplicant utilities use this so it has
been removed.
2020-11-03 13:57:37 -06:00
James Prestwood
26c4b8b4fe test-runner: add switch for genl debugging
Providing 'iwd-genl' in the list of verbose options will enable
IWD_GENL_DEBUG.
2020-10-22 09:24:55 -05:00
James Prestwood
334b03478b test-runner: enable valgrind track origins
This gives a more exact location to where a memory problem
is happening. It also uses more RAM so the VM was upgraded to
256MB from 192MB.
2020-10-22 09:24:55 -05:00
Andrew Zaborowski
04487f575b test-runner: Reserve radios for wpa_supplicant
Add support for a WPA_SUPPLICANT section in  hw.conf where
'radN=<config_path>' lines will only reserve radios and create
interfaces for the autotest to be able to start wpa_supplicant on them,
i.e. this prevents iwd or hostapd from being started on them but doesn't
start a wpa_supplicant instance by itself.
2020-10-19 17:20:21 -05:00
James Prestwood
313d8dbbed auto-t: remove requirement for /var/lib/{ead,iwd}
The host systems configuration directories for IWD/EAD were
being mounted in the virtual machine. This required that the
host create these directories before hand. Instead we can
just set up the system and IWD/EAD to use directories in /tmp
that we create when we start the VM. This avoids the need for
any host configuration.
2020-10-14 13:03:17 -05:00
Andrew Zaborowski
db038022e8 test-runner: Make hwsim medium optional
Allow the "hwsim_medium=no" setting in hw.conf's SETUP section to
disable starting hwsim.  It looks like the packets going through
userspace add enough latency that active scans don't work, probe
responses don't arrive within the "dwell time" or probe requests are not
ACKed on time.  I've tried modifying tools/hwsim.c to respond with the
HWSIM_CMD_TX_INFO_FRAME cmd as the first thing after receiving a
HWSIM_CMD_FRAME and even skipping the queue in ell/genl.c by writing the
command synchronously, but neither helped enough to make the scans work.
This does not rule out that hwsim or the way our scans are done can be
fixed and that would obviously be better than what I did in this patch.
2020-09-29 13:20:29 -05:00
Andrew Zaborowski
b321db20ad test-runner: Add flags for DHCP and TLS verbose output
Add "iwd-tls" and "iwd-dhcp" flags for -v.  These should automatically
enable "-v iwd" output because of the substring matching.
2020-09-29 13:20:26 -05:00
Andrew Zaborowski
15c04bb395 test-runner: Enable --p2p when creating interfaces
Enable p2p for the hwsim radios so that we can test P2P.  This can be
made optional for the tests but it seems safe to enable always.
2020-09-29 13:20:24 -05:00
James Prestwood
1a02fdcefd test-runner: add iwmon options
This extends test-runner to also use iwmon if --log is enabled.
For this case the iwmon log will be found inside each test
log directory.

A new option, --monitor <file> was added in case full logging isn't
desired (potentially for timing issues) but a iwmon log is needed.
Be aware that when --monitor is used test-runner will mount the
entire parent directory. test-runner itself will only write to the
file specified, but just know that the parent directory is available
as read-write inside the VM.

--log takes precedence over --monitor, meaning the iwmon log will
be written to <logdir>/<test>/iwmon instead of the file specified
with --monitor if both options are provided.
2020-09-29 09:10:07 -05:00
James Prestwood
9897efe75c test-runner: allow EAD-based autotests
The virtual environment changed slightly adding two network adatpers
which are connected to the same backend so they can communicate with
each other (basically connected to a switch). The hostapd command
line was modified to allow no interfaces to be passed in which lets
us create zero radios but still specify a radius_config file.
2020-09-17 16:19:39 -05:00
James Prestwood
6b3d6f9d60 test-runner: open up dbus config for dbus-monitor
dbus-monitor was not able to eavesdrop on method calls without
some changed to the dbus config.
2020-09-16 14:35:38 -05:00