All the processes verbose output is just written to stdout, and very
hard to parse after the test runs. Having test-runner write the
processes output to separate log files is much nicer to view after
the test runs.
A read/write file system is created which is separate and isolated
from the current FS mount (which mounts the whole host file system).
Doing this requires the user to create a folder somewhere to be used
as the mount point. This folder is where all the log files will end
up after test-runner runs.
Since test-runner must be run as root, care was taken to keep the
log files owned by the user which runs test-runner. The UID/GID
are passed to the VM, and any log files created are chown'ed back
to the user who ran test-runner.
execute_program was refactored, and both verbose and log arguments
were removed. The verbose argument was not really required since
we can do check_verbosity(argv[0]) internally. The 'log' flag was
only used along with --shell, and now the user can simply use
--log instead of dumping /tmp/iwd.log.
You can use this feature by specifying --log[=path] to test-runner.
If no path is provided the current directory will be used. Using
the --log flag will result in all processes being run with the
--verbose flag.
A new folder will be created under the --log path. The folder will
be of the name "run-<year>-<month>-<dat>-<PID>". Under this folder
you will find any global process logs. These are processes that
are only run once for ALL tests (hwsim, ifconfig, dbus etc.). There
will also be folders for specific tests that were run. Inside these
test folders will be logs of processes that are run per-test (iwd,
hostapd, python etc.).
Coverity reported this as a leak, but the test queue is actually
getting freed later and does not need to be freed locally in add_path
This basically reverts c0863e5bc6
The -U parameter only allowed for a list of unit tests to be run.
Most of the time for sanity checking you want to run all the unit
tests so this has been changed to take an optional argument.
Now, the -U flag (by itself) will run all unit tests. Running a
single or list of unit tests can still be achieved by:
--unit-tests=test-eapol,test-crypto
This makes every full test run consistent. The test list was being
stored as a hashmap, which has no been changed to a queue so we can
insert each test sorted.
Specifically, this defines the behavior when --shell is used when no
specific test is specified. In this case test-runner will assume the
'shell' test/sandbox should be used as the test environment as
running all autotests with --shell is not useful or feasable.
This option allows the script to be called with a raw XML file. This
is mostly useful for testing, but since its already implemented we
might as well include it.
Some hotspot networks do not contain SSID_STR, which was required
for both naming the provisioning file as well as the 'Name' key.
The DisplayedOperatorName is a better option for this 'Name' key
and could also be used for the filename.
Now, DisplayedOperatorName is preferred, and if not found SSID_STR
is used.
This will allow the user to see the iwd output in /tmp/iwd.log.
execute_program was extended to take a 'log' flag. If true, this
will cause the programs output to be stored in /tmp/<name>.log.
This is only useful when using the --shell command as this file
will go away once the VM stops. The verbose flag always overrides
the logging functionality.
For now only iwd output is logged when using --shell.
It is sometimes valuable to just boot into a shell in order to manually
test functionality. Since test-runner already is setup to run a minimal
kernel with all the necessary requirements for hostapd/iwd it made
sense to allow the user to do this.
If -s,--shell is passed into test runner, no python tests will be run.
The hw.conf file is still used to setup IWD and hostapd so once booted
into the shell you can still (manually) run the test (e.g. via iwctl).
This also works when using USB/PCI passthrough. This makes testing
out different kernel version with real hardware much quicker than
using the host kernel.
This tool will convert an iOS 'mobileconfig' file into the IWD
format. The tool only supports PEAP and TTLS networks, including
hotspots.
It will also parse out any certificate chains found in the
mobileconfig file, and verify they lead to a root CA found on the
system. If they do, this root CA will be used as the CACert in
the provisioning file.
Two new hardware configuration keys were added:
[radX]
iftype_disable=station,ap,adhoc,p2p_client,p2p_go,mesh_point
cipher_disable=wep40,wep104,tkip,ccmp,bip
Any of the above values are supported and can be disabled.
Support is coming to configure radios with a specific set of interface
type and cipher support, so the input to create_hwsim_radio is better
suited to use a parameter structure rather than adding more parameters.
The radio_confs key was parsed in a way that required all radios
to be specified in the list. This isnt optimal, as you may want to
specially configure a certain radio, while keeping all the others
default.
This change reworks some logic and allows any radio to be specially
configured on its own.
mac80211_hwsim now allows setting supported iftypes/ciphers. This patch
enables this support in hwsim. Specific iftypes/ciphers can now be
disabled via the command line when creating a radio:
Disable iftypes:
--iftype-disable station,ap,adhoc,p2p_client,p2p_go,mesh_point
Disable cipher types:
--cipher-disable ccmp,tkip,wep
The test should be aborted if there are not enough radios that support
AP mode. The iftype attribute will now be parsed during the wiphy dump
and a flag is set on the wiphy so we know to skip this radio when
creating the hostapd instances. Since hostapd gets started first, it
will automatically choose all the radios it needs which support AP mode.
This leaves the remainder of the radios (potentially STA only) for IWD.
In the PCI/USB passthrough changes the wiphy ID was changed to be an
unsigned integer, where id zero corresponded to an error when in native
hardware mode. Along with this, the radio ID for hwsim was changed to a
pre-increment (only in test-runner), so the radio IDs would start at 1.
The repercussions were not fully investigated, but if they were it would
have been seen that hwsim creates radios IDs starting at zero. This left
test-runner and hwsim with unsynchronized radio IDs, and radio zero
never got deleted after each test causing each successive test to
discover old radio IDs.
-nodefconfig doesn't exist anymore and according to the docs it either
had the same meaning or was implied by -no-user-config so it wouldn't be
needed anyway. -balloon doesn't exist anymore and according to
https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg06985.html
"-balloon none" was a nop, but I suspect -nodefaults may have already
had the effect of disabling creation of the virtio-balloon device.
This patch allows the host machine to pass through its PCI/USB network
cards into the test-runner virtual machine. By doing this we can run
nearly all the same autotests using physical/real wireless hardware.
First off, utilizing this feature requires a properly configured host
machine. There are kernel boot parameters and config files that need to
be configured before any of this will work. Unfortunately there is no
way around this, and hence this feature is not particularly aimed for
"the masses", but rather for specially configured test machines.
A new configuration file was introduced (tools/hw.conf) which is just an
example, it should be edited to work with the host machine using it. This
file merely holds the PCI addresses/USB bus of the devices you wish to pass
through to qemu.
Passing in this hardware config file with --hw <file> tells test-runner
that you are attempting to use this new feature. The tests themselves
do not need to change, its the initial test setup that required some
changes.
Since we are no longer creating radios we must discover the radios that
are present (once in the VM). This is done using borrowed code from IWD
to dump wiphys and interfaces. As the wiphys/interfaces are dumped, we
build up the wiphy list. In the hwsim case we still build this list up
when we create the radios, which hasn't changed. This does lead us to
have some special cleaup, where in the native case we just 'reset' the
list into its state pre-test (removing any hostapd flags). And as before
with the hwsim case we fully destroy and free the wiphy list, since a
new list will be created on the next test (along with new radios).
There should not need to be any changes to the tests themselves, but
potentially to some hw.conf files. A new key was introduced, 'needs_hwsim'
which need to be set on any tests that require the hwsim dbus API. This
tells test-runner to skip this test, otherwise it would fail in native
mode.
One last minor detail; the wiphy->id was changed to an unsigned int. This
is to match the type the kernel uses when dumping wiphys. Because of
this '0' is now the error case for both hwsim and native mode rather than
-1. Error checks were updated accordingly.
Move the interface creation code from configure_hw_radios to
configure_hostapd_instaces so as not to create unneeded interfaces on
the wiphys that IWD is going to manage. We pass a wiphy whitelist to
IWD later and IWD now creates the interfaces it needs on those managed
wiphys. Change TEST_WIPHY_LIST format to only include the interface
name for the wiphys used by hostapd.
Note that we still remove interfaces just before removing the hwsim
radios on exit, it seems like there's no point removing the interfaces
in that case.
Drop a pointless asignment of has_hw_conf to false when it's already
false, fix index when accessing radio_conf_list. Apparently the SETUP
group is not used in any of our test and wasn't tested itself so could
as well have removed the code.
The option may be present but may not parse correctly as a list in which
case has_hw_conf will be true but radio_conf_list will be NULL and we
might crash.
