[RFC,v3,0/3] Add test to verify probe of devices from discoverable busses

Hi,

for this v3 I changed the approach for identifying devices in a stable
way from the match fields back to the hardware topology (used in v1).
The match fields were proposed as a way to avoid the possible issue of
PCI topology being reconfigured, but that wasn't observed on any real
system so far. However using match fields does allow for a real issue if
an external device similar to an internal one is connected to the
system, which results in a change of the match count and therefore a
test failure. So using the HW topology was chosen as the most reliable
approach.

The per-platform device description file now uses YAML following a
suggestion from Chris Obbard, and the test script was re-written in
python to handle the new YAML format.

A second sample board file is also now included for an x86 platform,
which contains an USB controller behind a PCI controller, which wasn't
possible to describe in v1.

Thanks,
Nícolas

v2: https://lore.kernel.org/all/20231127233558.868365-1-nfraprado@collabora.com
v1: https://lore.kernel.org/all/20231024211818.365844-1-nfraprado@collabora.com

Original cover letter:

This is part of an effort to improve detection of regressions impacting
device probe on all platforms. The recently merged DT kselftest [3]
detects probe issues for all devices described statically in the DT.
That leaves out devices discovered at run-time from discoverable busses.

This is where this test comes in. All of the devices that are connected
through discoverable busses (ie USB and PCI), and which are internal and
therefore always present, can be described in a per-platform file so
they can be checked for. The test will check that the device has been
instantiated and bound to a driver.

Patch 1 introduces the test. Patch 2 and 3 add the device definitions
for the google,spherion machine (Acer Chromebook 514) and XPS 13 as
examples.

This is the output from the test running on Spherion:

TAP version 13
Using board file: boards/google,spherion.yaml
1..8
ok 1 /usb2-controller@11200000/1.4.1/camera.device
ok 2 /usb2-controller@11200000/1.4.1/camera.0.driver
ok 3 /usb2-controller@11200000/1.4.1/camera.1.driver
ok 4 /usb2-controller@11200000/1.4.2/bluetooth.device
ok 5 /usb2-controller@11200000/1.4.2/bluetooth.0.driver
ok 6 /usb2-controller@11200000/1.4.2/bluetooth.1.driver
ok 7 /pci-controller@11230000/0.0/0.0/wifi.device
ok 8 /pci-controller@11230000/0.0/0.0/wifi.driver
Totals: pass:8 fail:0 xfail:0 xpass:0 skip:0 error:0

[3] https://lore.kernel.org/all/20230828211424.2964562-1-nfraprado@collabora.com/

Changes in v3:
- Reverted approach of encoding stable device reference in test file
from device match fields (from modalias) back to HW topology (from v1)
- Changed board file description to YAML
- Rewrote test script in python to handle YAML and support x86 platforms

Changes in v2:
- Changed approach of encoding stable device reference in test file from
HW topology to device match fields (the ones from modalias)
- Better documented test format

Nícolas F. R. A. Prado (3):
  kselftest: Add test to verify probe of devices from discoverable
    busses
  kselftest: devices: Add sample board file for google,spherion
  kselftest: devices: Add sample board file for XPS 13 9300

 tools/testing/selftests/Makefile              |   1 +
 tools/testing/selftests/devices/Makefile      |   4 +
 .../devices/boards/Dell Inc.,XPS 13 9300.yaml |  40 +++
 .../devices/boards/google,spherion.yaml       |  50 +++
 tools/testing/selftests/devices/ksft.py       |  90 +++++
 .../devices/test_discoverable_devices.py      | 318 ++++++++++++++++++
 6 files changed, 503 insertions(+)
 create mode 100644 tools/testing/selftests/devices/Makefile
 create mode 100644 tools/testing/selftests/devices/boards/Dell Inc.,XPS 13 9300.yaml
 create mode 100644 tools/testing/selftests/devices/boards/google,spherion.yaml
 create mode 100644 tools/testing/selftests/devices/ksft.py
 create mode 100755 tools/testing/selftests/devices/test_discoverable_devices.py

I have no opinion about the patches themselves, but just a heads-up
that "busses" may be regarded as a misspelling of "buses", e.g.,
https://lore.kernel.org/r/20231223184720.25645-1-tintinm2017@gmail.com,
I'm guessing because codespell complains about it.

Git grep says there are almost as many instances of "busses" as
"buses" in the kernel, so I don't go out of my way to change them.
Just FYI, doesn't matter to me either way.

Bjorn

On Tue, Jan 02, 2024 at 10:45:59AM +0300, Dan Carpenter wrote:

> Life hack: Don't put RFC in the subject.  Especially if it's a v2 or
> higher.  No one reads RFC patches.

RFC does tend to be useful in cases where you know that there are
substantial problems with the patches but are posting to solicit
feedback of some kind - otherwise people will tend to get annoyed when
they notice the problems.