diff mbox series

[5/5] test: dm: add scmi command test

Message ID 20231025051427.509602-6-takahiro.akashi@linaro.org
State Superseded
Headers show
Series cmd: add scmi command | expand

Commit Message

AKASHI Takahiro Oct. 25, 2023, 5:14 a.m. UTC 
In this test, "scmi" command is tested against different sub-commands.
Please note that scmi command is for debug purpose and is not intended
in production system.

Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>
---
v7
* make test assertions more flexible depending on the number of provided
  protocols
v4
* move 'base'-related changes to the prior commit
* add CONFIG_CMD_SCMI to sandbox_defconfig
v3
* change char to u8 in vendor/agent names
v2
* use helper functions, removing direct uses of ops
---
 configs/sandbox_defconfig |  1 +
 test/dm/scmi.c            | 81 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 82 insertions(+)

Comments

Tom Rini Nov. 10, 2023, 6:21 p.m. UTC | #1 
On Wed, Oct 25, 2023 at 02:14:27PM +0900, AKASHI Takahiro wrote:

> In this test, "scmi" command is tested against different sub-commands.
> Please note that scmi command is for debug purpose and is not intended
> in production system.
> 
> Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
> Reviewed-by: Simon Glass <sjg@chromium.org>
> Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>

The test part of this still fails:
https://source.denx.de/u-boot/u-boot/-/jobs/732077

I don't know why more output wasn't captured, when I run it locally
instead I get:
========================================== FAILURES ===========================================
_______________________________ test_ut[ut_dm_dm_test_scmi_cmd] _______________________________
test/py/u_boot_spawn.py:195: in expect
    c = os.read(self.fd, 1024).decode(errors='replace')
E   OSError: [Errno 5] Input/output error

During handling of the above exception, another exception occurred:
test/py/tests/test_ut.py:502: in test_ut
    output = u_boot_console.run_command('ut ' + ut_subtest)
test/py/u_boot_console_base.py:266: in run_command
    m = self.p.expect([self.prompt_compiled] + self.bad_patterns)
test/py/u_boot_spawn.py:204: in expect
    raise ValueError('U-Boot exited with %s' % info)
E   ValueError: U-Boot exited with signal 11 (SIGSEGV)
------------------------------------ Captured stdout call -------------------------------------
=> ut dm dm_test_scmi_cmd
Test: dm_test_scmi_cmd: scmi.c
SCMI device: scmi
  protocol version: 0x20000
  # of agents: 2
      0: platform
    > 1: OSPM
  # of protocols: 4
      Power domain management
      Clock management
      Reset domain management
      Voltage domain management
  vendor: U-Boot
  sub vendor: Sandbox
  impl version: 0x1
Denying access to device:0 failed (-13)
Denying access to protocol:0x14 on device:0 failed (-13)
Reset failed (-13)
Test: dm_test_scmi_cmd: scmi.c (flat tree)
SCMI device: Q
=================================== short test summary info ===================================
FAILED test/py/tests/test_ut.py::test_ut[ut_dm_dm_test_scmi_cmd] - ValueError: U-Boot exited...
AKASHI Takahiro Nov. 13, 2023, 1:46 a.m. UTC | #2 
Hi Tom,

On Fri, Nov 10, 2023 at 01:21:37PM -0500, Tom Rini wrote:
> On Wed, Oct 25, 2023 at 02:14:27PM +0900, AKASHI Takahiro wrote:
> 
> > In this test, "scmi" command is tested against different sub-commands.
> > Please note that scmi command is for debug purpose and is not intended
> > in production system.
> > 
> > Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
> > Reviewed-by: Simon Glass <sjg@chromium.org>
> > Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>
> 
> The test part of this still fails:
> https://source.denx.de/u-boot/u-boot/-/jobs/732077
> 
> I don't know why more output wasn't captured, when I run it locally
> instead I get:
> ========================================== FAILURES ===========================================
> _______________________________ test_ut[ut_dm_dm_test_scmi_cmd] _______________________________
> test/py/u_boot_spawn.py:195: in expect
>     c = os.read(self.fd, 1024).decode(errors='replace')
> E   OSError: [Errno 5] Input/output error
> 
> During handling of the above exception, another exception occurred:
> test/py/tests/test_ut.py:502: in test_ut
>     output = u_boot_console.run_command('ut ' + ut_subtest)
> test/py/u_boot_console_base.py:266: in run_command
>     m = self.p.expect([self.prompt_compiled] + self.bad_patterns)
> test/py/u_boot_spawn.py:204: in expect
>     raise ValueError('U-Boot exited with %s' % info)
> E   ValueError: U-Boot exited with signal 11 (SIGSEGV)


The command uses global variables which hold pointers to 'struct udevice'
which are to be shared between the main and the sub-commands.
Since pytest framework executes ut tests twice, once with a (normal?) device
tree and once with a flat tree,  udevices will be *voided* between
two executions.

I will fix it in v2.

Thanks,
-Takahiro Akashi


> ------------------------------------ Captured stdout call -------------------------------------
> => ut dm dm_test_scmi_cmd
> Test: dm_test_scmi_cmd: scmi.c
> SCMI device: scmi
>   protocol version: 0x20000
>   # of agents: 2
>       0: platform
>     > 1: OSPM
>   # of protocols: 4
>       Power domain management
>       Clock management
>       Reset domain management
>       Voltage domain management
>   vendor: U-Boot
>   sub vendor: Sandbox
>   impl version: 0x1
> Denying access to device:0 failed (-13)
> Denying access to protocol:0x14 on device:0 failed (-13)
> Reset failed (-13)
> Test: dm_test_scmi_cmd: scmi.c (flat tree)
> SCMI device: Q
> =================================== short test summary info ===================================
> FAILED test/py/tests/test_ut.py::test_ut[ut_dm_dm_test_scmi_cmd] - ValueError: U-Boot exited...
> 
> -- 
> Tom
Simon Glass Nov. 13, 2023, 6:01 p.m. UTC | #3 
Hi,

On Sun, 12 Nov 2023 at 18:46, AKASHI Takahiro
<takahiro.akashi@linaro.org> wrote:
>
> Hi Tom,
>
> On Fri, Nov 10, 2023 at 01:21:37PM -0500, Tom Rini wrote:
> > On Wed, Oct 25, 2023 at 02:14:27PM +0900, AKASHI Takahiro wrote:
> >
> > > In this test, "scmi" command is tested against different sub-commands.
> > > Please note that scmi command is for debug purpose and is not intended
> > > in production system.
> > >
> > > Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
> > > Reviewed-by: Simon Glass <sjg@chromium.org>
> > > Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>
> >
> > The test part of this still fails:
> > https://source.denx.de/u-boot/u-boot/-/jobs/732077
> >
> > I don't know why more output wasn't captured, when I run it locally
> > instead I get:
> > ========================================== FAILURES ===========================================
> > _______________________________ test_ut[ut_dm_dm_test_scmi_cmd] _______________________________
> > test/py/u_boot_spawn.py:195: in expect
> >     c = os.read(self.fd, 1024).decode(errors='replace')
> > E   OSError: [Errno 5] Input/output error
> >
> > During handling of the above exception, another exception occurred:
> > test/py/tests/test_ut.py:502: in test_ut
> >     output = u_boot_console.run_command('ut ' + ut_subtest)
> > test/py/u_boot_console_base.py:266: in run_command
> >     m = self.p.expect([self.prompt_compiled] + self.bad_patterns)
> > test/py/u_boot_spawn.py:204: in expect
> >     raise ValueError('U-Boot exited with %s' % info)
> > E   ValueError: U-Boot exited with signal 11 (SIGSEGV)
>
>
> The command uses global variables which hold pointers to 'struct udevice'
> which are to be shared between the main and the sub-commands.
> Since pytest framework executes ut tests twice, once with a (normal?) device
> tree and once with a flat tree,  udevices will be *voided* between
> two executions.

Are you able to put the var in the uclass-priv data instead? The state
should be cleared before running each DM test.

Regards,
Simon


>
> I will fix it in v2.
>
> Thanks,
> -Takahiro Akashi
>
>
> > ------------------------------------ Captured stdout call -------------------------------------
> > => ut dm dm_test_scmi_cmd
> > Test: dm_test_scmi_cmd: scmi.c
> > SCMI device: scmi
> >   protocol version: 0x20000
> >   # of agents: 2
> >       0: platform
> >     > 1: OSPM
> >   # of protocols: 4
> >       Power domain management
> >       Clock management
> >       Reset domain management
> >       Voltage domain management
> >   vendor: U-Boot
> >   sub vendor: Sandbox
> >   impl version: 0x1
> > Denying access to device:0 failed (-13)
> > Denying access to protocol:0x14 on device:0 failed (-13)
> > Reset failed (-13)
> > Test: dm_test_scmi_cmd: scmi.c (flat tree)
> > SCMI device: Q
> > =================================== short test summary info ===================================
> > FAILED test/py/tests/test_ut.py::test_ut[ut_dm_dm_test_scmi_cmd] - ValueError: U-Boot exited...
> >
> > --
> > Tom
>
>
AKASHI Takahiro Nov. 14, 2023, 1:41 a.m. UTC | #4 
On Mon, Nov 13, 2023 at 11:01:17AM -0700, Simon Glass wrote:
> Hi,
> 
> On Sun, 12 Nov 2023 at 18:46, AKASHI Takahiro
> <takahiro.akashi@linaro.org> wrote:
> >
> > Hi Tom,
> >
> > On Fri, Nov 10, 2023 at 01:21:37PM -0500, Tom Rini wrote:
> > > On Wed, Oct 25, 2023 at 02:14:27PM +0900, AKASHI Takahiro wrote:
> > >
> > > > In this test, "scmi" command is tested against different sub-commands.
> > > > Please note that scmi command is for debug purpose and is not intended
> > > > in production system.
> > > >
> > > > Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
> > > > Reviewed-by: Simon Glass <sjg@chromium.org>
> > > > Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>
> > >
> > > The test part of this still fails:
> > > https://source.denx.de/u-boot/u-boot/-/jobs/732077
> > >
> > > I don't know why more output wasn't captured, when I run it locally
> > > instead I get:
> > > ========================================== FAILURES ===========================================
> > > _______________________________ test_ut[ut_dm_dm_test_scmi_cmd] _______________________________
> > > test/py/u_boot_spawn.py:195: in expect
> > >     c = os.read(self.fd, 1024).decode(errors='replace')
> > > E   OSError: [Errno 5] Input/output error
> > >
> > > During handling of the above exception, another exception occurred:
> > > test/py/tests/test_ut.py:502: in test_ut
> > >     output = u_boot_console.run_command('ut ' + ut_subtest)
> > > test/py/u_boot_console_base.py:266: in run_command
> > >     m = self.p.expect([self.prompt_compiled] + self.bad_patterns)
> > > test/py/u_boot_spawn.py:204: in expect
> > >     raise ValueError('U-Boot exited with %s' % info)
> > > E   ValueError: U-Boot exited with signal 11 (SIGSEGV)
> >
> >
> > The command uses global variables which hold pointers to 'struct udevice'
> > which are to be shared between the main and the sub-commands.
> > Since pytest framework executes ut tests twice, once with a (normal?) device
> > tree and once with a flat tree,  udevices will be *voided* between
> > two executions.
> 
> Are you able to put the var in the uclass-priv data instead? The state
> should be cleared before running each DM test.

Well, I don't think we need such a trick.
As you can see, we may simply fetch/find necessary udevices
every time the command is called.
It is enough given that the command is mainly for debug purpose.

-Takahiro Akashi


> Regards,
> Simon
> 
> 
> >
> > I will fix it in v2.
> >
> > Thanks,
> > -Takahiro Akashi
> >
> >
> > > ------------------------------------ Captured stdout call -------------------------------------
> > > => ut dm dm_test_scmi_cmd
> > > Test: dm_test_scmi_cmd: scmi.c
> > > SCMI device: scmi
> > >   protocol version: 0x20000
> > >   # of agents: 2
> > >       0: platform
> > >     > 1: OSPM
> > >   # of protocols: 4
> > >       Power domain management
> > >       Clock management
> > >       Reset domain management
> > >       Voltage domain management
> > >   vendor: U-Boot
> > >   sub vendor: Sandbox
> > >   impl version: 0x1
> > > Denying access to device:0 failed (-13)
> > > Denying access to protocol:0x14 on device:0 failed (-13)
> > > Reset failed (-13)
> > > Test: dm_test_scmi_cmd: scmi.c (flat tree)
> > > SCMI device: Q
> > > =================================== short test summary info ===================================
> > > FAILED test/py/tests/test_ut.py::test_ut[ut_dm_dm_test_scmi_cmd] - ValueError: U-Boot exited...
> > >
> > > --
> > > Tom
> >
> >
Simon Glass Nov. 14, 2023, 11:29 p.m. UTC | #5 
Hi,

On Mon, 13 Nov 2023 at 18:41, AKASHI Takahiro
<takahiro.akashi@linaro.org> wrote:
>
> On Mon, Nov 13, 2023 at 11:01:17AM -0700, Simon Glass wrote:
> > Hi,
> >
> > On Sun, 12 Nov 2023 at 18:46, AKASHI Takahiro
> > <takahiro.akashi@linaro.org> wrote:
> > >
> > > Hi Tom,
> > >
> > > On Fri, Nov 10, 2023 at 01:21:37PM -0500, Tom Rini wrote:
> > > > On Wed, Oct 25, 2023 at 02:14:27PM +0900, AKASHI Takahiro wrote:
> > > >
> > > > > In this test, "scmi" command is tested against different sub-commands.
> > > > > Please note that scmi command is for debug purpose and is not intended
> > > > > in production system.
> > > > >
> > > > > Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
> > > > > Reviewed-by: Simon Glass <sjg@chromium.org>
> > > > > Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>
> > > >
> > > > The test part of this still fails:
> > > > https://source.denx.de/u-boot/u-boot/-/jobs/732077
> > > >
> > > > I don't know why more output wasn't captured, when I run it locally
> > > > instead I get:
> > > > ========================================== FAILURES ===========================================
> > > > _______________________________ test_ut[ut_dm_dm_test_scmi_cmd] _______________________________
> > > > test/py/u_boot_spawn.py:195: in expect
> > > >     c = os.read(self.fd, 1024).decode(errors='replace')
> > > > E   OSError: [Errno 5] Input/output error
> > > >
> > > > During handling of the above exception, another exception occurred:
> > > > test/py/tests/test_ut.py:502: in test_ut
> > > >     output = u_boot_console.run_command('ut ' + ut_subtest)
> > > > test/py/u_boot_console_base.py:266: in run_command
> > > >     m = self.p.expect([self.prompt_compiled] + self.bad_patterns)
> > > > test/py/u_boot_spawn.py:204: in expect
> > > >     raise ValueError('U-Boot exited with %s' % info)
> > > > E   ValueError: U-Boot exited with signal 11 (SIGSEGV)
> > >
> > >
> > > The command uses global variables which hold pointers to 'struct udevice'
> > > which are to be shared between the main and the sub-commands.
> > > Since pytest framework executes ut tests twice, once with a (normal?) device
> > > tree and once with a flat tree,  udevices will be *voided* between
> > > two executions.
> >
> > Are you able to put the var in the uclass-priv data instead? The state
> > should be cleared before running each DM test.
>
> Well, I don't think we need such a trick.
> As you can see, we may simply fetch/find necessary udevices
> every time the command is called.
> It is enough given that the command is mainly for debug purpose.

OK. Let's see how it goes. When you mention global vars I get a bit
nervous, but we do have these in some places.

Regards,
Simon


>
> -Takahiro Akashi
>
>
> > Regards,
> > Simon
> >
> >
> > >
> > > I will fix it in v2.
> > >
> > > Thanks,
> > > -Takahiro Akashi
> > >
> > >
> > > > ------------------------------------ Captured stdout call -------------------------------------
> > > > => ut dm dm_test_scmi_cmd
> > > > Test: dm_test_scmi_cmd: scmi.c
> > > > SCMI device: scmi
> > > >   protocol version: 0x20000
> > > >   # of agents: 2
> > > >       0: platform
> > > >     > 1: OSPM
> > > >   # of protocols: 4
> > > >       Power domain management
> > > >       Clock management
> > > >       Reset domain management
> > > >       Voltage domain management
> > > >   vendor: U-Boot
> > > >   sub vendor: Sandbox
> > > >   impl version: 0x1
> > > > Denying access to device:0 failed (-13)
> > > > Denying access to protocol:0x14 on device:0 failed (-13)
> > > > Reset failed (-13)
> > > > Test: dm_test_scmi_cmd: scmi.c (flat tree)
> > > > SCMI device: Q
> > > > =================================== short test summary info ===================================
> > > > FAILED test/py/tests/test_ut.py::test_ut[ut_dm_dm_test_scmi_cmd] - ValueError: U-Boot exited...
> > > >
> > > > --
> > > > Tom
> > >
> > >
diff mbox series

Patch

diff --git a/configs/sandbox_defconfig b/configs/sandbox_defconfig
index e3a2f9eb1708..34e3cc281d7f 100644
--- a/configs/sandbox_defconfig
+++ b/configs/sandbox_defconfig
@@ -120,6 +120,7 @@  CONFIG_CMD_REGULATOR=y
 CONFIG_CMD_AES=y
 CONFIG_CMD_TPM=y
 CONFIG_CMD_TPM_TEST=y
+CONFIG_CMD_SCMI=y
 CONFIG_CMD_BTRFS=y
 CONFIG_CMD_CBFS=y
 CONFIG_CMD_CRAMFS=y
diff --git a/test/dm/scmi.c b/test/dm/scmi.c
index 2f63f2da16fb..2bcf7ac6fcc3 100644
--- a/test/dm/scmi.c
+++ b/test/dm/scmi.c
@@ -19,6 +19,7 @@ 
 #include <scmi_agent.h>
 #include <scmi_agent-uclass.h>
 #include <scmi_protocols.h>
+#include <vsprintf.h>
 #include <asm/scmi_test.h>
 #include <dm/device-internal.h>
 #include <dm/test.h>
@@ -206,6 +207,86 @@  static int dm_test_scmi_base(struct unit_test_state *uts)
 
 DM_TEST(dm_test_scmi_base, UT_TESTF_SCAN_FDT);
 
+static int dm_test_scmi_cmd(struct unit_test_state *uts)
+{
+	struct udevice *agent_dev;
+	int num_proto = 0;
+	char cmd_out[30];
+
+	if (!IS_ENABLED(CONFIG_CMD_SCMI))
+		return 0;
+
+	/* preparation */
+	ut_assertok(uclass_get_device_by_name(UCLASS_SCMI_AGENT, "scmi",
+					      &agent_dev));
+	ut_assertnonnull(agent_dev);
+
+	/*
+	 * Estimate the number of provided protocols.
+	 * This estimation is correct as far as a corresponding
+	 * protocol support is added to sandbox fake serer.
+	 */
+	if (IS_ENABLED(CONFIG_SCMI_POWER_DOMAIN))
+		num_proto++;
+	if (IS_ENABLED(CONFIG_CLK_SCMI))
+		num_proto++;
+	if (IS_ENABLED(CONFIG_RESET_SCMI))
+		num_proto++;
+	if (IS_ENABLED(CONFIG_DM_REGULATOR_SCMI))
+		num_proto++;
+
+	/* scmi info */
+	ut_assertok(run_command("scmi info", 0));
+
+	ut_assert_nextline("SCMI device: scmi");
+	snprintf(cmd_out, 30, "  protocol version: 0x%x",
+		 SCMI_BASE_PROTOCOL_VERSION);
+	ut_assert_nextline(cmd_out);
+	ut_assert_nextline("  # of agents: 2");
+	ut_assert_nextline("      0: platform");
+	ut_assert_nextline("    > 1: OSPM");
+	snprintf(cmd_out, 30, "  # of protocols: %d", num_proto);
+	ut_assert_nextline(cmd_out);
+	if (IS_ENABLED(CONFIG_SCMI_POWER_DOMAIN))
+		ut_assert_nextline("      Power domain management");
+	if (IS_ENABLED(CONFIG_CLK_SCMI))
+		ut_assert_nextline("      Clock management");
+	if (IS_ENABLED(CONFIG_RESET_SCMI))
+		ut_assert_nextline("      Reset domain management");
+	if (IS_ENABLED(CONFIG_DM_REGULATOR_SCMI))
+		ut_assert_nextline("      Voltage domain management");
+	ut_assert_nextline("  vendor: U-Boot");
+	ut_assert_nextline("  sub vendor: Sandbox");
+	ut_assert_nextline("  impl version: 0x1");
+
+	ut_assert_console_end();
+
+	/* scmi perm_dev */
+	ut_assertok(run_command("scmi perm_dev 1 0 1", 0));
+	ut_assert_console_end();
+
+	ut_assert(run_command("scmi perm_dev 1 0 0", 0));
+	ut_assert_nextline("Denying access to device:0 failed (-13)");
+	ut_assert_console_end();
+
+	/* scmi perm_proto */
+	ut_assertok(run_command("scmi perm_proto 1 0 14 1", 0));
+	ut_assert_console_end();
+
+	ut_assert(run_command("scmi perm_proto 1 0 14 0", 0));
+	ut_assert_nextline("Denying access to protocol:0x14 on device:0 failed (-13)");
+	ut_assert_console_end();
+
+	/* scmi reset */
+	ut_assert(run_command("scmi reset 1 1", 0));
+	ut_assert_nextline("Reset failed (-13)");
+	ut_assert_console_end();
+
+	return 0;
+}
+
+DM_TEST(dm_test_scmi_cmd, UT_TESTF_SCAN_FDT);
+
 static int dm_test_scmi_power_domains(struct unit_test_state *uts)
 {
 	struct sandbox_scmi_agent *agent;