From patchwork Thu Jul 25 16:15:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anubhav Shelat X-Patchwork-Id: 815208 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2FAE419D083 for ; Thu, 25 Jul 2024 16:15:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721924129; cv=none; b=nh/iI+/MnTyffjFZ4pdxQYFwP/a/0qWucvk/3uP5zOPZeBWFRn7YMGoHvUdIatbN8QqLufO4ijUBNBSxiJGfK6pFy0C9mdKMk+HDWJyu6yM0tgnNb6KtTRT/8LEvpclwrif173NFuHOqoyvg1BCv3SsQxZxtdKY/g+EmmItrgcs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721924129; c=relaxed/simple; bh=Eq9FpcCqWsywQC2XpOyQQ2bvyVctKoyiC4HKmi5ycBY=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=S3s69nokebJGUPumC8G5KpetYJakkXex3fHDYry/89Z67VS89hkpEdWzxsYwzTFj+eloG68WKSPuohdPI5N9eoUxlRDZPbkwYcdkbujXEtgsBcUVzRQLa8uMKjK5Y3Sua+i7yZTRzMgl1aPzBaI9FJbHpTEyKL7b4Z8T3FobCXA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=euSc3P3U; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="euSc3P3U" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1721924127; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=VxVP0abJiLWgN7QsEfm0EhLJyI00GA3Sbj267qzvm2I=; b=euSc3P3UAyQh8sB43GlV6oHARpBADa4L/n4kXz6VV+/7w3tvdl/g5XoetCpjj+DBLoMuIj FocXM+F16VKqCnJiBIOwzEr3szbKVhIebyUG2HqYFSsJLBXurFX1jL4CLI8GZaRbzvp2lQ asFh05uhhXG3DbPmj2LYE5oKHhBAssE= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-29-eC_o4iwvM5aLvpp6xLfuLg-1; Thu, 25 Jul 2024 12:15:23 -0400 X-MC-Unique: eC_o4iwvM5aLvpp6xLfuLg-1 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7F0C019560B6 for ; Thu, 25 Jul 2024 16:15:22 +0000 (UTC) Received: from ashelat-na.remote.csb (unknown [10.22.64.15]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 56AEF3000194; Thu, 25 Jul 2024 16:15:21 +0000 (UTC) From: Anubhav Shelat To: linux-rt-users@vger.kernel.org Cc: jkacur@redhat.com, kcarcia@redhat.com, Anubhav Shelat Subject: [PATCH v5 1/2] rteval: Added functionality to allow user to set the cstate of specified cpus when running rteval Date: Thu, 25 Jul 2024 12:15:13 -0400 Message-ID: <20240725161514.1885152-1-ashelat@redhat.com> Precedence: bulk X-Mailing-List: linux-rt-users@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 We would like to be able to set the idle states of CPUs while running rteval. This patch adds the file cpupower.py and option '--idle-set' within rteval to use cpupower. The set idle state is applied to the cpulist given by '--measurement-cpulist'. cpupower.py provides the infrastructure to interface and execute the cpupower command, and the options in rteval-cmd let the user specify the idle state to be set and the CPUs to set it on. Signed-off-by: Anubhav Shelat --- rteval-cmd | 22 ++++++++- rteval/cpupower.py | 94 ++++++++++++++++++++++++++++++++++++++ rteval/modules/__init__.py | 2 +- 3 files changed, 116 insertions(+), 2 deletions(-) create mode 100644 rteval/cpupower.py diff --git a/rteval-cmd b/rteval-cmd index 19c82a0b64b3..9d87328302df 100755 --- a/rteval-cmd +++ b/rteval-cmd @@ -29,6 +29,7 @@ from rteval.Log import Log from rteval import RtEval, rtevalConfig from rteval.modules.loads import LoadModules from rteval.modules.measurement import MeasurementModules +from rteval import cpupower from rteval.version import RTEVAL_VERSION from rteval.systopology import SysTopology, parse_cpulist_from_config from rteval.modules.loads.kcompile import ModuleParameters @@ -149,7 +150,8 @@ def parse_options(cfg, parser, cmdargs): parser.add_argument("--noload", dest="rteval___noload", action="store_true", default=False, help="only run the measurements (don't run loads)") - + parser.add_argument('--idle-set', dest='rteval___idlestate', metavar='IDLESTATE', + default=None, help='Idle state to enable for this rteval run') if not cmdargs: cmdargs = ["--help"] @@ -172,6 +174,12 @@ def parse_options(cfg, parser, cmdargs): cmd_args = cmdargs[ind+1:] cmdargs = cmdargs[:ind+1] + # if --cpulist is specified, make sure --idle-set is too + if ((sys.argv.count('-c')+sys.argv.count('--cpulist')) > 0) and \ + (sys.argv.count('--idle-set') == 0): + print('--cpulist must be used with --idle-set') + cmdargs = ["--help"] + cmd_opts = parser.parse_args(args=cmdargs) # if no kernel version was provided for --source-download, set version to default @@ -265,6 +273,9 @@ if __name__ == '__main__': | (rtevcfg.debugging and Log.DEBUG) logger.SetLogVerbosity(loglev) + # check if cpupower is being used + if sys.argv.count('--idle-set') > 0: + rtevcfg.update({'usingCpupower': True}) # Load modules loadmods = LoadModules(config, logger=logger) measuremods = MeasurementModules(config, logger=logger) @@ -402,6 +413,11 @@ if __name__ == '__main__': if not os.path.isdir(rtevcfg.workdir): raise RuntimeError(f"work directory {rtevcfg.workdir} does not exist") + # if idle-set has been specified, enable idlestate in rtecfg.cpulist + cpupower_controller = None + if rtevcfg.idlestate: + cpupower_controller = cpupower.Cpupower(msrcfg.cpulist, rtevcfg.idlestate) + cpupower_controller.enable_idle_state() rteval = RtEval(config, loadmods, measuremods, logger) rteval.Prepare(rtevcfg.onlyload) @@ -421,6 +437,10 @@ if __name__ == '__main__': ec = rteval.Measure() logger.log(Log.DEBUG, f"exiting with exit code: {ec}") + # reset idlestate + if rtevcfg.idlestate: + cpupower_controller.restore_cstate() + sys.exit(ec) except KeyboardInterrupt: sys.exit(0) diff --git a/rteval/cpupower.py b/rteval/cpupower.py new file mode 100644 index 000000000000..9437b2451b54 --- /dev/null +++ b/rteval/cpupower.py @@ -0,0 +1,94 @@ +#! /user/bin/python3 +# SPDX-License-Identifier: GPL-2.0-or-later + +# Object to execute cpupower tool +# Copyright 2024 Anubhav Shelat + +import subprocess +import argparse +import os +import multiprocessing +import time + + +class Cpupower: + def __init__(self, cpulist, idlestate): + self.cpulist = cpulist + self.idlestate = idlestate + self.nstates = len(os.listdir('/sys/devices/system/cpu/cpu0/cpuidle/')) # number of cstates + self.cpu_count = multiprocessing.cpu_count() + self.cstate_cfg = [] + + + def enable_idle_state(self): + ''' Enable a specific cstate, while disabling all other cstates, and save the current cstate configuration ''' + self.cstate_cfg = self.get_cstate_cfg() + + # ensure that cstate is in range of available cstates + if int(self.idlestate) > self.nstates-1 or int(self.idlestate) < 0: + print(f'Idle state {self.idlestate} is out of range, using POLL') + self.idlestate = 0 + + # enable cstate and disable the rest + if (self.cpulist): + subprocess.run(['sudo', 'cpupower', '-c', self.cpulist,'idle-set', '-e', str(self.idlestate)], stdout=open(os.devnull, 'wb')) + for cs in range(self.nstates): + if str(cs) != self.idlestate: + subprocess.run(['sudo', 'cpupower', '-c', self.cpulist,'idle-set', '-d', str(cs)], stdout=open(os.devnull, 'wb')) + else: + subprocess.run(['sudo', 'cpupower', 'idle-set', '-e', str(self.idlestate)], stdout=open(os.devnull, 'wb')) + for cs in range(self.nstates): + if str(cs) != self.idlestate: + subprocess.run(['sudo', 'cpupower', 'idle-set', '-d', str(cs)], stdout=open(os.devnull, 'wb')) + + if self.cpulist: print(f'Idlestate {self.idlestate} enabled on CPUs {self.cpulist}') + else: print(f'Idlestate {self.idlestate} enabled on all CPUs') + + + def get_cstate_cfg(self): + ''' Store the current cstate config ''' + # cstate [m] configuration on cpu [n] can be found in '/sys/devices/system/cpu/cpu*/cpuidle/state*/disable' + cfg = [] + for cpu in range(self.cpu_count): + cfg.append([]) + for cs in range(self.nstates): + f = open('/sys/devices/system/cpu/cpu'+str(cpu)+'/cpuidle/state'+str(cs)+'/disable', 'r') + d = f.read(1) + cfg[cpu].append(d) # cstate_cfg[n][m] stores the m-th idle state on the n-th cpu + + return cfg + + + def restore_cstate(self): + for cpu in range(self.cpu_count): + for cs in range(self.nstates): + f = open('/sys/devices/system/cpu/cpu'+str(cpu)+'/cpuidle/state'+str(cs)+'/disable', 'w') + f.write(self.cstate_cfg[cpu][cs]) + f.close() + print('Idle state configuration restored') + + def get_idle_info(): + subprocess.call(['cpupower', 'idle-info']) + + +if __name__ == '__main__': + parser = argparse.ArgumentParser() + parser.add_argument('-c', '--cpu-list', required=False, default=None, + help='List of cpus to perform cpupower-idle-set operation on.') + parser.add_argument('-s', '--idle-set', default='', + help='Specify idle state to enable/disable') + parser.add_argument('--info', default=False, action='store_true', help='Get idle state information') + + args = parser.parse_args() + print(args) + cpulist = args.cpu_list + idlestate = args.cstate + if (idlestate): + cpupower = Cpupower(cpulist, idlestate) + cpupower.enable_idle_state() + time.sleep(10) + cpupower.restore_cstate() + elif (args.info): + Cpupower.get_idle_info() + + diff --git a/rteval/modules/__init__.py b/rteval/modules/__init__.py index 2a4eafae71c7..10cbd50ddfd8 100644 --- a/rteval/modules/__init__.py +++ b/rteval/modules/__init__.py @@ -282,7 +282,7 @@ reference from the first import""" grparser.add_argument(f'--{self.__modtype}-cpulist', dest=f'{self.__modtype}___cpulist', action='store', default="", help=f'CPU list where {self.__modtype} modules will run', - metavar='LIST') + metavar='CPULIST') for (modname, mod) in list(self.__modsloaded.items()): opts = mod.ModuleParameters() From patchwork Thu Jul 25 16:15:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anubhav Shelat X-Patchwork-Id: 814547 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D763617588 for ; Thu, 25 Jul 2024 16:15:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721924133; cv=none; b=gfEdJ8tEse2V0V1nUCaWmBHZuz2oNs7/uvhUV42WBz9BiRC71kVw3HZ5j7/UzNizOt69bZCLdekrL0ak1KpQRWlR4hUFZgtspXZBCaMnD167pVHTPJMQGVWna2tihsBOEoVYaA/ZRLsVlVwkkQdRvylu7TnTZD2NWpwmoctIrXg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721924133; c=relaxed/simple; bh=FqbMg3jrEOC0vibPNNAf4SR0DXcQYPq0GbtJENkBztk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=X0y1+viYH/rCyb5+MAyj0BRYjM7jElxiTy5IvcDIo7TY4YhsYEe6mYob8DJQeqhE/9UJKeWxfWdgFLvQ3dA/ZIBGsfKLz6V3U9F0V/BREk8HcBCWv00V5LY8P1EpR7D/tXg0urOcxHkwJOLSbz3wp9Yqoo2l/7SaT7/HrF515ns= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=fJgM8Eo7; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="fJgM8Eo7" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1721924130; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BWjJHRsgAiKSPGqFB1WmYYRMMJz4kuMq97PTFgBcxkk=; b=fJgM8Eo7T2qwdh4bGgc17caYd5FCKdLZrAbzLLleXHHifbOzpwWhYv6tYEwlapOdLW7Xmr 6oezVdEFDjM4PR/73Nr4w6TdWq9lFz8Db7Jt8Dqp+6zZRFFmt2tUw9gZV1Zsl/N8k8YGc1 slurgJepffVaJSsbaUDfODm1fjoysAo= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-171-HnzcKfogNIWcfpjYeE4FWQ-1; Thu, 25 Jul 2024 12:15:24 -0400 X-MC-Unique: HnzcKfogNIWcfpjYeE4FWQ-1 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E7BEC1955D4C for ; Thu, 25 Jul 2024 16:15:23 +0000 (UTC) Received: from ashelat-na.remote.csb (unknown [10.22.64.15]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id A65ED3000194; Thu, 25 Jul 2024 16:15:22 +0000 (UTC) From: Anubhav Shelat To: linux-rt-users@vger.kernel.org Cc: jkacur@redhat.com, kcarcia@redhat.com, Anubhav Shelat Subject: [PATCH v5 2/2] rteval: run cyclictest using '--default-system' when setting idle states Date: Thu, 25 Jul 2024 12:15:14 -0400 Message-ID: <20240725161514.1885152-2-ashelat@redhat.com> In-Reply-To: <20240725161514.1885152-1-ashelat@redhat.com> References: <20240725161514.1885152-1-ashelat@redhat.com> Precedence: bulk X-Mailing-List: linux-rt-users@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 When running cyclictest in rteval, cyclictest automatically disables idle states. This means whenever the user sets the idle state of a cpu list using '--idle-set' it is overridden by cyclictest. To fix this, the variable 'usingCpupower' is appended to the parameter dictionary that's passed to the Cyclictest measurement object which executes cyclictest in rteval. If '--idle-set' is specified when running rteval, 'usingCpupower' is set to true and the '--default-system' option is appended to the cyclictest command, which will prevent cyclictest from disabling cstates. Signed-off-by: Anubhav Shelat --- rteval/__init__.py | 1 + rteval/modules/measurement/cyclictest.py | 3 +++ 2 files changed, 4 insertions(+) diff --git a/rteval/__init__.py b/rteval/__init__.py index 4d3e0c23e5ab..8ded374d287e 100644 --- a/rteval/__init__.py +++ b/rteval/__init__.py @@ -119,6 +119,7 @@ class RtEval(rtevalReport): 'memsize':self._sysinfo.mem_get_size(), 'numanodes':self._sysinfo.mem_get_numa_nodes(), 'duration': float(self.__rtevcfg.duration), + 'usingCpupower': self.__rtevcfg.usingCpupower } if self._loadmods: diff --git a/rteval/modules/measurement/cyclictest.py b/rteval/modules/measurement/cyclictest.py index a9f5b0c4fba7..fffc8fe27681 100644 --- a/rteval/modules/measurement/cyclictest.py +++ b/rteval/modules/measurement/cyclictest.py @@ -251,6 +251,9 @@ class Cyclictest(rtevalModulePrototype): self.__cmd.append(f'-t{self.__numcores}') self.__cmd.append(f'-a{self.__cpulist}') + if (self.__cfg.usingCpupower): + self.__cmd.append('--default-system') + if 'threads' in self.__cfg and self.__cfg.threads: self.__cmd.append(f"-t{int(self.__cfg.threads)}")