From patchwork Tue Jan 29 10:33:12 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Senthil Kumaran X-Patchwork-Id: 14330 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id BC34223EAD for ; Tue, 29 Jan 2013 10:33:19 +0000 (UTC) Received: from mail-vc0-f180.google.com (mail-vc0-f180.google.com [209.85.220.180]) by fiordland.canonical.com (Postfix) with ESMTP id 496A5A1825F for ; Tue, 29 Jan 2013 10:33:19 +0000 (UTC) Received: by mail-vc0-f180.google.com with SMTP id fo13so159603vcb.25 for ; Tue, 29 Jan 2013 02:33:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:x-forwarded-to:x-forwarded-for:delivered-to:x-received :received-spf:content-type:mime-version:x-launchpad-project :x-launchpad-branch:x-launchpad-message-rationale :x-launchpad-branch-revision-number:x-launchpad-notification-type:to :from:subject:message-id:date:reply-to:sender:errors-to:precedence :x-generated-by:x-launchpad-hash:x-gm-message-state; bh=X6yLc0mjeBRk6RZnvYBOAFiT1N1UgJCesJsx3ZcMfss=; b=cZpLjUuDrTkbfWE6HRl4DxMCQOnVU46xnuhtmigZSiclsLTjDIDy9CvN7rKEaLGChi 3GKAQLcto+Uptiswk29YC55clR/lQZc5nH0tL+IU9dxvGHIp76Bst+TNidkS2mPGjLS3 ajj4FH9Q0oEPH/Tr0uzjY0XEgFI0OtiqPM3VBrBUvZCEZqmNPoRrcOnZ64oG0wuV6JMB OazwlvJ5P3JCY+/RJH3ITqfW0BGV7Wiq2nENQfP5JjLWXa+Bkwq3AwG+21Wm3u9v0gLd 6oYFlxL/rAoSn9BIECkvXE/4d+osdDRAbf63gzfUR+60kRZM2cERHXc+/7w8+j8CZJHu XFEA== X-Received: by 10.52.18.148 with SMTP id w20mr403543vdd.8.1359455598692; Tue, 29 Jan 2013 02:33:18 -0800 (PST) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.58.145.101 with SMTP id st5csp128847veb; Tue, 29 Jan 2013 02:33:17 -0800 (PST) X-Received: by 10.66.89.132 with SMTP id bo4mr1403412pab.62.1359455596828; Tue, 29 Jan 2013 02:33:16 -0800 (PST) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id wk7si12060030pbc.139.2013.01.29.02.33.14 (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 29 Jan 2013 02:33:16 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1U08V6-00082j-UQ for ; Tue, 29 Jan 2013 10:33:12 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id D4965E0025 for ; Tue, 29 Jan 2013 10:33:12 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-dispatcher X-Launchpad-Branch: ~linaro-validation/lava-dispatcher/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 548 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-dispatcher/trunk] Rev 548: Merge add-on-to-testdef-yaml branch which adds additional metadata parameters Message-Id: <20130129103312.11798.50225.launchpad@ackee.canonical.com> Date: Tue, 29 Jan 2013 10:33:12 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="16451"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: 58589d62999c515a99f3ecea037f64af701fa08b X-Gm-Message-State: ALoCoQmDvHDWp3UzkRdX077vSlZYSHk3S/P1GOgCKjXNtQKfFc0UfNlhq2+X5am/JgIhR0KWgDO0 Merge authors: Senthil Kumaran S (stylesen) Related merge proposals: https://code.launchpad.net/~stylesen/lava-dispatcher/add-on-to-testdef-yaml/+merge/144214 proposed by: Senthil Kumaran S (stylesen) ------------------------------------------------------------ revno: 548 [merge] committer: Senthil Kumaran branch nick: trunk timestamp: Tue 2013-01-29 16:01:10 +0530 message: Merge add-on-to-testdef-yaml branch which adds additional metadata parameters to test definition YAML format. modified: lava_dispatcher/actions/lava_test_shell.py lava_dispatcher/lava_test_shell.py lava_dispatcher/test_data.py lava_test_shell/lava-test-runner-android lava_test_shell/lava-test-runner-ubuntu --- lp:lava-dispatcher https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk You are subscribed to branch lp:lava-dispatcher. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk/+edit-subscription === modified file 'lava_dispatcher/actions/lava_test_shell.py' --- lava_dispatcher/actions/lava_test_shell.py 2013-01-15 17:23:13 +0000 +++ lava_dispatcher/actions/lava_test_shell.py 2013-01-28 08:51:59 +0000 @@ -53,6 +53,7 @@ # uuid The "analyzer_assigned_uuid" of the # test_run that is being generated. # testdef.yml The test definition. +# testdef_metadata Metadata extracted from test definition. # install.sh The install steps. # run.sh The run steps. # [repos] The test definition can specify bzr or git @@ -74,6 +75,7 @@ # pkgs.txt Ditto # ${IDX}_${TEST_ID}-${TIMESTAMP}/ # testdef.yml +# testdef_metadata # stdout.log # return_code The exit code of run.sh. # analyzer_assigned_uuid @@ -173,8 +175,7 @@ cwd = os.getcwd() gitdir = os.path.join(tmpdir, 'gittestrepo') try: - subprocess.check_call(['git', 'clone', testdef_repo, - gitdir]) + subprocess.check_call(['git', 'clone', testdef_repo, gitdir]) if revision: os.chdir(gitdir) subprocess.check_call(['git', 'checkout', revision]) @@ -201,6 +202,26 @@ logging.error('Unable to get test definition from bzr\n' + str(e)) +def _get_testdef_info(testdef): + metadata = {'os': '', 'devices': '', 'environment': ''} + metadata['version'] = str(testdef['metadata'].get('version')) + metadata['description'] = str(testdef['metadata'].get('description')) + metadata['format'] = str(testdef['metadata'].get('format')) + + # Convert list to comma separated string. + if testdef['metadata'].get('os'): + metadata['os'] = ','.join(testdef['metadata'].get('os')) + + if testdef['metadata'].get('devices'): + metadata['devices'] = ','.join(testdef['metadata'].get('devices')) + + if testdef['metadata'].get('environment'): + metadata['environment'] = ','.join( + testdef['metadata'].get('environment')) + + return metadata + + class TestDefinitionLoader(object): """ A TestDefinitionLoader knows how to load test definitions from the data @@ -227,7 +248,10 @@ idx = len(self.testdefs) - self._append_testdef(URLTestDefinition(self.context, idx, testdef)) + testdef_metadata = {'url': url, 'location': 'URL'} + testdef_metadata.update(_get_testdef_info(testdef)) + self._append_testdef(URLTestDefinition(self.context, idx, testdef, + testdef_metadata)) def load_from_repo(self, testdef_repo): tmpdir = utils.mkdtemp(self.tmpbase) @@ -289,9 +313,10 @@ A test definition that was loaded from a URL. """ - def __init__(self, context, idx, testdef): + def __init__(self, context, idx, testdef, testdef_metadata): self.context = context self.testdef = testdef + self.testdef_metadata = testdef_metadata self.idx = idx self.test_run_id = '%s_%s' % (idx, self.testdef['metadata']['name']) self.uuid = str(uuid4()) @@ -378,6 +403,9 @@ with open('%s/uuid' % hostdir, 'w') as f: f.write(self.uuid) + with open('%s/testdef_metadata' % hostdir, 'w') as f: + f.write(yaml.dump(self.testdef_metadata)) + if 'install' in self.testdef: self._create_repos(hostdir) self._create_target_install(hostdir, targetdir) @@ -408,7 +436,14 @@ """ def __init__(self, context, idx, testdef, repo, info): - URLTestDefinition.__init__(self, context, idx, testdef) + testdef_metadata = {} + testdef_metadata.update({'url': info['branch_url']}) + testdef_metadata.update({'location': info['branch_vcs'].upper()}) + testdef_metadata.update({'repo_rev': info['branch_revision']}) + testdef_metadata.update(_get_testdef_info(testdef)) + + URLTestDefinition.__init__(self, context, idx, testdef, + testdef_metadata) self.repo = repo self._sw_sources.append(info) === modified file 'lava_dispatcher/lava_test_shell.py' --- lava_dispatcher/lava_test_shell.py 2013-01-14 03:08:36 +0000 +++ lava_dispatcher/lava_test_shell.py 2013-01-28 08:29:21 +0000 @@ -292,8 +292,11 @@ uuid = _read_content(os.path.join(test_run_dir, 'analyzer_assigned_uuid')) attachments = _get_run_attachments(test_run_dir, testdef, stdout) attributes = _attributes_from_dir(os.path.join(test_run_dir, 'attributes')) + testdef_metadata = _read_content(os.path.join(test_run_dir, + 'testdef_metadata')) testdef = yaml.load(testdef) + testdef_metadata = yaml.load(testdef_metadata) if uuid in testdefs_by_uuid: sw_sources = testdefs_by_uuid[uuid]._sw_sources else: @@ -311,6 +314,7 @@ 'hardware_context': hwcontext, 'attachments': attachments, 'attributes': attributes, + 'testdef_metadata': testdef_metadata, } === modified file 'lava_dispatcher/test_data.py' --- lava_dispatcher/test_data.py 2012-07-31 20:20:02 +0000 +++ lava_dispatcher/test_data.py 2013-01-28 08:29:21 +0000 @@ -34,7 +34,8 @@ def __init__(self, test_id='lava'): self.job_status = 'pass' self.metadata = {} - self._test_run = { 'test_results':[], 'attachments':[], 'tags':[] } + self._test_run = { 'test_results':[], 'attachments':[], 'tags':[], + 'testdef_metadata':{} } self._test_run['test_id'] = test_id self._assign_date() self._assign_uuid() @@ -75,4 +76,3 @@ def get_test_run(self): self.add_result('job_complete', self.job_status) return self._test_run - === modified file 'lava_test_shell/lava-test-runner-android' --- lava_test_shell/lava-test-runner-android 2013-01-04 13:23:08 +0000 +++ lava_test_shell/lava-test-runner-android 2013-01-28 08:51:59 +0000 @@ -92,6 +92,7 @@ mkdir ${odir} mkdir ${odir}/attachments/ cp ${line}/testdef.yaml ${odir}/ + cp ${line}/testdef_metadata ${odir}/ cp ${line}/uuid ${odir}/analyzer_assigned_uuid cp ${line}/run.sh ${odir}/attachments/ echo 'text/plain' > ${odir}/attachments/run.sh.mimetype === modified file 'lava_test_shell/lava-test-runner-ubuntu' --- lava_test_shell/lava-test-runner-ubuntu 2013-01-14 16:30:05 +0000 +++ lava_test_shell/lava-test-runner-ubuntu 2013-01-28 08:51:59 +0000 @@ -76,6 +76,7 @@ mkdir ${odir}/attachments/ cp ${line}/uuid ${odir}/analyzer_assigned_uuid cp ${line}/testdef.yaml ${odir}/ + cp ${line}/testdef_metadata ${odir}/ cp ${line}/run.sh ${odir}/attachments/ echo 'text/plain' > ${odir}/attachments/run.sh.mimetype if [ -f ${line}/install.sh ]; then