From patchwork Fri Mar 21 09:58:00 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Campbell X-Patchwork-Id: 26800 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ob0-f200.google.com (mail-ob0-f200.google.com [209.85.214.200]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 74C4920143 for ; Fri, 21 Mar 2014 10:00:44 +0000 (UTC) Received: by mail-ob0-f200.google.com with SMTP id gq1sf8067784obb.3 for ; Fri, 21 Mar 2014 03:00:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:mime-version:cc:subject:precedence:list-id :list-unsubscribe:list-post:list-help:list-subscribe:sender :errors-to:x-original-sender:x-original-authentication-results :mailing-list:list-archive:content-type:content-transfer-encoding; bh=vYAZtWWfqpTcd6s8O+xjS+BODfdevWMgLVOUYZPMvHM=; b=PRC8PA/jq14XxZEO7O+xwI3JaMQHhquwZDunPnG6qZy5L+SgAc/qXVXGXhGuG1MZVz j6jk3dWRkNESGX9lkgyzI4t8kGRUCarx1wXgvRXyqFM9BEa/DFxrbD2rNaOl3A4WqqdK Nzp5qFq+xLD4L2Cc7KKmpkoeca7Q9c/FoFYNYysI7yEx4pClZHnh2GjC/fFuQRaw4cwF YptGJ3sEFrukA4CjuRCmXuAbsfb+k0A8rZrh+oZvgnBzIIUOQSTCP9hoNfuzA1ZHjNEA yXm7AAtdl2taE0gy0LFBEtjNi7cWGBgfbbTLLsc6tyJBqBzkTFYvq5dPQdGCwMoYSnuG 2hyA== X-Gm-Message-State: ALoCoQmcjKNg+hSIUKz50Y7Y0hDRMt3Jo6JAzNBjkNWOB698LudmsncvyC9uyxFcjSTu3HsaRZac X-Received: by 10.50.78.231 with SMTP id e7mr819582igx.5.1395396043812; Fri, 21 Mar 2014 03:00:43 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.27.138 with SMTP id 10ls610461qgx.52.gmail; Fri, 21 Mar 2014 03:00:43 -0700 (PDT) X-Received: by 10.58.243.39 with SMTP id wv7mr226470vec.51.1395396043574; Fri, 21 Mar 2014 03:00:43 -0700 (PDT) Received: from mail-ve0-f169.google.com (mail-ve0-f169.google.com [209.85.128.169]) by mx.google.com with ESMTPS id or7si1011453vcb.169.2014.03.21.03.00.43 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 21 Mar 2014 03:00:43 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.128.169 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.128.169; Received: by mail-ve0-f169.google.com with SMTP id pa12so2302617veb.14 for ; Fri, 21 Mar 2014 03:00:43 -0700 (PDT) X-Received: by 10.58.252.8 with SMTP id zo8mr76825vec.55.1395396043446; Fri, 21 Mar 2014 03:00:43 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.78.9 with SMTP id i9csp18467vck; Fri, 21 Mar 2014 03:00:43 -0700 (PDT) X-Received: by 10.58.243.39 with SMTP id wv7mr226432vec.51.1395396042913; Fri, 21 Mar 2014 03:00:42 -0700 (PDT) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id ku5si830196vec.190.2014.03.21.03.00.42 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Fri, 21 Mar 2014 03:00:42 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xen.org designates 50.57.142.19 as permitted sender) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WQwDG-0007Fl-Bb; Fri, 21 Mar 2014 09:58:06 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WQwDE-0007FV-UB for xen-devel@lists.xen.org; Fri, 21 Mar 2014 09:58:05 +0000 Received: from [85.158.137.68:48042] by server-1.bemta-3.messagelabs.com id 63/3B-11134-C2D0C235; Fri, 21 Mar 2014 09:58:04 +0000 X-Env-Sender: Ian.Campbell@citrix.com X-Msg-Ref: server-10.tower-31.messagelabs.com!1395395881!2102562!1 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n X-StarScan-Received: X-StarScan-Version: 6.11.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 19949 invoked from network); 21 Mar 2014 09:58:03 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-10.tower-31.messagelabs.com with RC4-SHA encrypted SMTP; 21 Mar 2014 09:58:03 -0000 X-IronPort-AV: E=Sophos;i="4.97,702,1389744000"; d="scan'208";a="112210576" Received: from accessns.citrite.net (HELO FTLPEX01CL03.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 21 Mar 2014 09:58:02 +0000 Received: from norwich.cam.xci-test.com (10.80.248.129) by smtprelay.citrix.com (10.13.107.80) with Microsoft SMTP Server id 14.2.342.4; Fri, 21 Mar 2014 05:58:00 -0400 Received: from spare.cam.xci-test.com ([10.80.2.80] helo=kazak.uk.xensource.com.) by norwich.cam.xci-test.com with esmtp (Exim 4.72) (envelope-from ) id 1WQwDA-0003MQ-MA; Fri, 21 Mar 2014 09:58:00 +0000 From: Ian Campbell To: Date: Fri, 21 Mar 2014 09:58:00 +0000 Message-ID: <1395395880-28830-2-git-send-email-ian.campbell@citrix.com> X-Mailer: git-send-email 1.8.5.2 In-Reply-To: <1395395880-28830-1-git-send-email-ian.campbell@citrix.com> References: <1395395880-28830-1-git-send-email-ian.campbell@citrix.com> MIME-Version: 1.0 X-DLP: MIA2 Cc: Ian Campbell , xen-devel@lists.xen.org Subject: [Xen-devel] [PATCH OSSTEST v2 2/2] standalone mode convenience frontend X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ian.campbell@citrix.com X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.128.169 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: I can never remember all the various env vars which I can/should set so this tool provides a command line veneer over the basics. It also does some sanity checks for things which keep tripping me up (inability to read apache logs, lack of ssh-agent) which fail in more or less obscure ways. Signed-off-by: Ian Campbell Acked-by: Ian Jackson --- v2: Remove redundant use of env Add a helper for logging and error checking Log the make-flight output --- standalone | 232 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 232 insertions(+) create mode 100755 standalone diff --git a/standalone b/standalone new file mode 100755 index 0000000..ab4f2bc --- /dev/null +++ b/standalone @@ -0,0 +1,232 @@ +#!/bin/bash + +set -e + +usage() { + cat >&2 <&2 + usage + exit 1 +fi + +op=$1 ; shift + +if [ x$op = x--help ] ; then + usage + exit 0 +fi + +TEMP=$(getopt -o c:f:h:rR --long config:,flight:,host:,reuse,noreuse,reinstall,help -- "$@") + +eval set -- "$TEMP" + +config= +flight="standalone" +host= +reuse=1 # Don't blow away machines by default + +while true ; do + case "$1" in + -c|--config) config=$2; shift 2;; + -f|--flight) flight=$2; shift 2;; + -h|--host) host=$2; shift 2;; + -r|--reuse) reuse=1; shift 1;; + -R|--noreuse|--reinstall)reuse=0;shift 1;; + --help) usage; exit 0;; + --) shift ; break ;; + *) echo "Internal error!" ; exit 1 ;; + + esac +done + +if [ ! -r /var/log/apache2/access.log ] ; then + echo "WARNING: Cannot read apache logs. Some tests may fail" >&2 +fi + +if ! ssh-add -l >/dev/null ] ; then + echo "WARNING: Unable to access ssh-agent. Some tests may fail" >&2 +fi + +if [ $reuse -eq 0 ]; then + echo "WARNING: Will blow away machine..." + echo "Press ENTER to confirm." + read +fi + +if [ ! -f standalone.db ] ; then + echo "No standalone.db? Run standalone-reset." >&2 + exit 1 +fi + +need_flight() { + if [ -z "$flight" ] ; then + echo "run-job: Need a flight" >&2 + exit 1 + fi +} +need_host() { + if [ -z "$host" ] ; then + echo "run-job: Need a host" >&2 + exit 1 + fi +} + +ensure_logs() { + if [ ! -d "logs" ] ; then + mkdir "logs" + fi + if [ ! -d "logs/$flight" ] ; then + mkdir "logs/$flight" + fi +} + +with_logging() { + local log=$1; shift + $@ 2>&1 | tee "$log" + rc=${PIPESTATUS[0]} + if [ $rc -ne 0 ] ; then + echo "FAILED rc=${rc}" >&2 + fi +} + +# other potential ops: +# - run standalone reset + +case $op in + help) + usage + exit 0 + ;; + + make-flight) + need_flight + + if [ $# -lt 1 ] ; then + echo "make-flight: Need branch" >&2 + exit 1 + fi + + branch=$1; shift + + DAILY_BRANCH_PREEXEC_HOOK=exit\ 0 \ + OSSTEST_USE_HEAD=y \ + BRANCHES_ALWAYS="$branch" \ + OSSTEST_FLIGHT=$flight \ + OSSTEST_CONFIG=$config \ + OSSTEST_NO_BASELINE=y \ + with_logging logs/$flight/make-flight.log ./cr-daily-branch $@ $branch + ;; + + set-paths) + need_flight + + if [ $# -lt 1 ] ; then + echo "set-paths: Need job" >&2 + exit 1 + fi + +#build-amd64 path_dist build/dist.tar.gz +#build-amd64 path_kerndist build/kerndist.tar.gz +#build-amd64-pvops path_kerndist build/kerndist.tar.gz +#build-amd64 path_xendist build/xendist.tar.gz + + job=$1; shift + + for d in '' xen kern ; do + runvar="path_${d}dist" + path="build/${d}dist.tar.gz" + if [ -f "logs/$flight/$job/$path" ] ; then + ./cs-adjust-flight -v $flight runvar-set $job $runvar $path + fi + done + + if [ -f "logs/$flight/$job/build/kerndist.tar.gz" ] ; then + KERNEL_VER=$(tar tzf logs/$flight/$job/build/kerndist.tar.gz |\ + sed -n -e 's,^lib/modules/\(.*\)/modules.dep$,\1,p') + if [ -n "$KERNEL_VER" ]; then + ./cs-adjust-flight -v $flight \ + runvar-set $job kernel_ver $KERNEL_VER + fi + fi + ;; + + run-job) + need_flight; need_host + + if [ $# -lt 1 ] ; then + echo "run-job: Need job" >&2 + exit 1 + fi + + job=$1; shift + + ensure_logs + + OSSTEST_CONFIG=$config \ + OSSTEST_FLIGHT=$flight \ + OSSTEST_HOST_HOST=$host \ + OSSTEST_HOST_REUSE=$reuse \ + with_logging logs/$flight/$job.log ./sg-run-job $job + ;; + run-test) + need_flight; need_host + + if [ $# -lt 2 ] ; then + echo "run-test: Need job + test" >&2 + exit 1 + fi + + job=$1; shift + ts=$1; shift + + ensure_logs + + OSSTEST_CONFIG=$config \ + OSSTEST_FLIGHT=$flight \ + OSSTEST_CONFIG=$config \ + OSSTEST_HOST_REUSE=$reuse \ + OSSTEST_JOB=$job \ + with_logging logs/$flight/$job.$ts.log ./$ts host=$host $@ + ;; + *) + echo "Unknown op $op" ; exit 1 ;; +esac