From patchwork Tue Jan 10 17:38:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 640895 Delivered-To: patch@linaro.org Received: by 2002:a17:522:f3c4:b0:4b4:3859:abed with SMTP id in4csp2882699pvb; Tue, 10 Jan 2023 11:31:37 -0800 (PST) X-Google-Smtp-Source: AMrXdXv8zw4wXH9o9X8KBYtyRO9+NPy71QVpq7xg4M21EPcFPrxn3W80JQl8J2OnAYuOURBibaHZ X-Received: by 2002:a05:6214:3d86:b0:532:1448:eacd with SMTP id om6-20020a0562143d8600b005321448eacdmr29446295qvb.43.1673379097437; Tue, 10 Jan 2023 11:31:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673379097; cv=none; d=google.com; s=arc-20160816; b=A+60MBaPxciTZdRskM1S2GBBTItZdv3K4Y/RAhsuOWl/elHsOYVgkG8fEQiR0nmRtQ FBJoSxjK6sfUmTrptGcETI8MmVDs/uWsREzyH0E4H7/Wo5g27XePwXn+EW9QDaGJDBNA TR5SPBsDmGgeKFkFxZCzK8wjtVtY5XreY/fxaJR3oU+92zv2nEEehyeKBmKEq0qUSDj3 4POBHk1zs3DF3ap9yQCKsBQrnxXMQRjnYZx/0ukHL8GILOOHqVuUmoWKn581ANcez8dl raH66vbPuNIt3R1zhA6VvxhgnaHgXwJoeg9q+bubs0xU52LmoupJWTbgmTMsIFkZFPB5 kosQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=bkRhIgXl5AItP6ba03zv87lTI4x9mQkrreMEEEjOaX8=; b=yvGg7NpC/gzIhhDxBOnNFSse9+OyDDIcjF4hYogiSwAtWUNRvRGTHFUxrG0553HX8z nO0FKPz0EJ6UKa2p6S5jrI5q8GxL1wuN0fH9i39XNIW4kT3IwQw2vIUMShj3nmZi6HgR jaUKEKO9AGovZ8HJuFAcdn06Nf4U5/CEqzeoX3s4GKmRu92aQpG2ny0emUsZqB2ZvLlT 0coEpZGfaf/zGo63QaaAmUrba3Jf9zFI5W4w94pFdW8l9S7oQw8XwPSM3v9/vPPcABal H+UJwrKCVkIjcIWWmaM9u2HjhjVBX/2syXBlT+Cags5H4xMcjSuk3RVh64jpm2dnurUo kpCA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=b65qHxeO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTP id ke25-20020a056214301900b00532150b90d1si6397689qvb.225.2023.01.10.11.31.37 for ; Tue, 10 Jan 2023 11:31:37 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=b65qHxeO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pFIay-0003Le-Ek; Tue, 10 Jan 2023 12:39:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pFIaw-0003KI-EP for qemu-devel@nongnu.org; Tue, 10 Jan 2023 12:39:30 -0500 Received: from mail-wm1-x32b.google.com ([2a00:1450:4864:20::32b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pFIau-0001D0-Cy for qemu-devel@nongnu.org; Tue, 10 Jan 2023 12:39:30 -0500 Received: by mail-wm1-x32b.google.com with SMTP id bg13-20020a05600c3c8d00b003d9712b29d2so12528731wmb.2 for ; Tue, 10 Jan 2023 09:39:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bkRhIgXl5AItP6ba03zv87lTI4x9mQkrreMEEEjOaX8=; b=b65qHxeOTQRmnPnZNHnkijD7BVkvWLLedhS9k2IQf9v+Ct4+frnUPvYpicUSKRFRUl UAZdDPxBwfcII5/H2yIt0umM1km+hMSNqmcs/UsKbDdOpfZapj52J2PKXGk/ULeov2i4 HmtYjqCXG7o5E9u/h49fssbpS23L6cIredoTtEroZunXCDA+OB2Wt8kmqKXTg0caW1ql Fltfg5yvyN4c5JzqW4kzCu08NI9CwmbfBc32fU1JIyPTdahjQXPiQkL1nqAPnzqTlKix oVNWq65oVI/7R2Pbo8PtGt2IgsyhghSZ1ylC+hJ3NGYiixbVJg73c1mEryEHsUJPgQ02 B1WA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bkRhIgXl5AItP6ba03zv87lTI4x9mQkrreMEEEjOaX8=; b=Ba7lu6R8/nMd97xFj3JsQCwvwMtAPekugmuYvquhvCaFAFIlWBKNp6K25YNUq9hau7 /PKyH9CoGl6KdU9eMTLiBbqzQzHRXhGoLeXWfQ7AcYPBvq17nXAVhNWNntUdHK9owVcQ /XU3wCrnBwsYvkFBzBbinrbeM4erCS0my9PlTYcF5bbE46/3P63M4HA41G+W22U0zKHQ hOYq65AzZyu9JmzHs5tSowgWH13T+N8mN6r55opbA2sKJRHLpfi/f8GwmxApBtoX0ZKD utdM3vyESMhZUV4xmoKIDwEc+vc6ofhtMO2H5I8+BQIwQKsIwxoOK9qgqANRFy4En9tm yZrQ== X-Gm-Message-State: AFqh2kpoLrzfO1T7Lm+HG5HDlS8r/jCOBFEX5/EEaRVU8U/FMpEWhvcm 6vBBY28ppxToizCvkrQcFsMPDA== X-Received: by 2002:a05:600c:1d8a:b0:3d9:ebab:ccff with SMTP id p10-20020a05600c1d8a00b003d9ebabccffmr7904161wms.33.1673372365605; Tue, 10 Jan 2023 09:39:25 -0800 (PST) Received: from zen.linaroharston ([185.81.254.11]) by smtp.gmail.com with ESMTPSA id x7-20020a05600c188700b003c701c12a17sm19162023wmp.12.2023.01.10.09.39.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Jan 2023 09:39:24 -0800 (PST) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id D41211FFB8; Tue, 10 Jan 2023 17:39:22 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Cc: Richard Henderson , Alexandre Iooss , Beraldo Leal , Thomas Huth , John Snow , Eduardo Habkost , Elena Ufimtseva , Ed Maste , Yanan Wang , Cleber Rosa , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , Li-Wen Hsu , Markus Armbruster , Jagannathan Raman , =?utf-8?q?Daniel_P=2E_Berrang?= =?utf-8?q?=C3=A9?= , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?q?=C3=A9?= , Michael Roth , Wainer dos Santos Moschetta , =?utf-8?q?Alex_Benn?= =?utf-8?q?=C3=A9e?= , qemu-arm@nongnu.org, Marcel Apfelbaum , Peter Maydell , Paolo Bonzini , Mahmoud Mandour , John G Johnson Subject: [PATCH 01/26] scripts/ci: update gitlab-runner playbook to use latest runner Date: Tue, 10 Jan 2023 17:38:57 +0000 Message-Id: <20230110173922.265055-2-alex.bennee@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230110173922.265055-1-alex.bennee@linaro.org> References: <20230110173922.265055-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::32b; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org We were using quite and old runner on our machines and running into issues with stalling jobs. Gitlab in the meantime now reliably provide the latest packaged versions of the runner under a stable URL. This update: - creates a per-arch subdir for builds - switches from binary tarballs to deb packages - re-uses the same binary for the secondary runner - updates distro check for second to 22.04 Note this script isn't fully idempotent as we end up accumulating runners especially during testing. However we also want to be able to run twice with different GitLab keys (e.g. project and personal) so I think we just have to be mindful of that during testing. Signed-off-by: Alex Bennée --- v2 - only register aarch32 runner, move service start post both registers - tested on s390x --- scripts/ci/setup/gitlab-runner.yml | 56 +++++++----------------------- scripts/ci/setup/vars.yml.template | 2 -- 2 files changed, 13 insertions(+), 45 deletions(-) diff --git a/scripts/ci/setup/gitlab-runner.yml b/scripts/ci/setup/gitlab-runner.yml index 33128be85d..95d4199c03 100644 --- a/scripts/ci/setup/gitlab-runner.yml +++ b/scripts/ci/setup/gitlab-runner.yml @@ -50,60 +50,30 @@ - name: Download the matching gitlab-runner get_url: - dest: /usr/local/bin/gitlab-runner - url: "https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner-{{ gitlab_runner_os }}-{{ gitlab_runner_arch }}" - owner: gitlab-runner - group: gitlab-runner - mode: u=rwx,g=rwx,o=rx - - - name: Register the gitlab-runner - command: "/usr/local/bin/gitlab-runner register --non-interactive --url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_registration_token }} --executor shell --tag-list {{ ansible_facts[\"architecture\"] }},{{ ansible_facts[\"distribution\"]|lower }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{ ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'" - - - name: Install the gitlab-runner service using its own functionality - command: /usr/local/bin/gitlab-runner install --user gitlab-runner --working-directory /home/gitlab-runner - register: gitlab_runner_install_service_result - failed_when: "gitlab_runner_install_service_result.rc != 0 and \"already exists\" not in gitlab_runner_install_service_result.stderr" + dest: "/root/" + url: "https://gitlab-runner-downloads.s3.amazonaws.com/latest/deb/gitlab-runner_{{ gitlab_runner_arch }}.deb" - - name: Enable the gitlab-runner service - service: - name: gitlab-runner - state: started - enabled: yes + - name: Install gitlab-runner via package manager + apt: deb="/root/gitlab-runner_{{ gitlab_runner_arch }}.deb" - - name: Download secondary gitlab-runner - get_url: - dest: /usr/local/bin/gitlab-runner-arm - url: "https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner-{{ gitlab_runner_os }}-arm" - owner: gitlab-runner - group: gitlab-runner - mode: u=rwx,g=rwx,o=rx - when: - - ansible_facts['distribution'] == 'Ubuntu' - - ansible_facts['architecture'] == 'aarch64' - - ansible_facts['distribution_version'] == '20.04' + - name: Register the gitlab-runner + command: "/usr/bin/gitlab-runner register --non-interactive --url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_registration_token }} --executor shell --tag-list {{ ansible_facts[\"architecture\"] }},{{ ansible_facts[\"distribution\"]|lower }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{ ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'" + # The secondary runner will still run under the single gitlab-runner service - name: Register secondary gitlab-runner - command: "/usr/local/bin/gitlab-runner-arm register --non-interactive --url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_registration_token }} --executor shell --tag-list aarch32,{{ ansible_facts[\"distribution\"]|lower }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{ ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'" + command: "/usr/bin/gitlab-runner register --non-interactive --url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_registration_token }} --executor shell --tag-list aarch32,{{ ansible_facts[\"distribution\"]|lower }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{ ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'" when: - ansible_facts['distribution'] == 'Ubuntu' - ansible_facts['architecture'] == 'aarch64' - - ansible_facts['distribution_version'] == '20.04' + - ansible_facts['distribution_version'] == '22.04' - - name: Install the secondary gitlab-runner service using its own functionality - command: /usr/local/bin/gitlab-runner-arm install --user gitlab-runner --working-directory /home/gitlab-runner/arm -n gitlab-runner-arm + - name: Install the gitlab-runner service using its own functionality + command: "/usr/bin/gitlab-runner install --user gitlab-runner --working-directory /home/gitlab-runner" register: gitlab_runner_install_service_result failed_when: "gitlab_runner_install_service_result.rc != 0 and \"already exists\" not in gitlab_runner_install_service_result.stderr" - when: - - ansible_facts['distribution'] == 'Ubuntu' - - ansible_facts['architecture'] == 'aarch64' - - ansible_facts['distribution_version'] == '20.04' - - name: Enable the secondary gitlab-runner service + - name: Enable the gitlab-runner service service: - name: gitlab-runner-arm + name: gitlab-runner state: started enabled: yes - when: - - ansible_facts['distribution'] == 'Ubuntu' - - ansible_facts['architecture'] == 'aarch64' - - ansible_facts['distribution_version'] == '20.04' diff --git a/scripts/ci/setup/vars.yml.template b/scripts/ci/setup/vars.yml.template index e48089761f..4b355fb80f 100644 --- a/scripts/ci/setup/vars.yml.template +++ b/scripts/ci/setup/vars.yml.template @@ -1,5 +1,3 @@ -# The version of the gitlab-runner to use -gitlab_runner_version: 13.12.0 # The URL of the gitlab server to use, usually https://gitlab.com unless you're # using a private GitLab instance gitlab_runner_server_url: https://gitlab.com