From patchwork Tue May 16 19:40:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682355 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp638614ltp; Tue, 16 May 2023 12:42:31 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7gObtqV0wugbrsQ1ybpCEER3BHYrRbShWTSZQRoOr89eQYmq5f2OJDAGkPwYyoo94cvzHZ X-Received: by 2002:ad4:5f4a:0:b0:5ef:8c79:fe98 with SMTP id p10-20020ad45f4a000000b005ef8c79fe98mr65604776qvg.8.1684266151196; Tue, 16 May 2023 12:42:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266151; cv=none; d=google.com; s=arc-20160816; b=SYsUh9z6/wqAQbIpOy+SIhO0EDN3S3EQo8dEKDmnoSeKhcGFjRav+ZOegfQ4Uh/5sl p12n1odteaxKlYYhjalCrWGTqdIfTSbBXqW93fUc4/fAbZzYBEoqLHy5wOX4fC8eAwTk EcZ7xsIsMVCwHpm3NM5QwVAgC25ijJHoap9Lhv+sjSEjnt67RmN+tnJpiPkJttksvxXk Mbo7pH1stLY2JLFEQjJw64bTvhGszpKLwvXic0IZwVSszzBeGCuKn9APxi7uk7FmYjLK xcLT8r1B9DeaBDXHqUPwy4Qgu8YHvPEJmWg2haQjwRaqzicUCd1UuZOmi8N1+C4x5gSN 2HAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=miBGGjW/ZQfOCs7U5dTa3vNRbLi2IQsdXFfT88TdwaM=; b=xAzONs/66r0FNenqkmknJ99uhU9jUzO8jPtPE0OAgNInYYcwjCEjFREvCDOG6SY3Ff CW8tnN7SY5a60V+5ZfgIHT9Yu9xcHd7hzi/kvQgec5kvT+nRkY0g8R2k2M9bdfYuso0F yo4oMwSxanZTxh6okAcwxepiV5x5X0dbnXoTu3FxnazVSeYN/Mn68Pe2/Du0FoHvmVIY GwlTjGTniT4olJ6XzPZMHTnMpkqTytktN8kcB/mz4b+N2qEf/7pJvHR2T70cHMkHXfTa W6cIfVtn+ZrGMGcU7GvOKhACaWk+t9SN0ZzOMzoVhGBW4vHic79i0EYnO5k/o3JvNBqS G2LQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=o2I0qmSJ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j26-20020a05620a147a00b006fcaae20037si177502qkl.741.2023.05.16.12.42.31 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:42:31 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=o2I0qmSJ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0YU-0006YV-TV; Tue, 16 May 2023 15:41:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0YT-0006Y7-91 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:53 -0400 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0YQ-0002j7-Go for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:53 -0400 Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-64389a44895so11562438b3a.1 for ; Tue, 16 May 2023 12:41:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266108; x=1686858108; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=miBGGjW/ZQfOCs7U5dTa3vNRbLi2IQsdXFfT88TdwaM=; b=o2I0qmSJlPWgILo6l5yrAlzCq+d2joADAau5WLC/Rgn1+koI9CjC8aPrI9WtiuQMOR 8oJ477/L6DY4oHU9V/5IVsKDIcDMa4heQWQLNodlHouoYFSI+jmHor8DHpwm8Gf+wuAy C/kDW/5EgMa/d0xq5LwwxBOfBgCHPJmaP8DYUuuD6Bp2M0j2ZE/zowbsMT+DeqOnZrnr ovdtJZ8G7F4zIoDGM9Nnj8Xi573xB8d7IhwTqaY8ZTXs9qtyBrCEyjP05eHjCQV1KfV8 mr3ui3QE9uWfmsdSI7P22tkcktdOVMYkGEYIEcgsV430kOg1Fa+4Kkz0NILqLmkI3/AS LH1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266108; x=1686858108; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=miBGGjW/ZQfOCs7U5dTa3vNRbLi2IQsdXFfT88TdwaM=; b=AvDN9oKwJs2Hq0citO3zJIN1/jZ/2THLcaP7idlk6ReJPXD37gdBq0SRidP0kMQytc i5Veo3uXbtFOxJmT25bogFDNu4rHJYHXXUU+ugFJ7CZRr2UF1xG3A8x0IFwN5ahPQGBU jDJ1U+eEFBnT/adNW8SmkndsRuf9dE3v6+wNMqV0q3dotwdCIt+nn4TB4QbWiOIy3fhp ouCLGTAb1ZqnngQ5iSYMTbUjtE20V5cIw37t4b4LKvxXFf39hOXfSnwloWDwyXuh3ntL s0MzxXYeewC1oHr5CrIC4slHs/z//8WTMysH3spHwRLBIyQfpbVRY/jPRiLgY1Kdjmtl eLvw== X-Gm-Message-State: AC+VfDzmXTyDKS5wda12lBFOs9uKNYzryyyKgiqXFXFWkifqvsFtaDbI mPS+2IQOiuEJhPMVzAEOnkiRoBNCNcm8DNHvL1o= X-Received: by 2002:a05:6a00:2303:b0:64c:ae1c:337f with SMTP id h3-20020a056a00230300b0064cae1c337fmr8508034pfh.12.1684266107833; Tue, 16 May 2023 12:41:47 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.41.47 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:41:47 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PULL 01/80] tcg/i386: Set P_REXW in tcg_out_addi_ptr Date: Tue, 16 May 2023 12:40:26 -0700 Message-Id: <20230516194145.1749305-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::429; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x429.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org The REXW bit must be set to produce a 64-bit pointer result; the bit is disabled in 32-bit mode, so we can do this unconditionally. Fixes: 7d9e1ee424b0 ("tcg/i386: Adjust assert in tcg_out_addi_ptr") Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1592 Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1642 Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.c.inc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index a01bfad773..9fc5592f5d 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -1091,7 +1091,7 @@ static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, { /* This function is only used for passing structs by reference. */ tcg_debug_assert(imm == (int32_t)imm); - tcg_out_modrm_offset(s, OPC_LEA, rd, rs, imm); + tcg_out_modrm_offset(s, OPC_LEA | P_REXW, rd, rs, imm); } static inline void tcg_out_pushi(TCGContext *s, tcg_target_long val) From patchwork Tue May 16 19:40:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682361 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp638885ltp; Tue, 16 May 2023 12:43:13 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5Kr/hDlWqUQwvwwrkz7C99xMIS7e6PptvmTvNv+oIM5x4cmf7kXFFFAT0O0kMPiqJFQvSZ X-Received: by 2002:ac8:5a85:0:b0:3ef:38a2:9fdb with SMTP id c5-20020ac85a85000000b003ef38a29fdbmr57471586qtc.60.1684266193235; Tue, 16 May 2023 12:43:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266193; cv=none; d=google.com; s=arc-20160816; b=ojuMNBIk97rNS1oJgVVKFBWX0lz8tPfjDmBfvEIJkUphJg1IQOoq5zTUjdTu2tpW1/ rNubJvU7Ar6Y/PvE4yH0sIyPXZxgewk9tOKXo79lDHvc69cv96DxeGvcmDuTuWzDSv42 rHptQ4Z/LOaqRx+ruvkYRDVdPPhaXeCNNtLCecytyX46yBYvgim/Uxze5nPD1Q4dGCuP rUL4GQ5tqMbJnh7pDWksyjhpJhN/Rv8MmhDdqBmi19c6TX5xlXGFLFEleuf4QDuwp6vD thCdXWT+nHro27lt2Ey6RXbmOIR0hd/gEgxfq5T13rJlW3XrtVq7eqzoQMCWPt4pjlzG Gucw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=3OQSR0QapQPCRrI0vNmh20VxsIRATUp7x7RlDjZ+0Sc=; b=tRQYP/vVVoQbSfR13qovGz5y1aUyXfaK25KKam+oXbl3nqzMHo06N6vlRM00eWa267 ySssM4w6vhvkrIB4MeHppCA+fPc1Z8DorNX3XPjDi+N4wSPyZ7zN68/1rWFKqgzTA7FH DgYzBaO+3pVO3D54tuMKWGX4azqq8N2kNGBIyDbRhq6WiKvri1OMtUKtuki4yrG90wJo zOggJNZRMY24iVTeM9lNBul9bvpAdOb+jGa4L7AZcnQmmX1XbCq0viH8QGs6fPHPLey1 BvkUdFyOi7lnMoIVPpYvUwBKlPikL1evXw5g256IGyU3179QrQ/T9lghDup01TWcRb7L Y3pg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TK0endBq; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f1-20020a05620a20c100b0074e1ee30485si249577qka.16.2023.05.16.12.43.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:43:13 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TK0endBq; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0YT-0006Y8-Bb; Tue, 16 May 2023 15:41:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0YR-0006Xl-UW for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:51 -0400 Received: from mail-pf1-x435.google.com ([2607:f8b0:4864:20::435]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0YQ-0002jI-7Q for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:51 -0400 Received: by mail-pf1-x435.google.com with SMTP id d2e1a72fcca58-6434e263962so10725304b3a.2 for ; Tue, 16 May 2023 12:41:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266109; x=1686858109; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3OQSR0QapQPCRrI0vNmh20VxsIRATUp7x7RlDjZ+0Sc=; b=TK0endBqSLA3EEfAbGGrIptlhb8V6TVqg6TI5hIfa5XvaEH0FAEyvLnja2bsYzpbXT Oy4LwQe1qoV9lAO9qDCmfbz0XGmHl2TK+bRtvCc8bsBU2woFk5y0YHM1cw/mpO5zhRBF AqganHTFpkWPY6hQA2Q16rRb/aBfwI9Im3ob4XpaJ69N2tPai1HK9ehFANNhgGtAVSzk ev8WvJd34Risten/6R28MJNDGDA7XC6z/iqzBsY5jk74WSoGItRrzdFk+vKUvpQdS3cV fyMdZZwi96JIlGjnZ+qWsI1DctIXBmbYARU8/3jncIBW2s3vc8xRRZvkDDwOLXfFRmSv 4vtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266109; x=1686858109; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3OQSR0QapQPCRrI0vNmh20VxsIRATUp7x7RlDjZ+0Sc=; b=R8GSnfP8xcwfTfNQ0m/lsgfb1kuN4EEmXQtR1Uhn7kEBQddou50WARZ0rXjn7CJSje Ry6BDp88aboi+WxXWZelZjFw5hSwFfInYe4p8QREkgnBQBv5gkLRt7J3gyGGaaZUzezx q2TKFi5FUrljPTcK3MJyJQpCAu7s8Y+zHI8zyuW8N6vF9mpzukT7hqS1pWfNmrBFxhZ8 h4n+x4jGETG5Xae9P3ZR1hwOzOIdM6MdvQ2XeFJ3pC5/YnTjOPe/cB5dFvvJfLCDsEAY uolXBRD3KQxAvmayIO7fwaevixkwUzQiUjoY3/THFx6UIjgLOKVMSYB/1OauAZWryy9D OT7g== X-Gm-Message-State: AC+VfDwGCr2VIYr5NbNn9+hUXejwvWxyq/tsZgO2zVDEBsLdQECwbWMG N2q5VjGwxxxm7XVQ1kHRak8uIIwaYzFUjcB7QQc= X-Received: by 2002:a05:6a00:21d2:b0:647:6a0:2534 with SMTP id t18-20020a056a0021d200b0064706a02534mr34363796pfj.16.1684266108862; Tue, 16 May 2023 12:41:48 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.41.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:41:48 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 02/80] include/exec/memop: Add MO_ATOM_* Date: Tue, 16 May 2023 12:40:27 -0700 Message-Id: <20230516194145.1749305-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::435; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x435.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org This field may be used to describe the precise atomicity requirements of the guest, which may then be used to constrain the methods by which it may be emulated by the host. For instance, the AArch64 LDP (32-bit) instruction changes semantics with ARMv8.4 LSE2, from MO_64 | MO_ATOM_IFALIGN_PAIR (64-bits, single-copy atomic only on 4 byte units, nonatomic if not aligned by 4), to MO_64 | MO_ATOM_WITHIN16 (64-bits, single-copy atomic within a 16 byte block) The former may be implemented with two 4 byte loads, or a single 8 byte load if that happens to be efficient on the host. The latter may not be implemented with two 4 byte loads and may also require a helper when misaligned. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- include/exec/memop.h | 37 +++++++++++++++++++++++++++++++++++++ tcg/tcg.c | 27 +++++++++++++++++++++------ 2 files changed, 58 insertions(+), 6 deletions(-) diff --git a/include/exec/memop.h b/include/exec/memop.h index 07f5f88188..a86dc6743a 100644 --- a/include/exec/memop.h +++ b/include/exec/memop.h @@ -72,6 +72,43 @@ typedef enum MemOp { MO_ALIGN_64 = 6 << MO_ASHIFT, MO_ALIGN = MO_AMASK, + /* + * MO_ATOM_* describes the atomicity requirements of the operation: + * MO_ATOM_IFALIGN: the operation must be single-copy atomic if it + * is aligned; if unaligned there is no atomicity. + * MO_ATOM_IFALIGN_PAIR: the entire operation may be considered to + * be a pair of half-sized operations which are packed together + * for convenience, with single-copy atomicity on each half if + * the half is aligned. + * This is the atomicity e.g. of Arm pre-FEAT_LSE2 LDP. + * MO_ATOM_WITHIN16: the operation is single-copy atomic, even if it + * is unaligned, so long as it does not cross a 16-byte boundary; + * if it crosses a 16-byte boundary there is no atomicity. + * This is the atomicity e.g. of Arm FEAT_LSE2 LDR. + * MO_ATOM_WITHIN16_PAIR: the entire operation is single-copy atomic, + * if it happens to be within a 16-byte boundary, otherwise it + * devolves to a pair of half-sized MO_ATOM_WITHIN16 operations. + * Depending on alignment, one or both will be single-copy atomic. + * This is the atomicity e.g. of Arm FEAT_LSE2 LDP. + * MO_ATOM_SUBALIGN: the operation is single-copy atomic by parts + * by the alignment. E.g. if the address is 0 mod 4, then each + * 4-byte subobject is single-copy atomic. + * This is the atomicity e.g. of IBM Power. + * MO_ATOM_NONE: the operation has no atomicity requirements. + * + * Note the default (i.e. 0) value is single-copy atomic to the + * size of the operation, if aligned. This retains the behaviour + * from before this field was introduced. + */ + MO_ATOM_SHIFT = 8, + MO_ATOM_IFALIGN = 0 << MO_ATOM_SHIFT, + MO_ATOM_IFALIGN_PAIR = 1 << MO_ATOM_SHIFT, + MO_ATOM_WITHIN16 = 2 << MO_ATOM_SHIFT, + MO_ATOM_WITHIN16_PAIR = 3 << MO_ATOM_SHIFT, + MO_ATOM_SUBALIGN = 4 << MO_ATOM_SHIFT, + MO_ATOM_NONE = 5 << MO_ATOM_SHIFT, + MO_ATOM_MASK = 7 << MO_ATOM_SHIFT, + /* Combinations of the above, for ease of use. */ MO_UB = MO_8, MO_UW = MO_16, diff --git a/tcg/tcg.c b/tcg/tcg.c index 1231c8ab4c..f156ca65f5 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -2195,6 +2195,15 @@ static const char * const alignment_name[(MO_AMASK >> MO_ASHIFT) + 1] = { [MO_ALIGN_64 >> MO_ASHIFT] = "al64+", }; +static const char * const atom_name[(MO_ATOM_MASK >> MO_ATOM_SHIFT) + 1] = { + [MO_ATOM_IFALIGN >> MO_ATOM_SHIFT] = "", + [MO_ATOM_IFALIGN_PAIR >> MO_ATOM_SHIFT] = "pair+", + [MO_ATOM_WITHIN16 >> MO_ATOM_SHIFT] = "w16+", + [MO_ATOM_WITHIN16_PAIR >> MO_ATOM_SHIFT] = "w16p+", + [MO_ATOM_SUBALIGN >> MO_ATOM_SHIFT] = "sub+", + [MO_ATOM_NONE >> MO_ATOM_SHIFT] = "noat+", +}; + static const char bswap_flag_name[][6] = { [TCG_BSWAP_IZ] = "iz", [TCG_BSWAP_OZ] = "oz", @@ -2330,17 +2339,23 @@ static void tcg_dump_ops(TCGContext *s, FILE *f, bool have_prefs) case INDEX_op_qemu_ld_i64: case INDEX_op_qemu_st_i64: { + const char *s_al, *s_op, *s_at; MemOpIdx oi = op->args[k++]; MemOp op = get_memop(oi); unsigned ix = get_mmuidx(oi); - if (op & ~(MO_AMASK | MO_BSWAP | MO_SSIZE)) { - col += ne_fprintf(f, ",$0x%x,%u", op, ix); + s_al = alignment_name[(op & MO_AMASK) >> MO_ASHIFT]; + s_op = ldst_name[op & (MO_BSWAP | MO_SSIZE)]; + s_at = atom_name[(op & MO_ATOM_MASK) >> MO_ATOM_SHIFT]; + op &= ~(MO_AMASK | MO_BSWAP | MO_SSIZE | MO_ATOM_MASK); + + /* If all fields are accounted for, print symbolically. */ + if (!op && s_al && s_op && s_at) { + col += ne_fprintf(f, ",%s%s%s,%u", + s_at, s_al, s_op, ix); } else { - const char *s_al, *s_op; - s_al = alignment_name[(op & MO_AMASK) >> MO_ASHIFT]; - s_op = ldst_name[op & (MO_BSWAP | MO_SSIZE)]; - col += ne_fprintf(f, ",%s%s,%u", s_al, s_op, ix); + op = get_memop(oi); + col += ne_fprintf(f, ",$0x%x,%u", op, ix); } i = 1; } From patchwork Tue May 16 19:40:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682406 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp20478wrt; Tue, 16 May 2023 12:57:20 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6pzKUIbU3mzbIjogzcePiX6JXDGuUt0OnKky5R6IKpdvWYlNDkU18xbY56yUBPL5FsydGF X-Received: by 2002:a05:622a:1310:b0:3ef:3f4b:31b8 with SMTP id v16-20020a05622a131000b003ef3f4b31b8mr66190838qtk.7.1684267039895; Tue, 16 May 2023 12:57:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267039; cv=none; d=google.com; s=arc-20160816; b=QLc+67hzX1QgJ6z8rbOPHAwUMpVMUJ5DUb2LlnG2Ad6pPGXvLk5f4cwy5kTyhiLJfN qbXTfyQkt2q8NjZDf5ouV4Qu2YjLiYtIjASkzvYI4Gf5k68NWBQbJ52ZPF3FYgR/nEZL k7Gimr597DQR2W6j6nO5scYPn6E8VDsKUMsQktKtrIczQKV9/hf41VisSL5D8+f7keUM ir7lUrdiMcurqvs5s9eRrFLFA+4S5b0nf0W/os/Kq17dfwnJzB5gvDdhwFxHdu48/DlS rkIu6pl/cPwOlkQWd+CR06wuZu0RIFmEcGqTnTrJofEkkAipaIRdjuJrX3IqM5IZc0a/ 6TPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=/u2p2q8dwfRIKl/CIjB4kbZZa8MNhXSzHudQqwU8vgU=; b=Xfl5RHOE+eByvgkbvRvbIQY2/7MWN+MARZ+5TFM6EZomNyjASzbVhw6eO9+JLUqn+D xZ6IlGRJr3VAWcxb0g35HUJgWSeRJFD/6g5ceUGQDEV87eUj0JID0s+1pyvc5j9SOgwu 9/m/5N6Uc70WxDyTPrz7+KwPTfCy9IKqqKXSHg5rhy2Ktri8Q9SxQTl5iDBKMiHYRC1H yIBUFYI6+mYd7GsjE9jJU11ZizijedeookDtQjtv5voLL7xP4MlgNqkiwsq+V0LMYFL4 2uCS3BdyuXV1iZaCm2L9tXa+QEB/nms5SYTCk4kks+mt+9xTykeui731X0cuPJfXhJ/j JMZQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bzkojjEg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t4-20020a05620a034400b0074676f15d3bsi268606qkm.25.2023.05.16.12.57.19 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:57:19 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bzkojjEg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0YW-0006Z8-B1; Tue, 16 May 2023 15:41:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0YU-0006YU-P3 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:54 -0400 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0YR-0002jW-KP for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:54 -0400 Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-643a6f993a7so9345216b3a.1 for ; Tue, 16 May 2023 12:41:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266110; x=1686858110; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/u2p2q8dwfRIKl/CIjB4kbZZa8MNhXSzHudQqwU8vgU=; b=bzkojjEgVO7zpjzcbDYpYMb+PhA8gS/q/fE6TLqfnn3NZxRzkeJTLCFNe6LRPiMGWe 3CE9fhuS7ydN1eHGw/oK/xCVmc43+vWVhSvkRWDMsRhUznbceuOr60PQVu4EtY5aTxlt U0d8m7/uph70AOQhZ17rwD1xlvA5ZOkzsJKVv3rKVnXkMTqjc9pdvnCn2zz0DeJt9Xb3 vtF5E0olA/UzYc7NtItp2lwT3u1kOBdWMJJ3aBJhip/b5S1DTAgRss9/141YlXkI2ymx YctbJy5+7fVPprPhB+GwEl0x3dyLUErCH0gSLHzT+PmGnDmq2vQyveck+s4/7NtMEaGV 52MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266110; x=1686858110; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/u2p2q8dwfRIKl/CIjB4kbZZa8MNhXSzHudQqwU8vgU=; b=I79Tw9GxeUsqbxaAHueL6bwQE7JBA0dCeU1gOdb59Mcw2rvyY7uLGOPxkXDd7QRWai XvQQQZbEJkqKDrxHj2Le/ciquIBxtyK2EOgib2kzikCG/LK5PdFYn4IKra3+tgALQN9l JNHH3KsUUFa85NwiqNBcicuM9xy1bTMs82FMmKQCTOvrmaPMjdlPj6UQzTM6M38UoWvy xZS9oL01HKpt21YVGpL36HIo1l2NspF85TnZxW597Z3uDpBt1Xugt4LFCIBg5+MriEOi nkdASIZG1dbVYziNqqS7wM6KCfYyzA//c24Axkjzh9QAld9XvGMMWOv7lWJjRKWaz0cz lFPA== X-Gm-Message-State: AC+VfDw9yYVGOvPO0AjDa1GiDQFRCtW3JA1kcdkFzt7VDE7Qhrnolh9v mibr0meFmq5PsXtZaMaO1I1M9mPdgwQPCM1gVkY= X-Received: by 2002:a05:6a00:1141:b0:64a:ff32:7349 with SMTP id b1-20020a056a00114100b0064aff327349mr16178713pfm.32.1684266109894; Tue, 16 May 2023 12:41:49 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.41.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:41:49 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 03/80] accel/tcg: Honor atomicity of loads Date: Tue, 16 May 2023 12:40:28 -0700 Message-Id: <20230516194145.1749305-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::432; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x432.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Create ldst_atomicity.c.inc. Not required for user-only code loads, because we've ensured that the page is read-only before beginning to translate code. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 175 +++++++--- accel/tcg/user-exec.c | 26 +- accel/tcg/ldst_atomicity.c.inc | 566 +++++++++++++++++++++++++++++++++ 3 files changed, 716 insertions(+), 51 deletions(-) create mode 100644 accel/tcg/ldst_atomicity.c.inc diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 617777055a..33e75ae962 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1668,6 +1668,9 @@ tb_page_addr_t get_page_addr_code_hostp(CPUArchState *env, target_ulong addr, return qemu_ram_addr_from_host_nofail(p); } +/* Load/store with atomicity primitives. */ +#include "ldst_atomicity.c.inc" + #ifdef CONFIG_PLUGIN /* * Perform a TLB lookup and populate the qemu_plugin_hwaddr structure. @@ -2035,35 +2038,7 @@ static void validate_memop(MemOpIdx oi, MemOp expected) * specifically for reading instructions from system memory. It is * called by the translation loop and in some helpers where the code * is disassembled. It shouldn't be called directly by guest code. - */ - -typedef uint64_t FullLoadHelper(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr); - -static inline uint64_t QEMU_ALWAYS_INLINE -load_memop(const void *haddr, MemOp op) -{ - switch (op) { - case MO_UB: - return ldub_p(haddr); - case MO_BEUW: - return lduw_be_p(haddr); - case MO_LEUW: - return lduw_le_p(haddr); - case MO_BEUL: - return (uint32_t)ldl_be_p(haddr); - case MO_LEUL: - return (uint32_t)ldl_le_p(haddr); - case MO_BEUQ: - return ldq_be_p(haddr); - case MO_LEUQ: - return ldq_le_p(haddr); - default: - qemu_build_not_reached(); - } -} - -/* + * * For the benefit of TCG generated code, we want to avoid the * complication of ABI-specific return type promotion and always * return a value extended to the register size of the host. This is @@ -2119,17 +2094,139 @@ static uint64_t do_ld_bytes_beN(MMULookupPageData *p, uint64_t ret_be) return ret_be; } +/** + * do_ld_parts_beN + * @p: translation parameters + * @ret_be: accumulated data + * + * As do_ld_bytes_beN, but atomically on each aligned part. + */ +static uint64_t do_ld_parts_beN(MMULookupPageData *p, uint64_t ret_be) +{ + void *haddr = p->haddr; + int size = p->size; + + do { + uint64_t x; + int n; + + /* + * Find minimum of alignment and size. + * This is slightly stronger than required by MO_ATOM_SUBALIGN, which + * would have only checked the low bits of addr|size once at the start, + * but is just as easy. + */ + switch (((uintptr_t)haddr | size) & 7) { + case 4: + x = cpu_to_be32(load_atomic4(haddr)); + ret_be = (ret_be << 32) | x; + n = 4; + break; + case 2: + case 6: + x = cpu_to_be16(load_atomic2(haddr)); + ret_be = (ret_be << 16) | x; + n = 2; + break; + default: + x = *(uint8_t *)haddr; + ret_be = (ret_be << 8) | x; + n = 1; + break; + case 0: + g_assert_not_reached(); + } + haddr += n; + size -= n; + } while (size != 0); + return ret_be; +} + +/** + * do_ld_parts_be4 + * @p: translation parameters + * @ret_be: accumulated data + * + * As do_ld_bytes_beN, but with one atomic load. + * Four aligned bytes are guaranteed to cover the load. + */ +static uint64_t do_ld_whole_be4(MMULookupPageData *p, uint64_t ret_be) +{ + int o = p->addr & 3; + uint32_t x = load_atomic4(p->haddr - o); + + x = cpu_to_be32(x); + x <<= o * 8; + x >>= (4 - p->size) * 8; + return (ret_be << (p->size * 8)) | x; +} + +/** + * do_ld_parts_be8 + * @p: translation parameters + * @ret_be: accumulated data + * + * As do_ld_bytes_beN, but with one atomic load. + * Eight aligned bytes are guaranteed to cover the load. + */ +static uint64_t do_ld_whole_be8(CPUArchState *env, uintptr_t ra, + MMULookupPageData *p, uint64_t ret_be) +{ + int o = p->addr & 7; + uint64_t x = load_atomic8_or_exit(env, ra, p->haddr - o); + + x = cpu_to_be64(x); + x <<= o * 8; + x >>= (8 - p->size) * 8; + return (ret_be << (p->size * 8)) | x; +} + /* * Wrapper for the above. */ static uint64_t do_ld_beN(CPUArchState *env, MMULookupPageData *p, - uint64_t ret_be, int mmu_idx, - MMUAccessType type, uintptr_t ra) + uint64_t ret_be, int mmu_idx, MMUAccessType type, + MemOp mop, uintptr_t ra) { + MemOp atom; + unsigned tmp, half_size; + if (unlikely(p->flags & TLB_MMIO)) { return do_ld_mmio_beN(env, p, ret_be, mmu_idx, type, ra); - } else { + } + + /* + * It is a given that we cross a page and therefore there is no + * atomicity for the load as a whole, but subobjects may need attention. + */ + atom = mop & MO_ATOM_MASK; + switch (atom) { + case MO_ATOM_SUBALIGN: + return do_ld_parts_beN(p, ret_be); + + case MO_ATOM_IFALIGN_PAIR: + case MO_ATOM_WITHIN16_PAIR: + tmp = mop & MO_SIZE; + tmp = tmp ? tmp - 1 : 0; + half_size = 1 << tmp; + if (atom == MO_ATOM_IFALIGN_PAIR + ? p->size == half_size + : p->size >= half_size) { + if (!HAVE_al8_fast && p->size < 4) { + return do_ld_whole_be4(p, ret_be); + } else { + return do_ld_whole_be8(env, ra, p, ret_be); + } + } + /* fall through */ + + case MO_ATOM_IFALIGN: + case MO_ATOM_WITHIN16: + case MO_ATOM_NONE: return do_ld_bytes_beN(p, ret_be); + + default: + g_assert_not_reached(); } } @@ -2153,7 +2250,7 @@ static uint16_t do_ld_2(CPUArchState *env, MMULookupPageData *p, int mmu_idx, } /* Perform the load host endian, then swap if necessary. */ - ret = load_memop(p->haddr, MO_UW); + ret = load_atom_2(env, ra, p->haddr, memop); if (memop & MO_BSWAP) { ret = bswap16(ret); } @@ -2170,7 +2267,7 @@ static uint32_t do_ld_4(CPUArchState *env, MMULookupPageData *p, int mmu_idx, } /* Perform the load host endian. */ - ret = load_memop(p->haddr, MO_UL); + ret = load_atom_4(env, ra, p->haddr, memop); if (memop & MO_BSWAP) { ret = bswap32(ret); } @@ -2187,7 +2284,7 @@ static uint64_t do_ld_8(CPUArchState *env, MMULookupPageData *p, int mmu_idx, } /* Perform the load host endian. */ - ret = load_memop(p->haddr, MO_UQ); + ret = load_atom_8(env, ra, p->haddr, memop); if (memop & MO_BSWAP) { ret = bswap64(ret); } @@ -2263,8 +2360,8 @@ static uint32_t do_ld4_mmu(CPUArchState *env, target_ulong addr, MemOpIdx oi, return do_ld_4(env, &l.page[0], l.mmu_idx, access_type, l.memop, ra); } - ret = do_ld_beN(env, &l.page[0], 0, l.mmu_idx, access_type, ra); - ret = do_ld_beN(env, &l.page[1], ret, l.mmu_idx, access_type, ra); + ret = do_ld_beN(env, &l.page[0], 0, l.mmu_idx, access_type, l.memop, ra); + ret = do_ld_beN(env, &l.page[1], ret, l.mmu_idx, access_type, l.memop, ra); if ((l.memop & MO_BSWAP) == MO_LE) { ret = bswap32(ret); } @@ -2297,8 +2394,8 @@ static uint64_t do_ld8_mmu(CPUArchState *env, target_ulong addr, MemOpIdx oi, return do_ld_8(env, &l.page[0], l.mmu_idx, access_type, l.memop, ra); } - ret = do_ld_beN(env, &l.page[0], 0, l.mmu_idx, access_type, ra); - ret = do_ld_beN(env, &l.page[1], ret, l.mmu_idx, access_type, ra); + ret = do_ld_beN(env, &l.page[0], 0, l.mmu_idx, access_type, l.memop, ra); + ret = do_ld_beN(env, &l.page[1], ret, l.mmu_idx, access_type, l.memop, ra); if ((l.memop & MO_BSWAP) == MO_LE) { ret = bswap64(ret); } diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index fc597a010d..fefc83cc8c 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -931,6 +931,8 @@ static void *cpu_mmu_lookup(CPUArchState *env, target_ulong addr, return ret; } +#include "ldst_atomicity.c.inc" + uint8_t cpu_ldb_mmu(CPUArchState *env, abi_ptr addr, MemOpIdx oi, uintptr_t ra) { @@ -953,10 +955,10 @@ uint16_t cpu_ldw_be_mmu(CPUArchState *env, abi_ptr addr, validate_memop(oi, MO_BEUW); haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD); - ret = lduw_be_p(haddr); + ret = load_atom_2(env, ra, haddr, get_memop(oi)); clear_helper_retaddr(); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return ret; + return cpu_to_be16(ret); } uint32_t cpu_ldl_be_mmu(CPUArchState *env, abi_ptr addr, @@ -967,10 +969,10 @@ uint32_t cpu_ldl_be_mmu(CPUArchState *env, abi_ptr addr, validate_memop(oi, MO_BEUL); haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD); - ret = ldl_be_p(haddr); + ret = load_atom_4(env, ra, haddr, get_memop(oi)); clear_helper_retaddr(); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return ret; + return cpu_to_be32(ret); } uint64_t cpu_ldq_be_mmu(CPUArchState *env, abi_ptr addr, @@ -981,10 +983,10 @@ uint64_t cpu_ldq_be_mmu(CPUArchState *env, abi_ptr addr, validate_memop(oi, MO_BEUQ); haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD); - ret = ldq_be_p(haddr); + ret = load_atom_8(env, ra, haddr, get_memop(oi)); clear_helper_retaddr(); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return ret; + return cpu_to_be64(ret); } uint16_t cpu_ldw_le_mmu(CPUArchState *env, abi_ptr addr, @@ -995,10 +997,10 @@ uint16_t cpu_ldw_le_mmu(CPUArchState *env, abi_ptr addr, validate_memop(oi, MO_LEUW); haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD); - ret = lduw_le_p(haddr); + ret = load_atom_2(env, ra, haddr, get_memop(oi)); clear_helper_retaddr(); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return ret; + return cpu_to_le16(ret); } uint32_t cpu_ldl_le_mmu(CPUArchState *env, abi_ptr addr, @@ -1009,10 +1011,10 @@ uint32_t cpu_ldl_le_mmu(CPUArchState *env, abi_ptr addr, validate_memop(oi, MO_LEUL); haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD); - ret = ldl_le_p(haddr); + ret = load_atom_4(env, ra, haddr, get_memop(oi)); clear_helper_retaddr(); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return ret; + return cpu_to_le32(ret); } uint64_t cpu_ldq_le_mmu(CPUArchState *env, abi_ptr addr, @@ -1023,10 +1025,10 @@ uint64_t cpu_ldq_le_mmu(CPUArchState *env, abi_ptr addr, validate_memop(oi, MO_LEUQ); haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD); - ret = ldq_le_p(haddr); + ret = load_atom_8(env, ra, haddr, get_memop(oi)); clear_helper_retaddr(); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return ret; + return cpu_to_le64(ret); } Int128 cpu_ld16_be_mmu(CPUArchState *env, abi_ptr addr, diff --git a/accel/tcg/ldst_atomicity.c.inc b/accel/tcg/ldst_atomicity.c.inc new file mode 100644 index 0000000000..a5bce641f8 --- /dev/null +++ b/accel/tcg/ldst_atomicity.c.inc @@ -0,0 +1,566 @@ +/* + * Routines common to user and system emulation of load/store. + * + * Copyright (c) 2022 Linaro, Ltd. + * + * SPDX-License-Identifier: GPL-2.0-or-later + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifdef CONFIG_ATOMIC64 +# define HAVE_al8 true +#else +# define HAVE_al8 false +#endif +#define HAVE_al8_fast (ATOMIC_REG_SIZE >= 8) + +#if defined(CONFIG_ATOMIC128) +# define HAVE_al16_fast true +#else +# define HAVE_al16_fast false +#endif + +/** + * required_atomicity: + * + * Return the lg2 bytes of atomicity required by @memop for @p. + * If the operation must be split into two operations to be + * examined separately for atomicity, return -lg2. + */ +static int required_atomicity(CPUArchState *env, uintptr_t p, MemOp memop) +{ + MemOp atom = memop & MO_ATOM_MASK; + MemOp size = memop & MO_SIZE; + MemOp half = size ? size - 1 : 0; + unsigned tmp; + int atmax; + + switch (atom) { + case MO_ATOM_NONE: + atmax = MO_8; + break; + + case MO_ATOM_IFALIGN_PAIR: + size = half; + /* fall through */ + + case MO_ATOM_IFALIGN: + tmp = (1 << size) - 1; + atmax = p & tmp ? MO_8 : size; + break; + + case MO_ATOM_WITHIN16: + tmp = p & 15; + atmax = (tmp + (1 << size) <= 16 ? size : MO_8); + break; + + case MO_ATOM_WITHIN16_PAIR: + tmp = p & 15; + if (tmp + (1 << size) <= 16) { + atmax = size; + } else if (tmp + (1 << half) == 16) { + /* + * The pair exactly straddles the boundary. + * Both halves are naturally aligned and atomic. + */ + atmax = half; + } else { + /* + * One of the pair crosses the boundary, and is non-atomic. + * The other of the pair does not cross, and is atomic. + */ + atmax = -half; + } + break; + + case MO_ATOM_SUBALIGN: + /* + * Examine the alignment of p to determine if there are subobjects + * that must be aligned. Note that we only really need ctz4() -- + * any more sigificant bits are discarded by the immediately + * following comparison. + */ + tmp = ctz32(p); + atmax = MIN(size, tmp); + break; + + default: + g_assert_not_reached(); + } + + /* + * Here we have the architectural atomicity of the operation. + * However, when executing in a serial context, we need no extra + * host atomicity in order to avoid racing. This reduction + * avoids looping with cpu_loop_exit_atomic. + */ + if (cpu_in_serial_context(env_cpu(env))) { + return MO_8; + } + return atmax; +} + +/** + * load_atomic2: + * @pv: host address + * + * Atomically load 2 aligned bytes from @pv. + */ +static inline uint16_t load_atomic2(void *pv) +{ + uint16_t *p = __builtin_assume_aligned(pv, 2); + return qatomic_read(p); +} + +/** + * load_atomic4: + * @pv: host address + * + * Atomically load 4 aligned bytes from @pv. + */ +static inline uint32_t load_atomic4(void *pv) +{ + uint32_t *p = __builtin_assume_aligned(pv, 4); + return qatomic_read(p); +} + +/** + * load_atomic8: + * @pv: host address + * + * Atomically load 8 aligned bytes from @pv. + */ +static inline uint64_t load_atomic8(void *pv) +{ + uint64_t *p = __builtin_assume_aligned(pv, 8); + + qemu_build_assert(HAVE_al8); + return qatomic_read__nocheck(p); +} + +/** + * load_atomic16: + * @pv: host address + * + * Atomically load 16 aligned bytes from @pv. + */ +static inline Int128 load_atomic16(void *pv) +{ +#ifdef CONFIG_ATOMIC128 + __uint128_t *p = __builtin_assume_aligned(pv, 16); + Int128Alias r; + + r.u = qatomic_read__nocheck(p); + return r.s; +#else + qemu_build_not_reached(); +#endif +} + +/** + * load_atomic8_or_exit: + * @env: cpu context + * @ra: host unwind address + * @pv: host address + * + * Atomically load 8 aligned bytes from @pv. + * If this is not possible, longjmp out to restart serially. + */ +static uint64_t load_atomic8_or_exit(CPUArchState *env, uintptr_t ra, void *pv) +{ + if (HAVE_al8) { + return load_atomic8(pv); + } + +#ifdef CONFIG_USER_ONLY + /* + * If the page is not writable, then assume the value is immutable + * and requires no locking. This ignores the case of MAP_SHARED with + * another process, because the fallback start_exclusive solution + * provides no protection across processes. + */ + if (!page_check_range(h2g(pv), 8, PAGE_WRITE)) { + uint64_t *p = __builtin_assume_aligned(pv, 8); + return *p; + } +#endif + + /* Ultimate fallback: re-execute in serial context. */ + cpu_loop_exit_atomic(env_cpu(env), ra); +} + +/** + * load_atomic16_or_exit: + * @env: cpu context + * @ra: host unwind address + * @pv: host address + * + * Atomically load 16 aligned bytes from @pv. + * If this is not possible, longjmp out to restart serially. + */ +static Int128 load_atomic16_or_exit(CPUArchState *env, uintptr_t ra, void *pv) +{ + Int128 *p = __builtin_assume_aligned(pv, 16); + + if (HAVE_al16_fast) { + return load_atomic16(p); + } + +#ifdef CONFIG_USER_ONLY + /* + * We can only use cmpxchg to emulate a load if the page is writable. + * If the page is not writable, then assume the value is immutable + * and requires no locking. This ignores the case of MAP_SHARED with + * another process, because the fallback start_exclusive solution + * provides no protection across processes. + */ + if (!page_check_range(h2g(p), 16, PAGE_WRITE)) { + return *p; + } +#endif + + /* + * In system mode all guest pages are writable, and for user-only + * we have just checked writability. Try cmpxchg. + */ +#if defined(CONFIG_CMPXCHG128) + /* Swap 0 with 0, with the side-effect of returning the old value. */ + { + Int128Alias r; + r.u = __sync_val_compare_and_swap_16((__uint128_t *)p, 0, 0); + return r.s; + } +#endif + + /* Ultimate fallback: re-execute in serial context. */ + cpu_loop_exit_atomic(env_cpu(env), ra); +} + +/** + * load_atom_extract_al4x2: + * @pv: host address + * + * Load 4 bytes from @p, from two sequential atomic 4-byte loads. + */ +static uint32_t load_atom_extract_al4x2(void *pv) +{ + uintptr_t pi = (uintptr_t)pv; + int sh = (pi & 3) * 8; + uint32_t a, b; + + pv = (void *)(pi & ~3); + a = load_atomic4(pv); + b = load_atomic4(pv + 4); + + if (HOST_BIG_ENDIAN) { + return (a << sh) | (b >> (-sh & 31)); + } else { + return (a >> sh) | (b << (-sh & 31)); + } +} + +/** + * load_atom_extract_al8x2: + * @pv: host address + * + * Load 8 bytes from @p, from two sequential atomic 8-byte loads. + */ +static uint64_t load_atom_extract_al8x2(void *pv) +{ + uintptr_t pi = (uintptr_t)pv; + int sh = (pi & 7) * 8; + uint64_t a, b; + + pv = (void *)(pi & ~7); + a = load_atomic8(pv); + b = load_atomic8(pv + 8); + + if (HOST_BIG_ENDIAN) { + return (a << sh) | (b >> (-sh & 63)); + } else { + return (a >> sh) | (b << (-sh & 63)); + } +} + +/** + * load_atom_extract_al8_or_exit: + * @env: cpu context + * @ra: host unwind address + * @pv: host address + * @s: object size in bytes, @s <= 4. + * + * Atomically load @s bytes from @p, when p % s != 0, and [p, p+s-1] does + * not cross an 8-byte boundary. This means that we can perform an atomic + * 8-byte load and extract. + * The value is returned in the low bits of a uint32_t. + */ +static uint32_t load_atom_extract_al8_or_exit(CPUArchState *env, uintptr_t ra, + void *pv, int s) +{ + uintptr_t pi = (uintptr_t)pv; + int o = pi & 7; + int shr = (HOST_BIG_ENDIAN ? 8 - s - o : o) * 8; + + pv = (void *)(pi & ~7); + return load_atomic8_or_exit(env, ra, pv) >> shr; +} + +/** + * load_atom_extract_al16_or_exit: + * @env: cpu context + * @ra: host unwind address + * @p: host address + * @s: object size in bytes, @s <= 8. + * + * Atomically load @s bytes from @p, when p % 16 < 8 + * and p % 16 + s > 8. I.e. does not cross a 16-byte + * boundary, but *does* cross an 8-byte boundary. + * This is the slow version, so we must have eliminated + * any faster load_atom_extract_al8_or_exit case. + * + * If this is not possible, longjmp out to restart serially. + */ +static uint64_t load_atom_extract_al16_or_exit(CPUArchState *env, uintptr_t ra, + void *pv, int s) +{ + uintptr_t pi = (uintptr_t)pv; + int o = pi & 7; + int shr = (HOST_BIG_ENDIAN ? 16 - s - o : o) * 8; + Int128 r; + + /* + * Note constraints above: p & 8 must be clear. + * Provoke SIGBUS if possible otherwise. + */ + pv = (void *)(pi & ~7); + r = load_atomic16_or_exit(env, ra, pv); + + r = int128_urshift(r, shr); + return int128_getlo(r); +} + +/** + * load_atom_extract_al16_or_al8: + * @p: host address + * @s: object size in bytes, @s <= 8. + * + * Load @s bytes from @p, when p % s != 0. If [p, p+s-1] does not + * cross an 16-byte boundary then the access must be 16-byte atomic, + * otherwise the access must be 8-byte atomic. + */ +static inline uint64_t load_atom_extract_al16_or_al8(void *pv, int s) +{ +#if defined(CONFIG_ATOMIC128) + uintptr_t pi = (uintptr_t)pv; + int o = pi & 7; + int shr = (HOST_BIG_ENDIAN ? 16 - s - o : o) * 8; + __uint128_t r; + + pv = (void *)(pi & ~7); + if (pi & 8) { + uint64_t *p8 = __builtin_assume_aligned(pv, 16, 8); + uint64_t a = qatomic_read__nocheck(p8); + uint64_t b = qatomic_read__nocheck(p8 + 1); + + if (HOST_BIG_ENDIAN) { + r = ((__uint128_t)a << 64) | b; + } else { + r = ((__uint128_t)b << 64) | a; + } + } else { + __uint128_t *p16 = __builtin_assume_aligned(pv, 16, 0); + r = qatomic_read__nocheck(p16); + } + return r >> shr; +#else + qemu_build_not_reached(); +#endif +} + +/** + * load_atom_4_by_2: + * @pv: host address + * + * Load 4 bytes from @pv, with two 2-byte atomic loads. + */ +static inline uint32_t load_atom_4_by_2(void *pv) +{ + uint32_t a = load_atomic2(pv); + uint32_t b = load_atomic2(pv + 2); + + if (HOST_BIG_ENDIAN) { + return (a << 16) | b; + } else { + return (b << 16) | a; + } +} + +/** + * load_atom_8_by_2: + * @pv: host address + * + * Load 8 bytes from @pv, with four 2-byte atomic loads. + */ +static inline uint64_t load_atom_8_by_2(void *pv) +{ + uint32_t a = load_atom_4_by_2(pv); + uint32_t b = load_atom_4_by_2(pv + 4); + + if (HOST_BIG_ENDIAN) { + return ((uint64_t)a << 32) | b; + } else { + return ((uint64_t)b << 32) | a; + } +} + +/** + * load_atom_8_by_4: + * @pv: host address + * + * Load 8 bytes from @pv, with two 4-byte atomic loads. + */ +static inline uint64_t load_atom_8_by_4(void *pv) +{ + uint32_t a = load_atomic4(pv); + uint32_t b = load_atomic4(pv + 4); + + if (HOST_BIG_ENDIAN) { + return ((uint64_t)a << 32) | b; + } else { + return ((uint64_t)b << 32) | a; + } +} + +/** + * load_atom_2: + * @p: host address + * @memop: the full memory op + * + * Load 2 bytes from @p, honoring the atomicity of @memop. + */ +static uint16_t load_atom_2(CPUArchState *env, uintptr_t ra, + void *pv, MemOp memop) +{ + uintptr_t pi = (uintptr_t)pv; + int atmax; + + if (likely((pi & 1) == 0)) { + return load_atomic2(pv); + } + if (HAVE_al16_fast) { + return load_atom_extract_al16_or_al8(pv, 2); + } + + atmax = required_atomicity(env, pi, memop); + switch (atmax) { + case MO_8: + return lduw_he_p(pv); + case MO_16: + /* The only case remaining is MO_ATOM_WITHIN16. */ + if (!HAVE_al8_fast && (pi & 3) == 1) { + /* Big or little endian, we want the middle two bytes. */ + return load_atomic4(pv - 1) >> 8; + } + if ((pi & 15) != 7) { + return load_atom_extract_al8_or_exit(env, ra, pv, 2); + } + return load_atom_extract_al16_or_exit(env, ra, pv, 2); + default: + g_assert_not_reached(); + } +} + +/** + * load_atom_4: + * @p: host address + * @memop: the full memory op + * + * Load 4 bytes from @p, honoring the atomicity of @memop. + */ +static uint32_t load_atom_4(CPUArchState *env, uintptr_t ra, + void *pv, MemOp memop) +{ + uintptr_t pi = (uintptr_t)pv; + int atmax; + + if (likely((pi & 3) == 0)) { + return load_atomic4(pv); + } + if (HAVE_al16_fast) { + return load_atom_extract_al16_or_al8(pv, 4); + } + + atmax = required_atomicity(env, pi, memop); + switch (atmax) { + case MO_8: + case MO_16: + case -MO_16: + /* + * For MO_ATOM_IFALIGN, this is more atomicity than required, + * but it's trivially supported on all hosts, better than 4 + * individual byte loads (when the host requires alignment), + * and overlaps with the MO_ATOM_SUBALIGN case of p % 2 == 0. + */ + return load_atom_extract_al4x2(pv); + case MO_32: + if (!(pi & 4)) { + return load_atom_extract_al8_or_exit(env, ra, pv, 4); + } + return load_atom_extract_al16_or_exit(env, ra, pv, 4); + default: + g_assert_not_reached(); + } +} + +/** + * load_atom_8: + * @p: host address + * @memop: the full memory op + * + * Load 8 bytes from @p, honoring the atomicity of @memop. + */ +static uint64_t load_atom_8(CPUArchState *env, uintptr_t ra, + void *pv, MemOp memop) +{ + uintptr_t pi = (uintptr_t)pv; + int atmax; + + /* + * If the host does not support 8-byte atomics, wait until we have + * examined the atomicity parameters below. + */ + if (HAVE_al8 && likely((pi & 7) == 0)) { + return load_atomic8(pv); + } + if (HAVE_al16_fast) { + return load_atom_extract_al16_or_al8(pv, 8); + } + + atmax = required_atomicity(env, pi, memop); + if (atmax == MO_64) { + if (!HAVE_al8 && (pi & 7) == 0) { + load_atomic8_or_exit(env, ra, pv); + } + return load_atom_extract_al16_or_exit(env, ra, pv, 8); + } + if (HAVE_al8_fast) { + return load_atom_extract_al8x2(pv); + } + switch (atmax) { + case MO_8: + return ldq_he_p(pv); + case MO_16: + return load_atom_8_by_2(pv); + case MO_32: + return load_atom_8_by_4(pv); + case -MO_32: + if (HAVE_al8) { + return load_atom_extract_al8x2(pv); + } + cpu_loop_exit_atomic(env_cpu(env), ra); + default: + g_assert_not_reached(); + } +} From patchwork Tue May 16 19:40:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682362 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp638886ltp; Tue, 16 May 2023 12:43:13 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6f/SDpSUXr7gOfIhWTWKlTt7DuCyNuXdRJIX9qcPPfMNX7sjCLzGWL8kH55ms3js7raw9p X-Received: by 2002:a05:622a:1743:b0:3f5:4da8:1a92 with SMTP id l3-20020a05622a174300b003f54da81a92mr2519841qtk.35.1684266193345; Tue, 16 May 2023 12:43:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266193; cv=none; d=google.com; s=arc-20160816; b=Peqtp0SxS/Odd1QI60ky8Xpc3MIewPjKtWfvFbesllA9/ThvOkk+bNWlXroUT+fS2T lODcvFe22h/RZ7ZKKQkUvP7C7bRAiVyNouKd5zpIp6KPM7dhIEIyV9m6WBtMui1+lgl4 rsYVbqec4t68ZtWXJkHcAW0nJ/hzltuP70/EHCxPuOk0U7VzGDu9qrv90ITj7ZcQ5drh 3UtcMrJ8bHPyxW2y5FpA0rWm8hRyTTFedJuBY9a4JzuvnujoG4PxtyluvI4DVQut8iZ7 nalMsKy+Tnlr4evxW7hoTIziP+WJawRYGT8Sgm0BS2BnHCl6mkLJKgFuJYNNa2vyQ48f O8gA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=OKbilkd0PwAQW9asTHpPZblhMxzcNmkGSyUJS+Fmjz4=; b=zjQrKQGWaQbxJYU/56EpExX4DUHIV7d87/ioOLVr+3MflA4wcgTRz5X4wB2UtxbuzW MqIEqloLuHwH5WAlIe4z6sMzGdjQ6Rjug9Abd+ZknAI6Hk7ZWHt9ez4U6CET6WqV9aiD gyZd071ygcpv5Waog+b8v6cG200F6jpE1Xq5CzFdlM24BH9Qg605zD0Du0/BmSsHLIwi 7gMxJELQ+Mlxuu1WbjktjXU0xMaYX5b59x7Tydf+DG/uOopPFQIra1hL0ps/KZ8FfiJw GusX06KyBdnLVH9wuxUup+VIrNunx64fUUUfKm5Cj+4eRhK4hgfP7rdQqj+T1eTq2QA8 ya+A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Q6XI5eLt; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b12-20020a05622a020c00b003ef57ffc9f9si12328337qtx.17.2023.05.16.12.43.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:43:13 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Q6XI5eLt; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Ya-0006dE-Lp; Tue, 16 May 2023 15:42:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0YV-0006Yt-Iy for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:55 -0400 Received: from mail-pf1-x431.google.com ([2607:f8b0:4864:20::431]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0YS-0002jf-C1 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:55 -0400 Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-643a1fed360so9087754b3a.3 for ; Tue, 16 May 2023 12:41:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266111; x=1686858111; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OKbilkd0PwAQW9asTHpPZblhMxzcNmkGSyUJS+Fmjz4=; b=Q6XI5eLtQzDB0NZGG9feTHsC8dt3v9I32bsgl4oKp6CUwJzQBLMlMQ7xxTdsOlWlhh ZJ4i4x0SV/d80xIPfVv5j36vAf+xehMW8nWBHpph+L8qdCZeFuJSNh3F/AGPttRQChnq 6yMRJqUJiIb5lUQg/g/KSsyJ5BOoJAAXUBI1AIQCYEZwyhRDJgF6I2tsF44pLgpUo7fR 1JX1vuL+ecIGAWwkrRSx2jUPVWMk71L2Fs6uHvb75EXb0ZGzl6YIrzNTQ8jdzbtE3gCp S2cHLGEGvJBNMCFzz9feQMRPPqwqv9cPkQxjbOiRF/dw0fLsb7IPehsFIsstTkkEvLcX hPWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266111; x=1686858111; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OKbilkd0PwAQW9asTHpPZblhMxzcNmkGSyUJS+Fmjz4=; b=iOMEIyf2rOX2BrzbAnx+lkox4W1iO+j/L0WSO1IiqWjigdxXyT0DbH6+HHad5OYV82 4r0corFSvNJZDNgKMDXZD6Hfjuj1hY3X2O+uFrdjwGidv2rCmyBShCVTE8GYcaz1lTpI ysk4fzEkL6LT0gvPRIKUQxLxH7iks81U+0TTx2K2Ki+ie8o7QYQ1y5belQIsZc2J2t2a unj5ZyOclBoZPC7VzpS4RNtgna+j4L1bXjcDywzs4racTM20jYYKu+XMDrkpmZcBka3u mNKwvbwd6alOIS6psEiTreKL2TdGPw9wkpnatkhASoPLp/wKWmslLZH4XD+vbPhUKaMn HTAg== X-Gm-Message-State: AC+VfDxLbhv+JaC6mlNradVhmxISX/yoca8HzgIRzfkq0UcAKnnDY+hX DGFdMy7qd/tJuW5nheOgGMvOLA2aGqY9IaSycBk= X-Received: by 2002:a05:6a00:21d0:b0:644:8172:3ea9 with SMTP id t16-20020a056a0021d000b0064481723ea9mr42263062pfj.15.1684266110913; Tue, 16 May 2023 12:41:50 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.41.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:41:50 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 04/80] accel/tcg: Honor atomicity of stores Date: Tue, 16 May 2023 12:40:29 -0700 Message-Id: <20230516194145.1749305-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::431; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x431.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 108 ++++---- accel/tcg/user-exec.c | 12 +- accel/tcg/ldst_atomicity.c.inc | 491 +++++++++++++++++++++++++++++++++ 3 files changed, 545 insertions(+), 66 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 33e75ae962..d910464c36 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -2599,36 +2599,6 @@ Int128 cpu_ld16_le_mmu(CPUArchState *env, abi_ptr addr, * Store Helpers */ -static inline void QEMU_ALWAYS_INLINE -store_memop(void *haddr, uint64_t val, MemOp op) -{ - switch (op) { - case MO_UB: - stb_p(haddr, val); - break; - case MO_BEUW: - stw_be_p(haddr, val); - break; - case MO_LEUW: - stw_le_p(haddr, val); - break; - case MO_BEUL: - stl_be_p(haddr, val); - break; - case MO_LEUL: - stl_le_p(haddr, val); - break; - case MO_BEUQ: - stq_be_p(haddr, val); - break; - case MO_LEUQ: - stq_le_p(haddr, val); - break; - default: - qemu_build_not_reached(); - } -} - /** * do_st_mmio_leN: * @env: cpu context @@ -2655,38 +2625,56 @@ static uint64_t do_st_mmio_leN(CPUArchState *env, MMULookupPageData *p, return val_le; } -/** - * do_st_bytes_leN: - * @p: translation parameters - * @val_le: data to store - * - * Store @p->size bytes at @p->haddr, which is RAM. - * The bytes to store are extracted in little-endian order from @val_le; - * return the bytes of @val_le beyond @p->size that have not been stored. - */ -static uint64_t do_st_bytes_leN(MMULookupPageData *p, uint64_t val_le) -{ - uint8_t *haddr = p->haddr; - int i, size = p->size; - - for (i = 0; i < size; i++, val_le >>= 8) { - haddr[i] = val_le; - } - return val_le; -} - /* * Wrapper for the above. */ static uint64_t do_st_leN(CPUArchState *env, MMULookupPageData *p, - uint64_t val_le, int mmu_idx, uintptr_t ra) + uint64_t val_le, int mmu_idx, + MemOp mop, uintptr_t ra) { + MemOp atom; + unsigned tmp, half_size; + if (unlikely(p->flags & TLB_MMIO)) { return do_st_mmio_leN(env, p, val_le, mmu_idx, ra); } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { return val_le >> (p->size * 8); - } else { - return do_st_bytes_leN(p, val_le); + } + + /* + * It is a given that we cross a page and therefore there is no atomicity + * for the store as a whole, but subobjects may need attention. + */ + atom = mop & MO_ATOM_MASK; + switch (atom) { + case MO_ATOM_SUBALIGN: + return store_parts_leN(p->haddr, p->size, val_le); + + case MO_ATOM_IFALIGN_PAIR: + case MO_ATOM_WITHIN16_PAIR: + tmp = mop & MO_SIZE; + tmp = tmp ? tmp - 1 : 0; + half_size = 1 << tmp; + if (atom == MO_ATOM_IFALIGN_PAIR + ? p->size == half_size + : p->size >= half_size) { + if (!HAVE_al8_fast && p->size <= 4) { + return store_whole_le4(p->haddr, p->size, val_le); + } else if (HAVE_al8) { + return store_whole_le8(p->haddr, p->size, val_le); + } else { + cpu_loop_exit_atomic(env_cpu(env), ra); + } + } + /* fall through */ + + case MO_ATOM_IFALIGN: + case MO_ATOM_WITHIN16: + case MO_ATOM_NONE: + return store_bytes_leN(p->haddr, p->size, val_le); + + default: + g_assert_not_reached(); } } @@ -2714,7 +2702,7 @@ static void do_st_2(CPUArchState *env, MMULookupPageData *p, uint16_t val, if (memop & MO_BSWAP) { val = bswap16(val); } - store_memop(p->haddr, val, MO_UW); + store_atom_2(env, ra, p->haddr, memop, val); } } @@ -2730,7 +2718,7 @@ static void do_st_4(CPUArchState *env, MMULookupPageData *p, uint32_t val, if (memop & MO_BSWAP) { val = bswap32(val); } - store_memop(p->haddr, val, MO_UL); + store_atom_4(env, ra, p->haddr, memop, val); } } @@ -2746,7 +2734,7 @@ static void do_st_8(CPUArchState *env, MMULookupPageData *p, uint64_t val, if (memop & MO_BSWAP) { val = bswap64(val); } - store_memop(p->haddr, val, MO_UQ); + store_atom_8(env, ra, p->haddr, memop, val); } } @@ -2815,8 +2803,8 @@ static void do_st4_mmu(CPUArchState *env, target_ulong addr, uint32_t val, if ((l.memop & MO_BSWAP) != MO_LE) { val = bswap32(val); } - val = do_st_leN(env, &l.page[0], val, l.mmu_idx, ra); - (void) do_st_leN(env, &l.page[1], val, l.mmu_idx, ra); + val = do_st_leN(env, &l.page[0], val, l.mmu_idx, l.memop, ra); + (void) do_st_leN(env, &l.page[1], val, l.mmu_idx, l.memop, ra); } void helper_le_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, @@ -2849,8 +2837,8 @@ static void do_st8_mmu(CPUArchState *env, target_ulong addr, uint64_t val, if ((l.memop & MO_BSWAP) != MO_LE) { val = bswap64(val); } - val = do_st_leN(env, &l.page[0], val, l.mmu_idx, ra); - (void) do_st_leN(env, &l.page[1], val, l.mmu_idx, ra); + val = do_st_leN(env, &l.page[0], val, l.mmu_idx, l.memop, ra); + (void) do_st_leN(env, &l.page[1], val, l.mmu_idx, l.memop, ra); } void helper_le_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index fefc83cc8c..b89fa35a83 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -1086,7 +1086,7 @@ void cpu_stw_be_mmu(CPUArchState *env, abi_ptr addr, uint16_t val, validate_memop(oi, MO_BEUW); haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE); - stw_be_p(haddr, val); + store_atom_2(env, ra, haddr, get_memop(oi), be16_to_cpu(val)); clear_helper_retaddr(); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } @@ -1098,7 +1098,7 @@ void cpu_stl_be_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, validate_memop(oi, MO_BEUL); haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE); - stl_be_p(haddr, val); + store_atom_4(env, ra, haddr, get_memop(oi), be32_to_cpu(val)); clear_helper_retaddr(); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } @@ -1110,7 +1110,7 @@ void cpu_stq_be_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, validate_memop(oi, MO_BEUQ); haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE); - stq_be_p(haddr, val); + store_atom_8(env, ra, haddr, get_memop(oi), be64_to_cpu(val)); clear_helper_retaddr(); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } @@ -1122,7 +1122,7 @@ void cpu_stw_le_mmu(CPUArchState *env, abi_ptr addr, uint16_t val, validate_memop(oi, MO_LEUW); haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE); - stw_le_p(haddr, val); + store_atom_2(env, ra, haddr, get_memop(oi), le16_to_cpu(val)); clear_helper_retaddr(); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } @@ -1134,7 +1134,7 @@ void cpu_stl_le_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, validate_memop(oi, MO_LEUL); haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE); - stl_le_p(haddr, val); + store_atom_4(env, ra, haddr, get_memop(oi), le32_to_cpu(val)); clear_helper_retaddr(); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } @@ -1146,7 +1146,7 @@ void cpu_stq_le_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, validate_memop(oi, MO_LEUQ); haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE); - stq_le_p(haddr, val); + store_atom_8(env, ra, haddr, get_memop(oi), le64_to_cpu(val)); clear_helper_retaddr(); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } diff --git a/accel/tcg/ldst_atomicity.c.inc b/accel/tcg/ldst_atomicity.c.inc index a5bce641f8..1f39e43896 100644 --- a/accel/tcg/ldst_atomicity.c.inc +++ b/accel/tcg/ldst_atomicity.c.inc @@ -21,6 +21,12 @@ #else # define HAVE_al16_fast false #endif +#if defined(CONFIG_ATOMIC128) || defined(CONFIG_CMPXCHG128) +# define HAVE_al16 true +#else +# define HAVE_al16 false +#endif + /** * required_atomicity: @@ -564,3 +570,488 @@ static uint64_t load_atom_8(CPUArchState *env, uintptr_t ra, g_assert_not_reached(); } } + +/** + * store_atomic2: + * @pv: host address + * @val: value to store + * + * Atomically store 2 aligned bytes to @pv. + */ +static inline void store_atomic2(void *pv, uint16_t val) +{ + uint16_t *p = __builtin_assume_aligned(pv, 2); + qatomic_set(p, val); +} + +/** + * store_atomic4: + * @pv: host address + * @val: value to store + * + * Atomically store 4 aligned bytes to @pv. + */ +static inline void store_atomic4(void *pv, uint32_t val) +{ + uint32_t *p = __builtin_assume_aligned(pv, 4); + qatomic_set(p, val); +} + +/** + * store_atomic8: + * @pv: host address + * @val: value to store + * + * Atomically store 8 aligned bytes to @pv. + */ +static inline void store_atomic8(void *pv, uint64_t val) +{ + uint64_t *p = __builtin_assume_aligned(pv, 8); + + qemu_build_assert(HAVE_al8); + qatomic_set__nocheck(p, val); +} + +/** + * store_atom_4x2 + */ +static inline void store_atom_4_by_2(void *pv, uint32_t val) +{ + store_atomic2(pv, val >> (HOST_BIG_ENDIAN ? 16 : 0)); + store_atomic2(pv + 2, val >> (HOST_BIG_ENDIAN ? 0 : 16)); +} + +/** + * store_atom_8_by_2 + */ +static inline void store_atom_8_by_2(void *pv, uint64_t val) +{ + store_atom_4_by_2(pv, val >> (HOST_BIG_ENDIAN ? 32 : 0)); + store_atom_4_by_2(pv + 4, val >> (HOST_BIG_ENDIAN ? 0 : 32)); +} + +/** + * store_atom_8_by_4 + */ +static inline void store_atom_8_by_4(void *pv, uint64_t val) +{ + store_atomic4(pv, val >> (HOST_BIG_ENDIAN ? 32 : 0)); + store_atomic4(pv + 4, val >> (HOST_BIG_ENDIAN ? 0 : 32)); +} + +/** + * store_atom_insert_al4: + * @p: host address + * @val: shifted value to store + * @msk: mask for value to store + * + * Atomically store @val to @p, masked by @msk. + */ +static void store_atom_insert_al4(uint32_t *p, uint32_t val, uint32_t msk) +{ + uint32_t old, new; + + p = __builtin_assume_aligned(p, 4); + old = qatomic_read(p); + do { + new = (old & ~msk) | val; + } while (!__atomic_compare_exchange_n(p, &old, new, true, + __ATOMIC_RELAXED, __ATOMIC_RELAXED)); +} + +/** + * store_atom_insert_al8: + * @p: host address + * @val: shifted value to store + * @msk: mask for value to store + * + * Atomically store @val to @p masked by @msk. + */ +static void store_atom_insert_al8(uint64_t *p, uint64_t val, uint64_t msk) +{ + uint64_t old, new; + + qemu_build_assert(HAVE_al8); + p = __builtin_assume_aligned(p, 8); + old = qatomic_read__nocheck(p); + do { + new = (old & ~msk) | val; + } while (!__atomic_compare_exchange_n(p, &old, new, true, + __ATOMIC_RELAXED, __ATOMIC_RELAXED)); +} + +/** + * store_atom_insert_al16: + * @p: host address + * @val: shifted value to store + * @msk: mask for value to store + * + * Atomically store @val to @p masked by @msk. + */ +static void store_atom_insert_al16(Int128 *ps, Int128Alias val, Int128Alias msk) +{ +#if defined(CONFIG_ATOMIC128) + __uint128_t *pu, old, new; + + /* With CONFIG_ATOMIC128, we can avoid the memory barriers. */ + pu = __builtin_assume_aligned(ps, 16); + old = *pu; + do { + new = (old & ~msk.u) | val.u; + } while (!__atomic_compare_exchange_n(pu, &old, new, true, + __ATOMIC_RELAXED, __ATOMIC_RELAXED)); +#elif defined(CONFIG_CMPXCHG128) + __uint128_t *pu, old, new; + + /* + * Without CONFIG_ATOMIC128, __atomic_compare_exchange_n will always + * defer to libatomic, so we must use __sync_*_compare_and_swap_16 + * and accept the sequential consistency that comes with it. + */ + pu = __builtin_assume_aligned(ps, 16); + do { + old = *pu; + new = (old & ~msk.u) | val.u; + } while (!__sync_bool_compare_and_swap_16(pu, old, new)); +#else + qemu_build_not_reached(); +#endif +} + +/** + * store_bytes_leN: + * @pv: host address + * @size: number of bytes to store + * @val_le: data to store + * + * Store @size bytes at @p. The bytes to store are extracted in little-endian order + * from @val_le; return the bytes of @val_le beyond @size that have not been stored. + */ +static uint64_t store_bytes_leN(void *pv, int size, uint64_t val_le) +{ + uint8_t *p = pv; + for (int i = 0; i < size; i++, val_le >>= 8) { + p[i] = val_le; + } + return val_le; +} + +/** + * store_parts_leN + * @pv: host address + * @size: number of bytes to store + * @val_le: data to store + * + * As store_bytes_leN, but atomically on each aligned part. + */ +G_GNUC_UNUSED +static uint64_t store_parts_leN(void *pv, int size, uint64_t val_le) +{ + do { + int n; + + /* Find minimum of alignment and size */ + switch (((uintptr_t)pv | size) & 7) { + case 4: + store_atomic4(pv, le32_to_cpu(val_le)); + val_le >>= 32; + n = 4; + break; + case 2: + case 6: + store_atomic2(pv, le16_to_cpu(val_le)); + val_le >>= 16; + n = 2; + break; + default: + *(uint8_t *)pv = val_le; + val_le >>= 8; + n = 1; + break; + case 0: + g_assert_not_reached(); + } + pv += n; + size -= n; + } while (size != 0); + + return val_le; +} + +/** + * store_whole_le4 + * @pv: host address + * @size: number of bytes to store + * @val_le: data to store + * + * As store_bytes_leN, but atomically as a whole. + * Four aligned bytes are guaranteed to cover the store. + */ +static uint64_t store_whole_le4(void *pv, int size, uint64_t val_le) +{ + int sz = size * 8; + int o = (uintptr_t)pv & 3; + int sh = o * 8; + uint32_t m = MAKE_64BIT_MASK(0, sz); + uint32_t v; + + if (HOST_BIG_ENDIAN) { + v = bswap32(val_le) >> sh; + m = bswap32(m) >> sh; + } else { + v = val_le << sh; + m <<= sh; + } + store_atom_insert_al4(pv - o, v, m); + return val_le >> sz; +} + +/** + * store_whole_le8 + * @pv: host address + * @size: number of bytes to store + * @val_le: data to store + * + * As store_bytes_leN, but atomically as a whole. + * Eight aligned bytes are guaranteed to cover the store. + */ +static uint64_t store_whole_le8(void *pv, int size, uint64_t val_le) +{ + int sz = size * 8; + int o = (uintptr_t)pv & 7; + int sh = o * 8; + uint64_t m = MAKE_64BIT_MASK(0, sz); + uint64_t v; + + qemu_build_assert(HAVE_al8); + if (HOST_BIG_ENDIAN) { + v = bswap64(val_le) >> sh; + m = bswap64(m) >> sh; + } else { + v = val_le << sh; + m <<= sh; + } + store_atom_insert_al8(pv - o, v, m); + return val_le >> sz; +} + +/** + * store_whole_le16 + * @pv: host address + * @size: number of bytes to store + * @val_le: data to store + * + * As store_bytes_leN, but atomically as a whole. + * 16 aligned bytes are guaranteed to cover the store. + */ +static uint64_t store_whole_le16(void *pv, int size, Int128 val_le) +{ + int sz = size * 8; + int o = (uintptr_t)pv & 15; + int sh = o * 8; + Int128 m, v; + + qemu_build_assert(HAVE_al16); + + /* Like MAKE_64BIT_MASK(0, sz), but larger. */ + if (sz <= 64) { + m = int128_make64(MAKE_64BIT_MASK(0, sz)); + } else { + m = int128_make128(-1, MAKE_64BIT_MASK(0, sz - 64)); + } + + if (HOST_BIG_ENDIAN) { + v = int128_urshift(bswap128(val_le), sh); + m = int128_urshift(bswap128(m), sh); + } else { + v = int128_lshift(val_le, sh); + m = int128_lshift(m, sh); + } + store_atom_insert_al16(pv - o, v, m); + + /* Unused if sz <= 64. */ + return int128_gethi(val_le) >> (sz - 64); +} + +/** + * store_atom_2: + * @p: host address + * @val: the value to store + * @memop: the full memory op + * + * Store 2 bytes to @p, honoring the atomicity of @memop. + */ +static void store_atom_2(CPUArchState *env, uintptr_t ra, + void *pv, MemOp memop, uint16_t val) +{ + uintptr_t pi = (uintptr_t)pv; + int atmax; + + if (likely((pi & 1) == 0)) { + store_atomic2(pv, val); + return; + } + + atmax = required_atomicity(env, pi, memop); + if (atmax == MO_8) { + stw_he_p(pv, val); + return; + } + + /* + * The only case remaining is MO_ATOM_WITHIN16. + * Big or little endian, we want the middle two bytes in each test. + */ + if ((pi & 3) == 1) { + store_atom_insert_al4(pv - 1, (uint32_t)val << 8, MAKE_64BIT_MASK(8, 16)); + return; + } else if ((pi & 7) == 3) { + if (HAVE_al8) { + store_atom_insert_al8(pv - 3, (uint64_t)val << 24, MAKE_64BIT_MASK(24, 16)); + return; + } + } else if ((pi & 15) == 7) { + if (HAVE_al16) { + Int128 v = int128_lshift(int128_make64(val), 56); + Int128 m = int128_lshift(int128_make64(0xffff), 56); + store_atom_insert_al16(pv - 7, v, m); + return; + } + } else { + g_assert_not_reached(); + } + + cpu_loop_exit_atomic(env_cpu(env), ra); +} + +/** + * store_atom_4: + * @p: host address + * @val: the value to store + * @memop: the full memory op + * + * Store 4 bytes to @p, honoring the atomicity of @memop. + */ +static void store_atom_4(CPUArchState *env, uintptr_t ra, + void *pv, MemOp memop, uint32_t val) +{ + uintptr_t pi = (uintptr_t)pv; + int atmax; + + if (likely((pi & 3) == 0)) { + store_atomic4(pv, val); + return; + } + + atmax = required_atomicity(env, pi, memop); + switch (atmax) { + case MO_8: + stl_he_p(pv, val); + return; + case MO_16: + store_atom_4_by_2(pv, val); + return; + case -MO_16: + { + uint32_t val_le = cpu_to_le32(val); + int s2 = pi & 3; + int s1 = 4 - s2; + + switch (s2) { + case 1: + val_le = store_whole_le4(pv, s1, val_le); + *(uint8_t *)(pv + 3) = val_le; + break; + case 3: + *(uint8_t *)pv = val_le; + store_whole_le4(pv + 1, s2, val_le >> 8); + break; + case 0: /* aligned */ + case 2: /* atmax MO_16 */ + default: + g_assert_not_reached(); + } + } + return; + case MO_32: + if ((pi & 7) < 4) { + if (HAVE_al8) { + store_whole_le8(pv, 4, cpu_to_le32(val)); + return; + } + } else { + if (HAVE_al16) { + store_whole_le16(pv, 4, int128_make64(cpu_to_le32(val))); + return; + } + } + cpu_loop_exit_atomic(env_cpu(env), ra); + default: + g_assert_not_reached(); + } +} + +/** + * store_atom_8: + * @p: host address + * @val: the value to store + * @memop: the full memory op + * + * Store 8 bytes to @p, honoring the atomicity of @memop. + */ +static void store_atom_8(CPUArchState *env, uintptr_t ra, + void *pv, MemOp memop, uint64_t val) +{ + uintptr_t pi = (uintptr_t)pv; + int atmax; + + if (HAVE_al8 && likely((pi & 7) == 0)) { + store_atomic8(pv, val); + return; + } + + atmax = required_atomicity(env, pi, memop); + switch (atmax) { + case MO_8: + stq_he_p(pv, val); + return; + case MO_16: + store_atom_8_by_2(pv, val); + return; + case MO_32: + store_atom_8_by_4(pv, val); + return; + case -MO_32: + if (HAVE_al8) { + uint64_t val_le = cpu_to_le64(val); + int s2 = pi & 7; + int s1 = 8 - s2; + + switch (s2) { + case 1 ... 3: + val_le = store_whole_le8(pv, s1, val_le); + store_bytes_leN(pv + s1, s2, val_le); + break; + case 5 ... 7: + val_le = store_bytes_leN(pv, s1, val_le); + store_whole_le8(pv + s1, s2, val_le); + break; + case 0: /* aligned */ + case 4: /* atmax MO_32 */ + default: + g_assert_not_reached(); + } + return; + } + break; + case MO_64: + if (HAVE_al16) { + store_whole_le16(pv, 8, int128_make64(cpu_to_le64(val))); + return; + } + break; + default: + g_assert_not_reached(); + } + cpu_loop_exit_atomic(env_cpu(env), ra); +} From patchwork Tue May 16 19:40:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682403 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp20242wrt; Tue, 16 May 2023 12:56:28 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7IbfnytHyOKYqzFh4AHBo4iYVen3ZzpQxP4KOaWzwybjl6sbuYSufYLz7jOc3ICWTBuJPk X-Received: by 2002:a05:6214:301c:b0:5ef:5481:595 with SMTP id ke28-20020a056214301c00b005ef54810595mr7949qvb.0.1684266988604; Tue, 16 May 2023 12:56:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266988; cv=none; d=google.com; s=arc-20160816; b=UJzbIZFZCDJtIlrvjgSwJ7/12TBRnlhc30LZWvC2PnE9pL8c6E2ZcdwNH1xhMOwLUn T3HPJIe7DCK592aj5Hvr9Z+QFNYcOIpJY/Y9LUILx17sbs7GsgiW9uvoZUowGiYTrD9c 2OwC5tlAOcEDII7TYUNZHnPLOujnw/ZfkPfhYuipoCDfUX7aqgEwWj6VDWHz+ZUT6FpY 00zR4kJ5ajv6beFRZnG1qCjbj2yTgm3NfaZOtuP/zCCHr4u1L8jMKJXmkowCyb6mJF+K 8XHOBcX8QfoYCvEx3d8xQ4RB2KGLxukJ4vcGsjzwXOR4ezM6JjOgnH3NfokznUb7MYcF Rszw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=1rT6mxnxXPA8+sX1SsMo/mLXdCzwZeK78b8sns2KLdw=; b=XNQYo4eEfzlmEpDfIo0nSgG2Mygfhf2zT6VI0ooYCII9uZVSEJ0UzaLqQQZ7H05qrX DGo7oQLZMf6MmSBX611+/a95bWQ1Hb1uiNc3QjvTsRgnFlSZLvlFAAHbxQQu36dEAdon PCrXsRvGXWMUlIU0dFY8i8TNZCeBkHTyYYE6XMmJa54V0TyqrFpa+UL4q/7mvvQ3mcYn vnCo4xAXoEAbsrUOHV65/Sa7vwpAh1h8K27EYmj+aoLCpuvL6ANWQtOFZGJo9jpiEngG 6sTQgGHZQfbLMSLP1g1fbliKoRgYMzlQW0PsxWpTrwM8Jc5PXsUiwjCVURSV5K76fhW8 VACw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="BATwgR/T"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a13-20020a05620a16cd00b0074acb879ac1si269487qkn.33.2023.05.16.12.56.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:56:28 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="BATwgR/T"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0YY-0006bx-G7; Tue, 16 May 2023 15:41:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0YX-0006a8-Aj for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:57 -0400 Received: from mail-pf1-x42b.google.com ([2607:f8b0:4864:20::42b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0YT-0002jw-Ap for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:57 -0400 Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-64ab2a37812so8108492b3a.1 for ; Tue, 16 May 2023 12:41:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266112; x=1686858112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1rT6mxnxXPA8+sX1SsMo/mLXdCzwZeK78b8sns2KLdw=; b=BATwgR/TjjaxksnHYBdfVoWfl8TMKsL0taDVlWRptXFscOtJrBvPuLPdyzOQJ4jOpQ FBKW3IKETARnZ0adaqtnOjRZQ4ALnAz2z5eHBROVtNmcBr/IGltFznginCOOaczSsHpp kImaGQVthw27Mz69Cv8JF8wPtd+4fGOrBzTywElUfvo9scZt1P9Xmt7+7TUdUi5S5zN4 +yhNLuXnxO6le686juc6BwPdnYi7tlhY9BOrd90Q1Ncy4a1SoBZr0+EhK+knQDvaLWr9 1hLXlt2aY4BVKAouAozenePEz0M2ySH8IPqQX/bi0oOSmMNvLPHs3zsFWdpRwoKx7imW VL+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266112; x=1686858112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1rT6mxnxXPA8+sX1SsMo/mLXdCzwZeK78b8sns2KLdw=; b=eE4Zhl3P/pUoGhVWbzTz5XTTLdQOK3uQBwsBPmDigPSR4y0aaYBxJJc6uSB29DCE4V jDLMa1xq+hEofhafuTly2uyoOSV+H636nX4n/OpxpJw+lkQvtzMzQ5UZAjIiJGA1FIi0 1Y8HPKCNnMe633I5Na6H03m3ntR2b40FL1d0iQ6jPIYg22thYFN2bx8lhRqERbYiik6S 4gvWHXD+XF0d92FtONa0bU3d02+j10iIep/h8h8JQY7u/U/HdXxZMOJ8YVvJ/1ZLR2of J4LtlQ3Rb4LZWkWpbPAo/7983xgMR6GVqPByvXKV5u2OneoJsnxmNH1oUL1ct6ibES19 vw6g== X-Gm-Message-State: AC+VfDwIN2iPB39SSESoyBWzumKjMjoONqVxYQIeykwblI9IhSZ+3w2T OoLBiNGV6kY+PnsRLIINp6yRSY4el488g97u4RM= X-Received: by 2002:a05:6a00:1894:b0:646:c56c:f0e0 with SMTP id x20-20020a056a00189400b00646c56cf0e0mr34252314pfh.15.1684266111795; Tue, 16 May 2023 12:41:51 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.41.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:41:51 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Peter Maydell Subject: [PULL 05/80] tcg: Unify helper_{be,le}_{ld,st}* Date: Tue, 16 May 2023 12:40:30 -0700 Message-Id: <20230516194145.1749305-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42b; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org With the current structure of cputlb.c, there is no difference between the little-endian and big-endian entry points, aside from the assert. Unify the pairs of functions. Hoist the qemu_{ld,st}_helpers arrays to tcg.c. Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- docs/devel/loads-stores.rst | 36 ++---- include/tcg/tcg-ldst.h | 60 ++++------ accel/tcg/cputlb.c | 190 ++++++++++--------------------- tcg/tcg.c | 21 ++++ tcg/tci.c | 61 ++++------ tcg/aarch64/tcg-target.c.inc | 33 ------ tcg/arm/tcg-target.c.inc | 37 ------ tcg/i386/tcg-target.c.inc | 30 +---- tcg/loongarch64/tcg-target.c.inc | 23 ---- tcg/mips/tcg-target.c.inc | 31 ----- tcg/ppc/tcg-target.c.inc | 30 +---- tcg/riscv/tcg-target.c.inc | 42 ------- tcg/s390x/tcg-target.c.inc | 31 +---- tcg/sparc64/tcg-target.c.inc | 32 +----- 14 files changed, 146 insertions(+), 511 deletions(-) diff --git a/docs/devel/loads-stores.rst b/docs/devel/loads-stores.rst index ad5dfe133e..d2cefc77a2 100644 --- a/docs/devel/loads-stores.rst +++ b/docs/devel/loads-stores.rst @@ -297,31 +297,20 @@ swap: ``translator_ld{sign}{size}_swap(env, ptr, swap)`` Regexes for git grep - ``\`` -``helper_*_{ld,st}*_mmu`` +``helper_{ld,st}*_mmu`` ~~~~~~~~~~~~~~~~~~~~~~~~~ These functions are intended primarily to be called by the code -generated by the TCG backend. They may also be called by target -CPU helper function code. Like the ``cpu_{ld,st}_mmuidx_ra`` functions -they perform accesses by guest virtual address, with a given ``mmuidx``. +generated by the TCG backend. Like the ``cpu_{ld,st}_mmu`` functions +they perform accesses by guest virtual address, with a given ``MemOpIdx``. -These functions specify an ``opindex`` parameter which encodes -(among other things) the mmu index to use for the access. This parameter -should be created by calling ``make_memop_idx()``. +They differ from ``cpu_{ld,st}_mmu`` in that they take the endianness +of the operation only from the MemOpIdx, and loads extend the return +value to the size of a host general register (``tcg_target_ulong``). -The ``retaddr`` parameter should be the result of GETPC() called directly -from the top level HELPER(foo) function (or 0 if no guest CPU state -unwinding is required). +load: ``helper_ld{sign}{size}_mmu(env, addr, opindex, retaddr)`` -**TODO** The names of these functions are a bit odd for historical -reasons because they were originally expected to be called only from -within generated code. We should rename them to bring them more in -line with the other memory access functions. The explicit endianness -is the only feature they have beyond ``*_mmuidx_ra``. - -load: ``helper_{endian}_ld{sign}{size}_mmu(env, addr, opindex, retaddr)`` - -store: ``helper_{endian}_st{size}_mmu(env, addr, val, opindex, retaddr)`` +store: ``helper_{size}_mmu(env, addr, val, opindex, retaddr)`` ``sign`` - (empty) : for 32 or 64 bit sizes @@ -334,14 +323,9 @@ store: ``helper_{endian}_st{size}_mmu(env, addr, val, opindex, retaddr)`` - ``l`` : 32 bits - ``q`` : 64 bits -``endian`` - - ``le`` : little endian - - ``be`` : big endian - - ``ret`` : target endianness - Regexes for git grep - - ``\`` - - ``\`` + - ``\`` + - ``\`` ``address_space_*`` ~~~~~~~~~~~~~~~~~~~ diff --git a/include/tcg/tcg-ldst.h b/include/tcg/tcg-ldst.h index 684e394b06..3d897ca942 100644 --- a/include/tcg/tcg-ldst.h +++ b/include/tcg/tcg-ldst.h @@ -28,51 +28,35 @@ #ifdef CONFIG_SOFTMMU /* Value zero-extended to tcg register size. */ -tcg_target_ulong helper_ret_ldub_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr); -tcg_target_ulong helper_le_lduw_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr); -tcg_target_ulong helper_le_ldul_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr); -uint64_t helper_le_ldq_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr); -tcg_target_ulong helper_be_lduw_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr); -tcg_target_ulong helper_be_ldul_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr); -uint64_t helper_be_ldq_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr); +tcg_target_ulong helper_ldub_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t retaddr); +tcg_target_ulong helper_lduw_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t retaddr); +tcg_target_ulong helper_ldul_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t retaddr); +uint64_t helper_ldq_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t retaddr); /* Value sign-extended to tcg register size. */ -tcg_target_ulong helper_ret_ldsb_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr); -tcg_target_ulong helper_le_ldsw_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr); -tcg_target_ulong helper_le_ldsl_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr); -tcg_target_ulong helper_be_ldsw_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr); -tcg_target_ulong helper_be_ldsl_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr); +tcg_target_ulong helper_ldsb_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t retaddr); +tcg_target_ulong helper_ldsw_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t retaddr); +tcg_target_ulong helper_ldsl_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t retaddr); /* * Value extended to at least uint32_t, so that some ABIs do not require * zero-extension from uint8_t or uint16_t. */ -void helper_ret_stb_mmu(CPUArchState *env, target_ulong addr, uint32_t val, - MemOpIdx oi, uintptr_t retaddr); -void helper_le_stw_mmu(CPUArchState *env, target_ulong addr, uint32_t val, - MemOpIdx oi, uintptr_t retaddr); -void helper_le_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, - MemOpIdx oi, uintptr_t retaddr); -void helper_le_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, - MemOpIdx oi, uintptr_t retaddr); -void helper_be_stw_mmu(CPUArchState *env, target_ulong addr, uint32_t val, - MemOpIdx oi, uintptr_t retaddr); -void helper_be_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, - MemOpIdx oi, uintptr_t retaddr); -void helper_be_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, - MemOpIdx oi, uintptr_t retaddr); +void helper_stb_mmu(CPUArchState *env, target_ulong addr, uint32_t val, + MemOpIdx oi, uintptr_t retaddr); +void helper_stw_mmu(CPUArchState *env, target_ulong addr, uint32_t val, + MemOpIdx oi, uintptr_t retaddr); +void helper_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, + MemOpIdx oi, uintptr_t retaddr); +void helper_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, + MemOpIdx oi, uintptr_t retaddr); #else diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index d910464c36..34796ef568 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -2012,25 +2012,6 @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, cpu_loop_exit_atomic(env_cpu(env), retaddr); } -/* - * Verify that we have passed the correct MemOp to the correct function. - * - * In the case of the helper_*_mmu functions, we will have done this by - * using the MemOp to look up the helper during code generation. - * - * In the case of the cpu_*_mmu functions, this is up to the caller. - * We could present one function to target code, and dispatch based on - * the MemOp, but so far we have worked hard to avoid an indirect function - * call along the memory path. - */ -static void validate_memop(MemOpIdx oi, MemOp expected) -{ -#ifdef CONFIG_DEBUG_TCG - MemOp have = get_memop(oi) & (MO_SIZE | MO_BSWAP); - assert(have == expected); -#endif -} - /* * Load Helpers * @@ -2303,10 +2284,10 @@ static uint8_t do_ld1_mmu(CPUArchState *env, target_ulong addr, MemOpIdx oi, return do_ld_1(env, &l.page[0], l.mmu_idx, access_type, ra); } -tcg_target_ulong helper_ret_ldub_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr) +tcg_target_ulong helper_ldub_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t retaddr) { - validate_memop(oi, MO_UB); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_8); return do_ld1_mmu(env, addr, oi, retaddr, MMU_DATA_LOAD); } @@ -2334,17 +2315,10 @@ static uint16_t do_ld2_mmu(CPUArchState *env, target_ulong addr, MemOpIdx oi, return ret; } -tcg_target_ulong helper_le_lduw_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr) +tcg_target_ulong helper_lduw_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t retaddr) { - validate_memop(oi, MO_LEUW); - return do_ld2_mmu(env, addr, oi, retaddr, MMU_DATA_LOAD); -} - -tcg_target_ulong helper_be_lduw_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr) -{ - validate_memop(oi, MO_BEUW); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_16); return do_ld2_mmu(env, addr, oi, retaddr, MMU_DATA_LOAD); } @@ -2368,17 +2342,10 @@ static uint32_t do_ld4_mmu(CPUArchState *env, target_ulong addr, MemOpIdx oi, return ret; } -tcg_target_ulong helper_le_ldul_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr) +tcg_target_ulong helper_ldul_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t retaddr) { - validate_memop(oi, MO_LEUL); - return do_ld4_mmu(env, addr, oi, retaddr, MMU_DATA_LOAD); -} - -tcg_target_ulong helper_be_ldul_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr) -{ - validate_memop(oi, MO_BEUL); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_32); return do_ld4_mmu(env, addr, oi, retaddr, MMU_DATA_LOAD); } @@ -2402,17 +2369,10 @@ static uint64_t do_ld8_mmu(CPUArchState *env, target_ulong addr, MemOpIdx oi, return ret; } -uint64_t helper_le_ldq_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr) +uint64_t helper_ldq_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t retaddr) { - validate_memop(oi, MO_LEUQ); - return do_ld8_mmu(env, addr, oi, retaddr, MMU_DATA_LOAD); -} - -uint64_t helper_be_ldq_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr) -{ - validate_memop(oi, MO_BEUQ); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_64); return do_ld8_mmu(env, addr, oi, retaddr, MMU_DATA_LOAD); } @@ -2421,35 +2381,22 @@ uint64_t helper_be_ldq_mmu(CPUArchState *env, target_ulong addr, * avoid this for 64-bit data, or for 32-bit data on 32-bit host. */ - -tcg_target_ulong helper_ret_ldsb_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr) +tcg_target_ulong helper_ldsb_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t retaddr) { - return (int8_t)helper_ret_ldub_mmu(env, addr, oi, retaddr); + return (int8_t)helper_ldub_mmu(env, addr, oi, retaddr); } -tcg_target_ulong helper_le_ldsw_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr) +tcg_target_ulong helper_ldsw_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t retaddr) { - return (int16_t)helper_le_lduw_mmu(env, addr, oi, retaddr); + return (int16_t)helper_lduw_mmu(env, addr, oi, retaddr); } -tcg_target_ulong helper_be_ldsw_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr) +tcg_target_ulong helper_ldsl_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t retaddr) { - return (int16_t)helper_be_lduw_mmu(env, addr, oi, retaddr); -} - -tcg_target_ulong helper_le_ldsl_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr) -{ - return (int32_t)helper_le_ldul_mmu(env, addr, oi, retaddr); -} - -tcg_target_ulong helper_be_ldsl_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr) -{ - return (int32_t)helper_be_ldul_mmu(env, addr, oi, retaddr); + return (int32_t)helper_ldul_mmu(env, addr, oi, retaddr); } /* @@ -2465,7 +2412,7 @@ uint8_t cpu_ldb_mmu(CPUArchState *env, abi_ptr addr, MemOpIdx oi, uintptr_t ra) { uint8_t ret; - validate_memop(oi, MO_UB); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_UB); ret = do_ld1_mmu(env, addr, oi, ra, MMU_DATA_LOAD); plugin_load_cb(env, addr, oi); return ret; @@ -2476,7 +2423,7 @@ uint16_t cpu_ldw_be_mmu(CPUArchState *env, abi_ptr addr, { uint16_t ret; - validate_memop(oi, MO_BEUW); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_BEUW); ret = do_ld2_mmu(env, addr, oi, ra, MMU_DATA_LOAD); plugin_load_cb(env, addr, oi); return ret; @@ -2487,7 +2434,7 @@ uint32_t cpu_ldl_be_mmu(CPUArchState *env, abi_ptr addr, { uint32_t ret; - validate_memop(oi, MO_BEUL); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_BEUL); ret = do_ld4_mmu(env, addr, oi, ra, MMU_DATA_LOAD); plugin_load_cb(env, addr, oi); return ret; @@ -2498,7 +2445,7 @@ uint64_t cpu_ldq_be_mmu(CPUArchState *env, abi_ptr addr, { uint64_t ret; - validate_memop(oi, MO_BEUQ); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_BEUQ); ret = do_ld8_mmu(env, addr, oi, ra, MMU_DATA_LOAD); plugin_load_cb(env, addr, oi); return ret; @@ -2509,7 +2456,7 @@ uint16_t cpu_ldw_le_mmu(CPUArchState *env, abi_ptr addr, { uint16_t ret; - validate_memop(oi, MO_LEUW); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_LEUW); ret = do_ld2_mmu(env, addr, oi, ra, MMU_DATA_LOAD); plugin_load_cb(env, addr, oi); return ret; @@ -2520,7 +2467,7 @@ uint32_t cpu_ldl_le_mmu(CPUArchState *env, abi_ptr addr, { uint32_t ret; - validate_memop(oi, MO_LEUL); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_LEUL); ret = do_ld4_mmu(env, addr, oi, ra, MMU_DATA_LOAD); plugin_load_cb(env, addr, oi); return ret; @@ -2531,7 +2478,7 @@ uint64_t cpu_ldq_le_mmu(CPUArchState *env, abi_ptr addr, { uint64_t ret; - validate_memop(oi, MO_LEUQ); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_LEUQ); ret = do_ld8_mmu(env, addr, oi, ra, MMU_DATA_LOAD); plugin_load_cb(env, addr, oi); return ret; @@ -2559,8 +2506,8 @@ Int128 cpu_ld16_be_mmu(CPUArchState *env, abi_ptr addr, mop = (mop & ~(MO_SIZE | MO_AMASK)) | MO_64 | MO_UNALN; new_oi = make_memop_idx(mop, mmu_idx); - h = helper_be_ldq_mmu(env, addr, new_oi, ra); - l = helper_be_ldq_mmu(env, addr + 8, new_oi, ra); + h = helper_ldq_mmu(env, addr, new_oi, ra); + l = helper_ldq_mmu(env, addr + 8, new_oi, ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); return int128_make128(l, h); @@ -2588,8 +2535,8 @@ Int128 cpu_ld16_le_mmu(CPUArchState *env, abi_ptr addr, mop = (mop & ~(MO_SIZE | MO_AMASK)) | MO_64 | MO_UNALN; new_oi = make_memop_idx(mop, mmu_idx); - l = helper_le_ldq_mmu(env, addr, new_oi, ra); - h = helper_le_ldq_mmu(env, addr + 8, new_oi, ra); + l = helper_ldq_mmu(env, addr, new_oi, ra); + h = helper_ldq_mmu(env, addr + 8, new_oi, ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); return int128_make128(l, h); @@ -2738,13 +2685,13 @@ static void do_st_8(CPUArchState *env, MMULookupPageData *p, uint64_t val, } } -void helper_ret_stb_mmu(CPUArchState *env, target_ulong addr, uint32_t val, - MemOpIdx oi, uintptr_t ra) +void helper_stb_mmu(CPUArchState *env, target_ulong addr, uint32_t val, + MemOpIdx oi, uintptr_t ra) { MMULookupLocals l; bool crosspage; - validate_memop(oi, MO_UB); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_8); crosspage = mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE, &l); tcg_debug_assert(!crosspage); @@ -2773,17 +2720,10 @@ static void do_st2_mmu(CPUArchState *env, target_ulong addr, uint16_t val, do_st_1(env, &l.page[1], b, l.mmu_idx, ra); } -void helper_le_stw_mmu(CPUArchState *env, target_ulong addr, uint32_t val, - MemOpIdx oi, uintptr_t retaddr) +void helper_stw_mmu(CPUArchState *env, target_ulong addr, uint32_t val, + MemOpIdx oi, uintptr_t retaddr) { - validate_memop(oi, MO_LEUW); - do_st2_mmu(env, addr, val, oi, retaddr); -} - -void helper_be_stw_mmu(CPUArchState *env, target_ulong addr, uint32_t val, - MemOpIdx oi, uintptr_t retaddr) -{ - validate_memop(oi, MO_BEUW); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_16); do_st2_mmu(env, addr, val, oi, retaddr); } @@ -2807,17 +2747,10 @@ static void do_st4_mmu(CPUArchState *env, target_ulong addr, uint32_t val, (void) do_st_leN(env, &l.page[1], val, l.mmu_idx, l.memop, ra); } -void helper_le_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, - MemOpIdx oi, uintptr_t retaddr) +void helper_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, + MemOpIdx oi, uintptr_t retaddr) { - validate_memop(oi, MO_LEUL); - do_st4_mmu(env, addr, val, oi, retaddr); -} - -void helper_be_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, - MemOpIdx oi, uintptr_t retaddr) -{ - validate_memop(oi, MO_BEUL); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_32); do_st4_mmu(env, addr, val, oi, retaddr); } @@ -2841,17 +2774,10 @@ static void do_st8_mmu(CPUArchState *env, target_ulong addr, uint64_t val, (void) do_st_leN(env, &l.page[1], val, l.mmu_idx, l.memop, ra); } -void helper_le_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, - MemOpIdx oi, uintptr_t retaddr) +void helper_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, + MemOpIdx oi, uintptr_t retaddr) { - validate_memop(oi, MO_LEUQ); - do_st8_mmu(env, addr, val, oi, retaddr); -} - -void helper_be_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, - MemOpIdx oi, uintptr_t retaddr) -{ - validate_memop(oi, MO_BEUQ); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_64); do_st8_mmu(env, addr, val, oi, retaddr); } @@ -2867,49 +2793,55 @@ static void plugin_store_cb(CPUArchState *env, abi_ptr addr, MemOpIdx oi) void cpu_stb_mmu(CPUArchState *env, target_ulong addr, uint8_t val, MemOpIdx oi, uintptr_t retaddr) { - helper_ret_stb_mmu(env, addr, val, oi, retaddr); + helper_stb_mmu(env, addr, val, oi, retaddr); plugin_store_cb(env, addr, oi); } void cpu_stw_be_mmu(CPUArchState *env, target_ulong addr, uint16_t val, MemOpIdx oi, uintptr_t retaddr) { - helper_be_stw_mmu(env, addr, val, oi, retaddr); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_BEUW); + do_st2_mmu(env, addr, val, oi, retaddr); plugin_store_cb(env, addr, oi); } void cpu_stl_be_mmu(CPUArchState *env, target_ulong addr, uint32_t val, MemOpIdx oi, uintptr_t retaddr) { - helper_be_stl_mmu(env, addr, val, oi, retaddr); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_BEUL); + do_st4_mmu(env, addr, val, oi, retaddr); plugin_store_cb(env, addr, oi); } void cpu_stq_be_mmu(CPUArchState *env, target_ulong addr, uint64_t val, MemOpIdx oi, uintptr_t retaddr) { - helper_be_stq_mmu(env, addr, val, oi, retaddr); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_BEUQ); + do_st8_mmu(env, addr, val, oi, retaddr); plugin_store_cb(env, addr, oi); } void cpu_stw_le_mmu(CPUArchState *env, target_ulong addr, uint16_t val, MemOpIdx oi, uintptr_t retaddr) { - helper_le_stw_mmu(env, addr, val, oi, retaddr); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_LEUW); + do_st2_mmu(env, addr, val, oi, retaddr); plugin_store_cb(env, addr, oi); } void cpu_stl_le_mmu(CPUArchState *env, target_ulong addr, uint32_t val, MemOpIdx oi, uintptr_t retaddr) { - helper_le_stl_mmu(env, addr, val, oi, retaddr); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_LEUL); + do_st4_mmu(env, addr, val, oi, retaddr); plugin_store_cb(env, addr, oi); } void cpu_stq_le_mmu(CPUArchState *env, target_ulong addr, uint64_t val, MemOpIdx oi, uintptr_t retaddr) { - helper_le_stq_mmu(env, addr, val, oi, retaddr); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_LEUQ); + do_st8_mmu(env, addr, val, oi, retaddr); plugin_store_cb(env, addr, oi); } @@ -2934,8 +2866,8 @@ void cpu_st16_be_mmu(CPUArchState *env, abi_ptr addr, Int128 val, mop = (mop & ~(MO_SIZE | MO_AMASK)) | MO_64 | MO_UNALN; new_oi = make_memop_idx(mop, mmu_idx); - helper_be_stq_mmu(env, addr, int128_gethi(val), new_oi, ra); - helper_be_stq_mmu(env, addr + 8, int128_getlo(val), new_oi, ra); + helper_stq_mmu(env, addr, int128_gethi(val), new_oi, ra); + helper_stq_mmu(env, addr + 8, int128_getlo(val), new_oi, ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } @@ -2961,8 +2893,8 @@ void cpu_st16_le_mmu(CPUArchState *env, abi_ptr addr, Int128 val, mop = (mop & ~(MO_SIZE | MO_AMASK)) | MO_64 | MO_UNALN; new_oi = make_memop_idx(mop, mmu_idx); - helper_le_stq_mmu(env, addr, int128_getlo(val), new_oi, ra); - helper_le_stq_mmu(env, addr + 8, int128_gethi(val), new_oi, ra); + helper_stq_mmu(env, addr, int128_getlo(val), new_oi, ra); + helper_stq_mmu(env, addr + 8, int128_gethi(val), new_oi, ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } diff --git a/tcg/tcg.c b/tcg/tcg.c index f156ca65f5..f5f9d8f7e8 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -197,6 +197,27 @@ static void tcg_out_st_helper_args(TCGContext *s, const TCGLabelQemuLdst *l, const TCGLdstHelperParam *p) __attribute__((unused)); +#ifdef CONFIG_SOFTMMU +static void * const qemu_ld_helpers[MO_SSIZE + 1] = { + [MO_UB] = helper_ldub_mmu, + [MO_SB] = helper_ldsb_mmu, + [MO_UW] = helper_lduw_mmu, + [MO_SW] = helper_ldsw_mmu, + [MO_UL] = helper_ldul_mmu, + [MO_UQ] = helper_ldq_mmu, +#if TCG_TARGET_REG_BITS == 64 + [MO_SL] = helper_ldsl_mmu, +#endif +}; + +static void * const qemu_st_helpers[MO_SIZE + 1] = { + [MO_8] = helper_stb_mmu, + [MO_16] = helper_stw_mmu, + [MO_32] = helper_stl_mmu, + [MO_64] = helper_stq_mmu, +}; +#endif + TCGContext tcg_init_ctx; __thread TCGContext *tcg_ctx; diff --git a/tcg/tci.c b/tcg/tci.c index fc67e7e767..5bde2e1f2e 100644 --- a/tcg/tci.c +++ b/tcg/tci.c @@ -293,31 +293,21 @@ static uint64_t tci_qemu_ld(CPUArchState *env, target_ulong taddr, uintptr_t ra = (uintptr_t)tb_ptr; #ifdef CONFIG_SOFTMMU - switch (mop & (MO_BSWAP | MO_SSIZE)) { + switch (mop & MO_SSIZE) { case MO_UB: - return helper_ret_ldub_mmu(env, taddr, oi, ra); + return helper_ldub_mmu(env, taddr, oi, ra); case MO_SB: - return helper_ret_ldsb_mmu(env, taddr, oi, ra); - case MO_LEUW: - return helper_le_lduw_mmu(env, taddr, oi, ra); - case MO_LESW: - return helper_le_ldsw_mmu(env, taddr, oi, ra); - case MO_LEUL: - return helper_le_ldul_mmu(env, taddr, oi, ra); - case MO_LESL: - return helper_le_ldsl_mmu(env, taddr, oi, ra); - case MO_LEUQ: - return helper_le_ldq_mmu(env, taddr, oi, ra); - case MO_BEUW: - return helper_be_lduw_mmu(env, taddr, oi, ra); - case MO_BESW: - return helper_be_ldsw_mmu(env, taddr, oi, ra); - case MO_BEUL: - return helper_be_ldul_mmu(env, taddr, oi, ra); - case MO_BESL: - return helper_be_ldsl_mmu(env, taddr, oi, ra); - case MO_BEUQ: - return helper_be_ldq_mmu(env, taddr, oi, ra); + return helper_ldsb_mmu(env, taddr, oi, ra); + case MO_UW: + return helper_lduw_mmu(env, taddr, oi, ra); + case MO_SW: + return helper_ldsw_mmu(env, taddr, oi, ra); + case MO_UL: + return helper_ldul_mmu(env, taddr, oi, ra); + case MO_SL: + return helper_ldsl_mmu(env, taddr, oi, ra); + case MO_UQ: + return helper_ldq_mmu(env, taddr, oi, ra); default: g_assert_not_reached(); } @@ -382,27 +372,18 @@ static void tci_qemu_st(CPUArchState *env, target_ulong taddr, uint64_t val, uintptr_t ra = (uintptr_t)tb_ptr; #ifdef CONFIG_SOFTMMU - switch (mop & (MO_BSWAP | MO_SIZE)) { + switch (mop & MO_SIZE) { case MO_UB: - helper_ret_stb_mmu(env, taddr, val, oi, ra); + helper_stb_mmu(env, taddr, val, oi, ra); break; - case MO_LEUW: - helper_le_stw_mmu(env, taddr, val, oi, ra); + case MO_UW: + helper_stw_mmu(env, taddr, val, oi, ra); break; - case MO_LEUL: - helper_le_stl_mmu(env, taddr, val, oi, ra); + case MO_UL: + helper_stl_mmu(env, taddr, val, oi, ra); break; - case MO_LEUQ: - helper_le_stq_mmu(env, taddr, val, oi, ra); - break; - case MO_BEUW: - helper_be_stw_mmu(env, taddr, val, oi, ra); - break; - case MO_BEUL: - helper_be_stl_mmu(env, taddr, val, oi, ra); - break; - case MO_BEUQ: - helper_be_stq_mmu(env, taddr, val, oi, ra); + case MO_UQ: + helper_stq_mmu(env, taddr, val, oi, ra); break; default: g_assert_not_reached(); diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index 62dd22d73c..e6636c1f8b 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -1587,39 +1587,6 @@ typedef struct { } HostAddress; #ifdef CONFIG_SOFTMMU -/* helper signature: helper_ret_ld_mmu(CPUState *env, target_ulong addr, - * MemOpIdx oi, uintptr_t ra) - */ -static void * const qemu_ld_helpers[MO_SIZE + 1] = { - [MO_8] = helper_ret_ldub_mmu, -#if HOST_BIG_ENDIAN - [MO_16] = helper_be_lduw_mmu, - [MO_32] = helper_be_ldul_mmu, - [MO_64] = helper_be_ldq_mmu, -#else - [MO_16] = helper_le_lduw_mmu, - [MO_32] = helper_le_ldul_mmu, - [MO_64] = helper_le_ldq_mmu, -#endif -}; - -/* helper signature: helper_ret_st_mmu(CPUState *env, target_ulong addr, - * uintxx_t val, MemOpIdx oi, - * uintptr_t ra) - */ -static void * const qemu_st_helpers[MO_SIZE + 1] = { - [MO_8] = helper_ret_stb_mmu, -#if HOST_BIG_ENDIAN - [MO_16] = helper_be_stw_mmu, - [MO_32] = helper_be_stl_mmu, - [MO_64] = helper_be_stq_mmu, -#else - [MO_16] = helper_le_stw_mmu, - [MO_32] = helper_le_stl_mmu, - [MO_64] = helper_le_stq_mmu, -#endif -}; - static const TCGLdstHelperParam ldst_helper_param = { .ntmp = 1, .tmp = { TCG_REG_TMP } }; diff --git a/tcg/arm/tcg-target.c.inc b/tcg/arm/tcg-target.c.inc index df514e56fc..8b0d526659 100644 --- a/tcg/arm/tcg-target.c.inc +++ b/tcg/arm/tcg-target.c.inc @@ -1333,43 +1333,6 @@ typedef struct { } HostAddress; #ifdef CONFIG_SOFTMMU -/* helper signature: helper_ret_ld_mmu(CPUState *env, target_ulong addr, - * int mmu_idx, uintptr_t ra) - */ -static void * const qemu_ld_helpers[MO_SSIZE + 1] = { - [MO_UB] = helper_ret_ldub_mmu, - [MO_SB] = helper_ret_ldsb_mmu, -#if HOST_BIG_ENDIAN - [MO_UW] = helper_be_lduw_mmu, - [MO_UL] = helper_be_ldul_mmu, - [MO_UQ] = helper_be_ldq_mmu, - [MO_SW] = helper_be_ldsw_mmu, - [MO_SL] = helper_be_ldul_mmu, -#else - [MO_UW] = helper_le_lduw_mmu, - [MO_UL] = helper_le_ldul_mmu, - [MO_UQ] = helper_le_ldq_mmu, - [MO_SW] = helper_le_ldsw_mmu, - [MO_SL] = helper_le_ldul_mmu, -#endif -}; - -/* helper signature: helper_ret_st_mmu(CPUState *env, target_ulong addr, - * uintxx_t val, int mmu_idx, uintptr_t ra) - */ -static void * const qemu_st_helpers[MO_SIZE + 1] = { - [MO_8] = helper_ret_stb_mmu, -#if HOST_BIG_ENDIAN - [MO_16] = helper_be_stw_mmu, - [MO_32] = helper_be_stl_mmu, - [MO_64] = helper_be_stq_mmu, -#else - [MO_16] = helper_le_stw_mmu, - [MO_32] = helper_le_stl_mmu, - [MO_64] = helper_le_stq_mmu, -#endif -}; - static TCGReg ldst_ra_gen(TCGContext *s, const TCGLabelQemuLdst *l, int arg) { /* We arrive at the slow path via "BLNE", so R14 contains l->raddr. */ diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index 9fc5592f5d..826f7764c9 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -1776,32 +1776,6 @@ typedef struct { } HostAddress; #if defined(CONFIG_SOFTMMU) -/* helper signature: helper_ret_ld_mmu(CPUState *env, target_ulong addr, - * int mmu_idx, uintptr_t ra) - */ -static void * const qemu_ld_helpers[(MO_SIZE | MO_BSWAP) + 1] = { - [MO_UB] = helper_ret_ldub_mmu, - [MO_LEUW] = helper_le_lduw_mmu, - [MO_LEUL] = helper_le_ldul_mmu, - [MO_LEUQ] = helper_le_ldq_mmu, - [MO_BEUW] = helper_be_lduw_mmu, - [MO_BEUL] = helper_be_ldul_mmu, - [MO_BEUQ] = helper_be_ldq_mmu, -}; - -/* helper signature: helper_ret_st_mmu(CPUState *env, target_ulong addr, - * uintxx_t val, int mmu_idx, uintptr_t ra) - */ -static void * const qemu_st_helpers[(MO_SIZE | MO_BSWAP) + 1] = { - [MO_UB] = helper_ret_stb_mmu, - [MO_LEUW] = helper_le_stw_mmu, - [MO_LEUL] = helper_le_stl_mmu, - [MO_LEUQ] = helper_le_stq_mmu, - [MO_BEUW] = helper_be_stw_mmu, - [MO_BEUL] = helper_be_stl_mmu, - [MO_BEUQ] = helper_be_stq_mmu, -}; - /* * Because i686 has no register parameters and because x86_64 has xchg * to handle addr/data register overlap, we have placed all input arguments @@ -1842,7 +1816,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) } tcg_out_ld_helper_args(s, l, &ldst_helper_param); - tcg_out_branch(s, 1, qemu_ld_helpers[opc & (MO_BSWAP | MO_SIZE)]); + tcg_out_branch(s, 1, qemu_ld_helpers[opc & MO_SIZE]); tcg_out_ld_helper_ret(s, l, false, &ldst_helper_param); tcg_out_jmp(s, l->raddr); @@ -1864,7 +1838,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) } tcg_out_st_helper_args(s, l, &ldst_helper_param); - tcg_out_branch(s, 1, qemu_st_helpers[opc & (MO_BSWAP | MO_SIZE)]); + tcg_out_branch(s, 1, qemu_st_helpers[opc & MO_SIZE]); tcg_out_jmp(s, l->raddr); return true; diff --git a/tcg/loongarch64/tcg-target.c.inc b/tcg/loongarch64/tcg-target.c.inc index 83fa45c802..d1bc29826f 100644 --- a/tcg/loongarch64/tcg-target.c.inc +++ b/tcg/loongarch64/tcg-target.c.inc @@ -784,29 +784,6 @@ static bool tcg_out_sti(TCGContext *s, TCGType type, TCGArg val, */ #if defined(CONFIG_SOFTMMU) -/* - * helper signature: helper_ret_ld_mmu(CPUState *env, target_ulong addr, - * MemOpIdx oi, uintptr_t ra) - */ -static void * const qemu_ld_helpers[4] = { - [MO_8] = helper_ret_ldub_mmu, - [MO_16] = helper_le_lduw_mmu, - [MO_32] = helper_le_ldul_mmu, - [MO_64] = helper_le_ldq_mmu, -}; - -/* - * helper signature: helper_ret_st_mmu(CPUState *env, target_ulong addr, - * uintxx_t val, MemOpIdx oi, - * uintptr_t ra) - */ -static void * const qemu_st_helpers[4] = { - [MO_8] = helper_ret_stb_mmu, - [MO_16] = helper_le_stw_mmu, - [MO_32] = helper_le_stl_mmu, - [MO_64] = helper_le_stq_mmu, -}; - static bool tcg_out_goto(TCGContext *s, const tcg_insn_unit *target) { tcg_out_opc_b(s, 0); diff --git a/tcg/mips/tcg-target.c.inc b/tcg/mips/tcg-target.c.inc index 5ad9867882..7770ef46bd 100644 --- a/tcg/mips/tcg-target.c.inc +++ b/tcg/mips/tcg-target.c.inc @@ -1076,37 +1076,6 @@ static void tcg_out_call(TCGContext *s, const tcg_insn_unit *arg, } #if defined(CONFIG_SOFTMMU) -static void * const qemu_ld_helpers[MO_SSIZE + 1] = { - [MO_UB] = helper_ret_ldub_mmu, - [MO_SB] = helper_ret_ldsb_mmu, -#if HOST_BIG_ENDIAN - [MO_UW] = helper_be_lduw_mmu, - [MO_SW] = helper_be_ldsw_mmu, - [MO_UL] = helper_be_ldul_mmu, - [MO_SL] = helper_be_ldsl_mmu, - [MO_UQ] = helper_be_ldq_mmu, -#else - [MO_UW] = helper_le_lduw_mmu, - [MO_SW] = helper_le_ldsw_mmu, - [MO_UL] = helper_le_ldul_mmu, - [MO_UQ] = helper_le_ldq_mmu, - [MO_SL] = helper_le_ldsl_mmu, -#endif -}; - -static void * const qemu_st_helpers[MO_SIZE + 1] = { - [MO_UB] = helper_ret_stb_mmu, -#if HOST_BIG_ENDIAN - [MO_UW] = helper_be_stw_mmu, - [MO_UL] = helper_be_stl_mmu, - [MO_UQ] = helper_be_stq_mmu, -#else - [MO_UW] = helper_le_stw_mmu, - [MO_UL] = helper_le_stl_mmu, - [MO_UQ] = helper_le_stq_mmu, -#endif -}; - /* We have four temps, we might as well expose three of them. */ static const TCGLdstHelperParam ldst_helper_param = { .ntmp = 3, .tmp = { TCG_TMP0, TCG_TMP1, TCG_TMP2 } diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc index 29bfbfcc61..6a81916e64 100644 --- a/tcg/ppc/tcg-target.c.inc +++ b/tcg/ppc/tcg-target.c.inc @@ -1963,32 +1963,6 @@ static const uint32_t qemu_stx_opc[(MO_SIZE + MO_BSWAP) + 1] = { }; #if defined (CONFIG_SOFTMMU) -/* helper signature: helper_ld_mmu(CPUState *env, target_ulong addr, - * int mmu_idx, uintptr_t ra) - */ -static void * const qemu_ld_helpers[(MO_SIZE | MO_BSWAP) + 1] = { - [MO_UB] = helper_ret_ldub_mmu, - [MO_LEUW] = helper_le_lduw_mmu, - [MO_LEUL] = helper_le_ldul_mmu, - [MO_LEUQ] = helper_le_ldq_mmu, - [MO_BEUW] = helper_be_lduw_mmu, - [MO_BEUL] = helper_be_ldul_mmu, - [MO_BEUQ] = helper_be_ldq_mmu, -}; - -/* helper signature: helper_st_mmu(CPUState *env, target_ulong addr, - * uintxx_t val, int mmu_idx, uintptr_t ra) - */ -static void * const qemu_st_helpers[(MO_SIZE | MO_BSWAP) + 1] = { - [MO_UB] = helper_ret_stb_mmu, - [MO_LEUW] = helper_le_stw_mmu, - [MO_LEUL] = helper_le_stl_mmu, - [MO_LEUQ] = helper_le_stq_mmu, - [MO_BEUW] = helper_be_stw_mmu, - [MO_BEUL] = helper_be_stl_mmu, - [MO_BEUQ] = helper_be_stq_mmu, -}; - static TCGReg ldst_ra_gen(TCGContext *s, const TCGLabelQemuLdst *l, int arg) { if (arg < 0) { @@ -2017,7 +1991,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) } tcg_out_ld_helper_args(s, lb, &ldst_helper_param); - tcg_out_call_int(s, LK, qemu_ld_helpers[opc & (MO_BSWAP | MO_SIZE)]); + tcg_out_call_int(s, LK, qemu_ld_helpers[opc & MO_SIZE]); tcg_out_ld_helper_ret(s, lb, false, &ldst_helper_param); tcg_out_b(s, 0, lb->raddr); @@ -2033,7 +2007,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) } tcg_out_st_helper_args(s, lb, &ldst_helper_param); - tcg_out_call_int(s, LK, qemu_st_helpers[opc & (MO_BSWAP | MO_SIZE)]); + tcg_out_call_int(s, LK, qemu_st_helpers[opc & MO_SIZE]); tcg_out_b(s, 0, lb->raddr); return true; diff --git a/tcg/riscv/tcg-target.c.inc b/tcg/riscv/tcg-target.c.inc index d12b824d8c..8ed0e2f210 100644 --- a/tcg/riscv/tcg-target.c.inc +++ b/tcg/riscv/tcg-target.c.inc @@ -847,48 +847,6 @@ static void tcg_out_mb(TCGContext *s, TCGArg a0) */ #if defined(CONFIG_SOFTMMU) -/* helper signature: helper_ret_ld_mmu(CPUState *env, target_ulong addr, - * MemOpIdx oi, uintptr_t ra) - */ -static void * const qemu_ld_helpers[MO_SSIZE + 1] = { - [MO_UB] = helper_ret_ldub_mmu, - [MO_SB] = helper_ret_ldsb_mmu, -#if HOST_BIG_ENDIAN - [MO_UW] = helper_be_lduw_mmu, - [MO_SW] = helper_be_ldsw_mmu, - [MO_UL] = helper_be_ldul_mmu, -#if TCG_TARGET_REG_BITS == 64 - [MO_SL] = helper_be_ldsl_mmu, -#endif - [MO_UQ] = helper_be_ldq_mmu, -#else - [MO_UW] = helper_le_lduw_mmu, - [MO_SW] = helper_le_ldsw_mmu, - [MO_UL] = helper_le_ldul_mmu, -#if TCG_TARGET_REG_BITS == 64 - [MO_SL] = helper_le_ldsl_mmu, -#endif - [MO_UQ] = helper_le_ldq_mmu, -#endif -}; - -/* helper signature: helper_ret_st_mmu(CPUState *env, target_ulong addr, - * uintxx_t val, MemOpIdx oi, - * uintptr_t ra) - */ -static void * const qemu_st_helpers[MO_SIZE + 1] = { - [MO_8] = helper_ret_stb_mmu, -#if HOST_BIG_ENDIAN - [MO_16] = helper_be_stw_mmu, - [MO_32] = helper_be_stl_mmu, - [MO_64] = helper_be_stq_mmu, -#else - [MO_16] = helper_le_stw_mmu, - [MO_32] = helper_le_stl_mmu, - [MO_64] = helper_le_stq_mmu, -#endif -}; - static void tcg_out_goto(TCGContext *s, const tcg_insn_unit *target) { tcg_out_opc_jump(s, OPC_JAL, TCG_REG_ZERO, 0); diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc index aacbaf21d5..968977be98 100644 --- a/tcg/s390x/tcg-target.c.inc +++ b/tcg/s390x/tcg-target.c.inc @@ -438,33 +438,6 @@ static const uint8_t tcg_cond_to_ltr_cond[] = { [TCG_COND_GEU] = S390_CC_ALWAYS, }; -#ifdef CONFIG_SOFTMMU -static void * const qemu_ld_helpers[(MO_SSIZE | MO_BSWAP) + 1] = { - [MO_UB] = helper_ret_ldub_mmu, - [MO_SB] = helper_ret_ldsb_mmu, - [MO_LEUW] = helper_le_lduw_mmu, - [MO_LESW] = helper_le_ldsw_mmu, - [MO_LEUL] = helper_le_ldul_mmu, - [MO_LESL] = helper_le_ldsl_mmu, - [MO_LEUQ] = helper_le_ldq_mmu, - [MO_BEUW] = helper_be_lduw_mmu, - [MO_BESW] = helper_be_ldsw_mmu, - [MO_BEUL] = helper_be_ldul_mmu, - [MO_BESL] = helper_be_ldsl_mmu, - [MO_BEUQ] = helper_be_ldq_mmu, -}; - -static void * const qemu_st_helpers[(MO_SIZE | MO_BSWAP) + 1] = { - [MO_UB] = helper_ret_stb_mmu, - [MO_LEUW] = helper_le_stw_mmu, - [MO_LEUL] = helper_le_stl_mmu, - [MO_LEUQ] = helper_le_stq_mmu, - [MO_BEUW] = helper_be_stw_mmu, - [MO_BEUL] = helper_be_stl_mmu, - [MO_BEUQ] = helper_be_stq_mmu, -}; -#endif - static const tcg_insn_unit *tb_ret_addr; uint64_t s390_facilities[3]; @@ -1721,7 +1694,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) } tcg_out_ld_helper_args(s, lb, &ldst_helper_param); - tcg_out_call_int(s, qemu_ld_helpers[opc & (MO_BSWAP | MO_SIZE)]); + tcg_out_call_int(s, qemu_ld_helpers[opc & MO_SIZE]); tcg_out_ld_helper_ret(s, lb, false, &ldst_helper_param); tgen_gotoi(s, S390_CC_ALWAYS, lb->raddr); @@ -1738,7 +1711,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) } tcg_out_st_helper_args(s, lb, &ldst_helper_param); - tcg_out_call_int(s, qemu_st_helpers[opc & (MO_BSWAP | MO_SIZE)]); + tcg_out_call_int(s, qemu_st_helpers[opc & MO_SIZE]); tgen_gotoi(s, S390_CC_ALWAYS, lb->raddr); return true; diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index 7e6466d3b6..e997db2645 100644 --- a/tcg/sparc64/tcg-target.c.inc +++ b/tcg/sparc64/tcg-target.c.inc @@ -919,33 +919,11 @@ static void tcg_out_mb(TCGContext *s, TCGArg a0) } #ifdef CONFIG_SOFTMMU -static const tcg_insn_unit *qemu_ld_trampoline[(MO_SSIZE | MO_BSWAP) + 1]; -static const tcg_insn_unit *qemu_st_trampoline[(MO_SIZE | MO_BSWAP) + 1]; +static const tcg_insn_unit *qemu_ld_trampoline[MO_SSIZE + 1]; +static const tcg_insn_unit *qemu_st_trampoline[MO_SIZE + 1]; static void build_trampolines(TCGContext *s) { - static void * const qemu_ld_helpers[] = { - [MO_UB] = helper_ret_ldub_mmu, - [MO_SB] = helper_ret_ldsb_mmu, - [MO_LEUW] = helper_le_lduw_mmu, - [MO_LESW] = helper_le_ldsw_mmu, - [MO_LEUL] = helper_le_ldul_mmu, - [MO_LEUQ] = helper_le_ldq_mmu, - [MO_BEUW] = helper_be_lduw_mmu, - [MO_BESW] = helper_be_ldsw_mmu, - [MO_BEUL] = helper_be_ldul_mmu, - [MO_BEUQ] = helper_be_ldq_mmu, - }; - static void * const qemu_st_helpers[] = { - [MO_UB] = helper_ret_stb_mmu, - [MO_LEUW] = helper_le_stw_mmu, - [MO_LEUL] = helper_le_stl_mmu, - [MO_LEUQ] = helper_le_stq_mmu, - [MO_BEUW] = helper_be_stw_mmu, - [MO_BEUL] = helper_be_stl_mmu, - [MO_BEUQ] = helper_be_stq_mmu, - }; - int i; for (i = 0; i < ARRAY_SIZE(qemu_ld_helpers); ++i) { @@ -1210,9 +1188,9 @@ static void tcg_out_qemu_ld(TCGContext *s, TCGReg data, TCGReg addr, /* We use the helpers to extend SB and SW data, leaving the case of SL needing explicit extending below. */ if ((memop & MO_SSIZE) == MO_SL) { - func = qemu_ld_trampoline[memop & (MO_BSWAP | MO_SIZE)]; + func = qemu_ld_trampoline[MO_UL]; } else { - func = qemu_ld_trampoline[memop & (MO_BSWAP | MO_SSIZE)]; + func = qemu_ld_trampoline[memop & MO_SSIZE]; } tcg_debug_assert(func != NULL); tcg_out_call_nodelay(s, func, false); @@ -1353,7 +1331,7 @@ static void tcg_out_qemu_st(TCGContext *s, TCGReg data, TCGReg addr, tcg_out_movext(s, (memop & MO_SIZE) == MO_64 ? TCG_TYPE_I64 : TCG_TYPE_I32, TCG_REG_O2, data_type, memop & MO_SIZE, data); - func = qemu_st_trampoline[memop & (MO_BSWAP | MO_SIZE)]; + func = qemu_st_trampoline[memop & MO_SIZE]; tcg_debug_assert(func != NULL); tcg_out_call_nodelay(s, func, false); /* delay slot */ From patchwork Tue May 16 19:40:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682415 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp20840wrt; Tue, 16 May 2023 12:58:40 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5VjYCrmdni8rRU34nL8mZMrA6s8Z8Odj0FIf2VQG2VlkR7IH3bbsZVRtNb99LvCWfzOQvz X-Received: by 2002:a5d:6342:0:b0:309:3a5a:86a2 with SMTP id b2-20020a5d6342000000b003093a5a86a2mr1309942wrw.38.1684267120001; Tue, 16 May 2023 12:58:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267119; cv=none; d=google.com; s=arc-20160816; b=yaGCcEUOnvm69nGBOPKcWb9eok+9kf8SpSkNqQyMG/6iGUOd9DErer7m62BJTBOtrM kbtRQpjnC5QUP2aI1f+xXsG7kGxPd10BdfjAWlKOP8V9Ao4Ult+I2SSnD/q66puc/Z5/ p8nCcw/fmqmCtEixz7VpAlCnIkfcdXM0sWxSwh5eQQRs+a1TymlVN4V5gprtiRIrfDLQ RHJx8XLuIny6Quex5nJbdg/uFsQ/eygwxDURp0uYUwJ05Uy02k1o35neBrkKHiYnhiyF cBzyr3J9eSau77kIBaIWW4sITKStjj5U0ia+R2KhmL02BZBUHK4abH86rrreSRQc35Vt 1jZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=zr1FUkJkAhrNHuYcLCFTMcK4YsDCqMv67DuUT8Pk2Gs=; b=WBM5AzZTfbe5v9aXk2nayXvkMqGZSIO7K8MtvieA64LDcuS1dgdKSNeEMbGYxcCSRl ghjMVd8lbqfsiofW8YX4eI1mhV2pQliQ8WugNxv5pXIAn4Gv3uqegcH9Wnsfa0vB3ZSD FYEwn2DtpIuuyLcP9Nu13mzp6YLY9Tk40uyI1t/TCBFTQOTV3XqfpVFmwe724lwSUoYW VcN37gELp961TmhqAJGnzMnKHS8qSzRzZ7fv9PdkD7r7QGH8ZnN4kv0aWlfE91c//2FV mfl7dFbnYvKx3K6OYWhopoIHrwnZE5aOFelNnI+DFCAka96n4OT58cL11c3rJv7XbEau BzTA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hj7Tj4Vh; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a16-20020adfeed0000000b003064546d069si175295wrp.721.2023.05.16.12.58.39 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:58:39 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hj7Tj4Vh; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0YZ-0006c8-95; Tue, 16 May 2023 15:41:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0YW-0006Zf-RY for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:56 -0400 Received: from mail-pg1-x52a.google.com ([2607:f8b0:4864:20::52a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0YU-0002kC-54 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:56 -0400 Received: by mail-pg1-x52a.google.com with SMTP id 41be03b00d2f7-52cb78647ecso9266100a12.1 for ; Tue, 16 May 2023 12:41:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266113; x=1686858113; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zr1FUkJkAhrNHuYcLCFTMcK4YsDCqMv67DuUT8Pk2Gs=; b=hj7Tj4Vhwsd2ITXiBEIdzH6ZI+GU0E6imhai4cQhBH/oHK2xbTXHXuJLQgMKMrmeUP 3dD/ulCTIpAxgqXdlSySGLEQhxyUUt0yyHW8gJg32vnZPDCfsfLyk1bO8xyLlcleODrI ulSgV2OsKrkmH1zaNFAaNvNDfMZF0jiCGQv1krRztyvzJzZE+f9ttz7Ka2ktIjlawg4/ 5vRupnScGJ2IfJIzLoSrcu5x+o4K5z2oMl6nKpvWFFcCH4qT4/szWjpoORk0aSS3weHT A3V5Lrz6/VP6CEVlcgJ9x4s6e1XXtsStrMv9YJslF/tyH8D0cYTKstnoZGvp4nXvvhLd wRCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266113; x=1686858113; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zr1FUkJkAhrNHuYcLCFTMcK4YsDCqMv67DuUT8Pk2Gs=; b=H7FfpCrzWPabLct6qnLaHItI2QbsoBdY6MVin6Nt/IOo/AKTJ0hgCpscA7rPfBrwXx 93TD1vLpDjWbbYXBj7oEZ9xtM5Ife+lT3pQ6g+O5pzfV26ObL4hfQdXg1jYvdwov2ig9 o/QRsXblT09GGr1NDARfg+F2pMAKZsJfLWIuc84sf3wXsav2FmC4pVE+aYZiQD8DV4On dimiF9U8qIJsZFK6q5x9C2O1fh0NI98PPZBkEYzrKOKHitt69JYUHyFR77TXa/swTnNP INZDb/9JYdQfiNptBpKXH0nANDT1LD/bYzFAxs4Q/ZYdOgkkgVXKKzEKScd9VpbSFsb3 BO7g== X-Gm-Message-State: AC+VfDzOyB5WcYQhspVpG2oUlxPRgf6FE64kBGopMnIhAFHQ7HL8gpLw zOClCNHNJUt+Seeqost3PH3tqOmYgAQjCDUhD+s= X-Received: by 2002:a05:6a00:22cd:b0:64a:f8c9:a421 with SMTP id f13-20020a056a0022cd00b0064af8c9a421mr20174754pfj.32.1684266112753; Tue, 16 May 2023 12:41:52 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.41.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:41:52 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 06/80] accel/tcg: Implement helper_{ld,st}*_mmu for user-only Date: Tue, 16 May 2023 12:40:31 -0700 Message-Id: <20230516194145.1749305-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52a; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org TCG backends may need to defer to a helper to implement the atomicity required by a given operation. Mirror the interface used in system mode. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- include/tcg/tcg-ldst.h | 6 +- accel/tcg/user-exec.c | 393 ++++++++++++++++++++++++++++------------- tcg/tcg.c | 6 +- 3 files changed, 278 insertions(+), 127 deletions(-) diff --git a/include/tcg/tcg-ldst.h b/include/tcg/tcg-ldst.h index 3d897ca942..57fafa14b1 100644 --- a/include/tcg/tcg-ldst.h +++ b/include/tcg/tcg-ldst.h @@ -25,8 +25,6 @@ #ifndef TCG_LDST_H #define TCG_LDST_H -#ifdef CONFIG_SOFTMMU - /* Value zero-extended to tcg register size. */ tcg_target_ulong helper_ldub_mmu(CPUArchState *env, target_ulong addr, MemOpIdx oi, uintptr_t retaddr); @@ -58,10 +56,10 @@ void helper_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, void helper_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, MemOpIdx oi, uintptr_t retaddr); -#else +#ifdef CONFIG_USER_ONLY G_NORETURN void helper_unaligned_ld(CPUArchState *env, target_ulong addr); G_NORETURN void helper_unaligned_st(CPUArchState *env, target_ulong addr); -#endif /* CONFIG_SOFTMMU */ +#endif /* CONFIG_USER_ONLY */ #endif /* TCG_LDST_H */ diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index b89fa35a83..d9f9766b7f 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -889,21 +889,6 @@ void page_reset_target_data(target_ulong start, target_ulong last) { } /* The softmmu versions of these helpers are in cputlb.c. */ -/* - * Verify that we have passed the correct MemOp to the correct function. - * - * We could present one function to target code, and dispatch based on - * the MemOp, but so far we have worked hard to avoid an indirect function - * call along the memory path. - */ -static void validate_memop(MemOpIdx oi, MemOp expected) -{ -#ifdef CONFIG_DEBUG_TCG - MemOp have = get_memop(oi) & (MO_SIZE | MO_BSWAP); - assert(have == expected); -#endif -} - void helper_unaligned_ld(CPUArchState *env, target_ulong addr) { cpu_loop_exit_sigbus(env_cpu(env), addr, MMU_DATA_LOAD, GETPC()); @@ -914,10 +899,9 @@ void helper_unaligned_st(CPUArchState *env, target_ulong addr) cpu_loop_exit_sigbus(env_cpu(env), addr, MMU_DATA_STORE, GETPC()); } -static void *cpu_mmu_lookup(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t ra, MMUAccessType type) +static void *cpu_mmu_lookup(CPUArchState *env, abi_ptr addr, + MemOp mop, uintptr_t ra, MMUAccessType type) { - MemOp mop = get_memop(oi); int a_bits = get_alignment_bits(mop); void *ret; @@ -933,100 +917,206 @@ static void *cpu_mmu_lookup(CPUArchState *env, target_ulong addr, #include "ldst_atomicity.c.inc" -uint8_t cpu_ldb_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) +static uint8_t do_ld1_mmu(CPUArchState *env, abi_ptr addr, + MemOp mop, uintptr_t ra) { void *haddr; uint8_t ret; - validate_memop(oi, MO_UB); - haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD); + tcg_debug_assert((mop & MO_SIZE) == MO_8); + haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_LOAD); ret = ldub_p(haddr); clear_helper_retaddr(); + return ret; +} + +tcg_target_ulong helper_ldub_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t ra) +{ + return do_ld1_mmu(env, addr, get_memop(oi), ra); +} + +tcg_target_ulong helper_ldsb_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t ra) +{ + return (int8_t)do_ld1_mmu(env, addr, get_memop(oi), ra); +} + +uint8_t cpu_ldb_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) +{ + uint8_t ret = do_ld1_mmu(env, addr, get_memop(oi), ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); return ret; } +static uint16_t do_ld2_he_mmu(CPUArchState *env, abi_ptr addr, + MemOp mop, uintptr_t ra) +{ + void *haddr; + uint16_t ret; + + tcg_debug_assert((mop & MO_SIZE) == MO_16); + haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_LOAD); + ret = load_atom_2(env, ra, haddr, mop); + clear_helper_retaddr(); + return ret; +} + +tcg_target_ulong helper_lduw_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t ra) +{ + MemOp mop = get_memop(oi); + uint16_t ret = do_ld2_he_mmu(env, addr, mop, ra); + + if (mop & MO_BSWAP) { + ret = bswap16(ret); + } + return ret; +} + +tcg_target_ulong helper_ldsw_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t ra) +{ + MemOp mop = get_memop(oi); + int16_t ret = do_ld2_he_mmu(env, addr, mop, ra); + + if (mop & MO_BSWAP) { + ret = bswap16(ret); + } + return ret; +} + uint16_t cpu_ldw_be_mmu(CPUArchState *env, abi_ptr addr, MemOpIdx oi, uintptr_t ra) { - void *haddr; + MemOp mop = get_memop(oi); uint16_t ret; - validate_memop(oi, MO_BEUW); - haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD); - ret = load_atom_2(env, ra, haddr, get_memop(oi)); - clear_helper_retaddr(); + tcg_debug_assert((mop & MO_BSWAP) == MO_BE); + ret = do_ld2_he_mmu(env, addr, mop, ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); return cpu_to_be16(ret); } -uint32_t cpu_ldl_be_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - void *haddr; - uint32_t ret; - - validate_memop(oi, MO_BEUL); - haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD); - ret = load_atom_4(env, ra, haddr, get_memop(oi)); - clear_helper_retaddr(); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return cpu_to_be32(ret); -} - -uint64_t cpu_ldq_be_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - void *haddr; - uint64_t ret; - - validate_memop(oi, MO_BEUQ); - haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD); - ret = load_atom_8(env, ra, haddr, get_memop(oi)); - clear_helper_retaddr(); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return cpu_to_be64(ret); -} - uint16_t cpu_ldw_le_mmu(CPUArchState *env, abi_ptr addr, MemOpIdx oi, uintptr_t ra) { - void *haddr; + MemOp mop = get_memop(oi); uint16_t ret; - validate_memop(oi, MO_LEUW); - haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD); - ret = load_atom_2(env, ra, haddr, get_memop(oi)); - clear_helper_retaddr(); + tcg_debug_assert((mop & MO_BSWAP) == MO_LE); + ret = do_ld2_he_mmu(env, addr, mop, ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); return cpu_to_le16(ret); } +static uint32_t do_ld4_he_mmu(CPUArchState *env, abi_ptr addr, + MemOp mop, uintptr_t ra) +{ + void *haddr; + uint32_t ret; + + tcg_debug_assert((mop & MO_SIZE) == MO_32); + haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_LOAD); + ret = load_atom_4(env, ra, haddr, mop); + clear_helper_retaddr(); + return ret; +} + +tcg_target_ulong helper_ldul_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t ra) +{ + MemOp mop = get_memop(oi); + uint32_t ret = do_ld4_he_mmu(env, addr, mop, ra); + + if (mop & MO_BSWAP) { + ret = bswap32(ret); + } + return ret; +} + +tcg_target_ulong helper_ldsl_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t ra) +{ + MemOp mop = get_memop(oi); + int32_t ret = do_ld4_he_mmu(env, addr, mop, ra); + + if (mop & MO_BSWAP) { + ret = bswap32(ret); + } + return ret; +} + +uint32_t cpu_ldl_be_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) +{ + MemOp mop = get_memop(oi); + uint32_t ret; + + tcg_debug_assert((mop & MO_BSWAP) == MO_BE); + ret = do_ld4_he_mmu(env, addr, mop, ra); + qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); + return cpu_to_be32(ret); +} + uint32_t cpu_ldl_le_mmu(CPUArchState *env, abi_ptr addr, MemOpIdx oi, uintptr_t ra) { - void *haddr; + MemOp mop = get_memop(oi); uint32_t ret; - validate_memop(oi, MO_LEUL); - haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD); - ret = load_atom_4(env, ra, haddr, get_memop(oi)); - clear_helper_retaddr(); + tcg_debug_assert((mop & MO_BSWAP) == MO_LE); + ret = do_ld4_he_mmu(env, addr, mop, ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); return cpu_to_le32(ret); } +static uint64_t do_ld8_he_mmu(CPUArchState *env, abi_ptr addr, + MemOp mop, uintptr_t ra) +{ + void *haddr; + uint64_t ret; + + tcg_debug_assert((mop & MO_SIZE) == MO_64); + haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_LOAD); + ret = load_atom_8(env, ra, haddr, mop); + clear_helper_retaddr(); + return ret; +} + +uint64_t helper_ldq_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t ra) +{ + MemOp mop = get_memop(oi); + uint64_t ret = do_ld8_he_mmu(env, addr, mop, ra); + + if (mop & MO_BSWAP) { + ret = bswap64(ret); + } + return ret; +} + +uint64_t cpu_ldq_be_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) +{ + MemOp mop = get_memop(oi); + uint64_t ret; + + tcg_debug_assert((mop & MO_BSWAP) == MO_BE); + ret = do_ld8_he_mmu(env, addr, mop, ra); + qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); + return cpu_to_be64(ret); +} + uint64_t cpu_ldq_le_mmu(CPUArchState *env, abi_ptr addr, MemOpIdx oi, uintptr_t ra) { - void *haddr; + MemOp mop = get_memop(oi); uint64_t ret; - validate_memop(oi, MO_LEUQ); - haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD); - ret = load_atom_8(env, ra, haddr, get_memop(oi)); - clear_helper_retaddr(); + tcg_debug_assert((mop & MO_BSWAP) == MO_LE); + ret = do_ld8_he_mmu(env, addr, mop, ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); return cpu_to_le64(ret); } @@ -1037,7 +1127,7 @@ Int128 cpu_ld16_be_mmu(CPUArchState *env, abi_ptr addr, void *haddr; Int128 ret; - validate_memop(oi, MO_128 | MO_BE); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == (MO_128 | MO_BE)); haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD); memcpy(&ret, haddr, 16); clear_helper_retaddr(); @@ -1055,7 +1145,7 @@ Int128 cpu_ld16_le_mmu(CPUArchState *env, abi_ptr addr, void *haddr; Int128 ret; - validate_memop(oi, MO_128 | MO_LE); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == (MO_128 | MO_LE)); haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD); memcpy(&ret, haddr, 16); clear_helper_retaddr(); @@ -1067,87 +1157,153 @@ Int128 cpu_ld16_le_mmu(CPUArchState *env, abi_ptr addr, return ret; } -void cpu_stb_mmu(CPUArchState *env, abi_ptr addr, uint8_t val, - MemOpIdx oi, uintptr_t ra) +static void do_st1_mmu(CPUArchState *env, abi_ptr addr, uint8_t val, + MemOp mop, uintptr_t ra) { void *haddr; - validate_memop(oi, MO_UB); - haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE); + tcg_debug_assert((mop & MO_SIZE) == MO_8); + haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_STORE); stb_p(haddr, val); clear_helper_retaddr(); +} + +void helper_stb_mmu(CPUArchState *env, target_ulong addr, uint32_t val, + MemOpIdx oi, uintptr_t ra) +{ + do_st1_mmu(env, addr, val, get_memop(oi), ra); +} + +void cpu_stb_mmu(CPUArchState *env, abi_ptr addr, uint8_t val, + MemOpIdx oi, uintptr_t ra) +{ + do_st1_mmu(env, addr, val, get_memop(oi), ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } +static void do_st2_he_mmu(CPUArchState *env, abi_ptr addr, uint16_t val, + MemOp mop, uintptr_t ra) +{ + void *haddr; + + tcg_debug_assert((mop & MO_SIZE) == MO_16); + haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_STORE); + store_atom_2(env, ra, haddr, mop, val); + clear_helper_retaddr(); +} + +void helper_stw_mmu(CPUArchState *env, target_ulong addr, uint32_t val, + MemOpIdx oi, uintptr_t ra) +{ + MemOp mop = get_memop(oi); + + if (mop & MO_BSWAP) { + val = bswap16(val); + } + do_st2_he_mmu(env, addr, val, mop, ra); +} + void cpu_stw_be_mmu(CPUArchState *env, abi_ptr addr, uint16_t val, MemOpIdx oi, uintptr_t ra) { - void *haddr; + MemOp mop = get_memop(oi); - validate_memop(oi, MO_BEUW); - haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE); - store_atom_2(env, ra, haddr, get_memop(oi), be16_to_cpu(val)); - clear_helper_retaddr(); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); -} - -void cpu_stl_be_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, - MemOpIdx oi, uintptr_t ra) -{ - void *haddr; - - validate_memop(oi, MO_BEUL); - haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE); - store_atom_4(env, ra, haddr, get_memop(oi), be32_to_cpu(val)); - clear_helper_retaddr(); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); -} - -void cpu_stq_be_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, - MemOpIdx oi, uintptr_t ra) -{ - void *haddr; - - validate_memop(oi, MO_BEUQ); - haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE); - store_atom_8(env, ra, haddr, get_memop(oi), be64_to_cpu(val)); - clear_helper_retaddr(); + tcg_debug_assert((mop & MO_BSWAP) == MO_BE); + do_st2_he_mmu(env, addr, be16_to_cpu(val), mop, ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } void cpu_stw_le_mmu(CPUArchState *env, abi_ptr addr, uint16_t val, MemOpIdx oi, uintptr_t ra) +{ + MemOp mop = get_memop(oi); + + tcg_debug_assert((mop & MO_BSWAP) == MO_LE); + do_st2_he_mmu(env, addr, le16_to_cpu(val), mop, ra); + qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); +} + +static void do_st4_he_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, + MemOp mop, uintptr_t ra) { void *haddr; - validate_memop(oi, MO_LEUW); - haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE); - store_atom_2(env, ra, haddr, get_memop(oi), le16_to_cpu(val)); + tcg_debug_assert((mop & MO_SIZE) == MO_32); + haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_STORE); + store_atom_4(env, ra, haddr, mop, val); clear_helper_retaddr(); +} + +void helper_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, + MemOpIdx oi, uintptr_t ra) +{ + MemOp mop = get_memop(oi); + + if (mop & MO_BSWAP) { + val = bswap32(val); + } + do_st4_he_mmu(env, addr, val, mop, ra); +} + +void cpu_stl_be_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, + MemOpIdx oi, uintptr_t ra) +{ + MemOp mop = get_memop(oi); + + tcg_debug_assert((mop & MO_BSWAP) == MO_BE); + do_st4_he_mmu(env, addr, be32_to_cpu(val), mop, ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } void cpu_stl_le_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, MemOpIdx oi, uintptr_t ra) +{ + MemOp mop = get_memop(oi); + + tcg_debug_assert((mop & MO_BSWAP) == MO_LE); + do_st4_he_mmu(env, addr, le32_to_cpu(val), mop, ra); + qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); +} + +static void do_st8_he_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, + MemOp mop, uintptr_t ra) { void *haddr; - validate_memop(oi, MO_LEUL); - haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE); - store_atom_4(env, ra, haddr, get_memop(oi), le32_to_cpu(val)); + tcg_debug_assert((mop & MO_SIZE) == MO_64); + haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_STORE); + store_atom_8(env, ra, haddr, mop, val); clear_helper_retaddr(); +} + +void helper_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, + MemOpIdx oi, uintptr_t ra) +{ + MemOp mop = get_memop(oi); + + if (mop & MO_BSWAP) { + val = bswap64(val); + } + do_st8_he_mmu(env, addr, val, mop, ra); +} + +void cpu_stq_be_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, + MemOpIdx oi, uintptr_t ra) +{ + MemOp mop = get_memop(oi); + + tcg_debug_assert((mop & MO_BSWAP) == MO_BE); + do_st8_he_mmu(env, addr, cpu_to_be64(val), mop, ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } void cpu_stq_le_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, MemOpIdx oi, uintptr_t ra) { - void *haddr; + MemOp mop = get_memop(oi); - validate_memop(oi, MO_LEUQ); - haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE); - store_atom_8(env, ra, haddr, get_memop(oi), le64_to_cpu(val)); - clear_helper_retaddr(); + tcg_debug_assert((mop & MO_BSWAP) == MO_LE); + do_st8_he_mmu(env, addr, cpu_to_le64(val), mop, ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } @@ -1156,7 +1312,7 @@ void cpu_st16_be_mmu(CPUArchState *env, abi_ptr addr, { void *haddr; - validate_memop(oi, MO_128 | MO_BE); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == (MO_128 | MO_BE)); haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE); if (!HOST_BIG_ENDIAN) { val = bswap128(val); @@ -1171,7 +1327,7 @@ void cpu_st16_le_mmu(CPUArchState *env, abi_ptr addr, { void *haddr; - validate_memop(oi, MO_128 | MO_LE); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == (MO_128 | MO_LE)); haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE); if (HOST_BIG_ENDIAN) { val = bswap128(val); @@ -1269,7 +1425,6 @@ uint64_t cpu_ldq_code_mmu(CPUArchState *env, abi_ptr addr, void *haddr; uint64_t ret; - validate_memop(oi, MO_BEUQ); haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD); ret = ldq_p(haddr); clear_helper_retaddr(); diff --git a/tcg/tcg.c b/tcg/tcg.c index f5f9d8f7e8..a864ff1f4b 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -197,8 +197,7 @@ static void tcg_out_st_helper_args(TCGContext *s, const TCGLabelQemuLdst *l, const TCGLdstHelperParam *p) __attribute__((unused)); -#ifdef CONFIG_SOFTMMU -static void * const qemu_ld_helpers[MO_SSIZE + 1] = { +static void * const qemu_ld_helpers[MO_SSIZE + 1] __attribute__((unused)) = { [MO_UB] = helper_ldub_mmu, [MO_SB] = helper_ldsb_mmu, [MO_UW] = helper_lduw_mmu, @@ -210,13 +209,12 @@ static void * const qemu_ld_helpers[MO_SSIZE + 1] = { #endif }; -static void * const qemu_st_helpers[MO_SIZE + 1] = { +static void * const qemu_st_helpers[MO_SIZE + 1] __attribute__((unused)) = { [MO_8] = helper_stb_mmu, [MO_16] = helper_stw_mmu, [MO_32] = helper_stl_mmu, [MO_64] = helper_stq_mmu, }; -#endif TCGContext tcg_init_ctx; __thread TCGContext *tcg_ctx; From patchwork Tue May 16 19:40:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682425 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp21117wrt; Tue, 16 May 2023 12:59:41 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7q0R0DwlraZCvigEBxKdQJsbDYJilPTobxVw1SHSTPOu9028TkMYSTeCO5pUp0RkJPl24a X-Received: by 2002:a5d:5503:0:b0:306:2d3d:a108 with SMTP id b3-20020a5d5503000000b003062d3da108mr28758459wrv.11.1684267181213; Tue, 16 May 2023 12:59:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267181; cv=none; d=google.com; s=arc-20160816; b=BWcR5nhip0Exg0zTAAPH07DAW8+uwBMx0M/IuQSSFBrTVwCNgsKaI1vJlWODaBGTmu R7tL0ue2iTVQ7Pmtfq7l15+OpP/kmc8CSsTy8zvUNK+CCiDfKiTkWd9q0Ya/dh2Vu6y1 wiFc6awCo5cF9BzTmSR7yLJNKtyssEH/NATAivUPlizRnG1kZ48s8KoTT8+oxBHDveIz f4RHmSRB+ximVd+GOGoEBzHRXYZmYU1+RcHHzAR11QWisO52ktxS/bx0W9c6bb3Hd1IB Tvtq3l6nUcP5ZCI0G041+Ohm3TKZpkmxV/sKiKzAqBAn7ICFffx+zjuiTDDqtRRnleC8 wR2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=aZUd62xGtvRPa1FRa9Mi8zYQ92RuZVwVBZs3Zj8eEbg=; b=ZO7fc9KKWhkfl8/UTs6PVxRrm3XMAJp6P8lmDN0fYqfAKj4lgOIbIQ1w91BNQLkjYe F4Tem3j26jrxw7/GQmODfdpmXsK1mBHeV9LkeA4TKBqRVUZOPkpZ7HHqtKUaRHKLEZVE pR6qrp06hJf57i8vbylxSyIRmCbXebe9te/7G+qDcujNBLsZ1aW3QV+K4IXd8vPNwoDC q5RSisv/hLYkmj5QAUm9mO77QDgcM6PsAZgHKVgNuWNlX73w1v/ZUNSDRdE9fjrBr4mw WKkkVFYpGopcOP7xPN7P7HZ3qXA4tziMr+FeBFpe9eXEgKd7md+OBdk7kowkZuYQY2xp a8Ug== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=XlVNmIKb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t14-20020a5d534e000000b002d6f7a90367si170643wrv.598.2023.05.16.12.59.41 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:59:41 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=XlVNmIKb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0YX-0006bf-PD; Tue, 16 May 2023 15:41:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0YW-0006ZW-F4 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:56 -0400 Received: from mail-pg1-x534.google.com ([2607:f8b0:4864:20::534]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0YU-0002kQ-UB for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:56 -0400 Received: by mail-pg1-x534.google.com with SMTP id 41be03b00d2f7-517ab9a4a13so13454317a12.1 for ; Tue, 16 May 2023 12:41:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266113; x=1686858113; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aZUd62xGtvRPa1FRa9Mi8zYQ92RuZVwVBZs3Zj8eEbg=; b=XlVNmIKbrrbGacnSgcgO5afg1+ME5yMMZcoSitTUmvRPk1x0rfPqlUvu55SlGIlgdF KWj2OxMKAGP291nWrFiU2L6mX+/zagcb6gvOJMTr8+cgaUGlhxvomsUiEmIhd+KG9VS+ kqCettpfBsdVt4AZ+cFoKQbGggSRCvXkhMcrsgcHE1Qj1w0tIAJuNmrp683NbubbE0et OTVRJ9Gd0rBEr6f1Ig1G51b/smA5WwM0/LpdMs/B+Dy+ogQknfFsJOlx+nEHu4Bhkw5A ZKnsyGiBhq0dWF7IvqNRbvVOY7ERKxeS7wu+ASw/C4izRo8mUmuvC/5prWoKUDX83rsz Vllw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266113; x=1686858113; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aZUd62xGtvRPa1FRa9Mi8zYQ92RuZVwVBZs3Zj8eEbg=; b=g6vLdoGJ8vjU1opq5yXJXt44L6WG2RgMMCuTCytDzu9plV4XUu7zb1Fdc3fDNp+hrx ZJRrgLyFRn3lreq2H5u62it/mwlBcihIb7fMyJMRV1GKSrc62Yoiy8s0hrfHEjIakhgu dB4P+3/h7QbBxpYZDuGMjNdsYbul8uVVI0GF9nUua8Jz23csWJ4edzGbeXD9Td7PheGC CHHKb33EeNSYvxdyfFdCB61ppV1W2P1vnZj/8QvI8EooanYiGxEgWk3/DT9fc5RjAVd4 fv+NhLOrGdfJ1j9p0phZoor5G5iTlM4IQzEmgxZmaI6kThLr+Bk9DR+N9PU0cmAAPHGm 7asQ== X-Gm-Message-State: AC+VfDxV5mrlMoGpDt/CPe9c4L8khfxamzTKNZqQwHJzLmj5MxvLMkBr K00XEbacaBvX1AMfhVkJTWPPsqCGopAtJk/Qstg= X-Received: by 2002:a05:6a21:3290:b0:100:9969:8cf with SMTP id yt16-20020a056a21329000b00100996908cfmr40995870pzb.49.1684266113671; Tue, 16 May 2023 12:41:53 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.41.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:41:53 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 07/80] tcg/tci: Use helper_{ld,st}*_mmu for user-only Date: Tue, 16 May 2023 12:40:32 -0700 Message-Id: <20230516194145.1749305-8-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::534; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x534.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org We can now fold these two pieces of code. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/tci.c | 89 ------------------------------------------------------- 1 file changed, 89 deletions(-) diff --git a/tcg/tci.c b/tcg/tci.c index 5bde2e1f2e..15f2f8c463 100644 --- a/tcg/tci.c +++ b/tcg/tci.c @@ -292,7 +292,6 @@ static uint64_t tci_qemu_ld(CPUArchState *env, target_ulong taddr, MemOp mop = get_memop(oi); uintptr_t ra = (uintptr_t)tb_ptr; -#ifdef CONFIG_SOFTMMU switch (mop & MO_SSIZE) { case MO_UB: return helper_ldub_mmu(env, taddr, oi, ra); @@ -311,58 +310,6 @@ static uint64_t tci_qemu_ld(CPUArchState *env, target_ulong taddr, default: g_assert_not_reached(); } -#else - void *haddr = g2h(env_cpu(env), taddr); - unsigned a_mask = (1u << get_alignment_bits(mop)) - 1; - uint64_t ret; - - set_helper_retaddr(ra); - if (taddr & a_mask) { - helper_unaligned_ld(env, taddr); - } - switch (mop & (MO_BSWAP | MO_SSIZE)) { - case MO_UB: - ret = ldub_p(haddr); - break; - case MO_SB: - ret = ldsb_p(haddr); - break; - case MO_LEUW: - ret = lduw_le_p(haddr); - break; - case MO_LESW: - ret = ldsw_le_p(haddr); - break; - case MO_LEUL: - ret = (uint32_t)ldl_le_p(haddr); - break; - case MO_LESL: - ret = (int32_t)ldl_le_p(haddr); - break; - case MO_LEUQ: - ret = ldq_le_p(haddr); - break; - case MO_BEUW: - ret = lduw_be_p(haddr); - break; - case MO_BESW: - ret = ldsw_be_p(haddr); - break; - case MO_BEUL: - ret = (uint32_t)ldl_be_p(haddr); - break; - case MO_BESL: - ret = (int32_t)ldl_be_p(haddr); - break; - case MO_BEUQ: - ret = ldq_be_p(haddr); - break; - default: - g_assert_not_reached(); - } - clear_helper_retaddr(); - return ret; -#endif } static void tci_qemu_st(CPUArchState *env, target_ulong taddr, uint64_t val, @@ -371,7 +318,6 @@ static void tci_qemu_st(CPUArchState *env, target_ulong taddr, uint64_t val, MemOp mop = get_memop(oi); uintptr_t ra = (uintptr_t)tb_ptr; -#ifdef CONFIG_SOFTMMU switch (mop & MO_SIZE) { case MO_UB: helper_stb_mmu(env, taddr, val, oi, ra); @@ -388,41 +334,6 @@ static void tci_qemu_st(CPUArchState *env, target_ulong taddr, uint64_t val, default: g_assert_not_reached(); } -#else - void *haddr = g2h(env_cpu(env), taddr); - unsigned a_mask = (1u << get_alignment_bits(mop)) - 1; - - set_helper_retaddr(ra); - if (taddr & a_mask) { - helper_unaligned_st(env, taddr); - } - switch (mop & (MO_BSWAP | MO_SIZE)) { - case MO_UB: - stb_p(haddr, val); - break; - case MO_LEUW: - stw_le_p(haddr, val); - break; - case MO_LEUL: - stl_le_p(haddr, val); - break; - case MO_LEUQ: - stq_le_p(haddr, val); - break; - case MO_BEUW: - stw_be_p(haddr, val); - break; - case MO_BEUL: - stl_be_p(haddr, val); - break; - case MO_BEUQ: - stq_be_p(haddr, val); - break; - default: - g_assert_not_reached(); - } - clear_helper_retaddr(); -#endif } #if TCG_TARGET_REG_BITS == 64 From patchwork Tue May 16 19:40:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682359 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp638787ltp; Tue, 16 May 2023 12:42:57 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6T05X0PEnUH4qhQetiKZ2JTMeubiHdRELwbKEePV1HZubRspHZUmHIGi7GmKudR9Np9tEI X-Received: by 2002:a05:622a:216:b0:3f3:9298:3335 with SMTP id b22-20020a05622a021600b003f392983335mr51548407qtx.15.1684266177369; Tue, 16 May 2023 12:42:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266177; cv=none; d=google.com; s=arc-20160816; b=XwBgM8uP3TA1DMxV5v3ebLLnI8GU0TYLbTMOWLvl95J2VsuCJ1m1SxSkW5PZoMyxNp Hk/+EzTvRO+3Vrji/Vyuc+Wi7Uv3qBFHUUEL7w0TOz0HSa46YgmwfWkiaawTQzVK3XgB ecR4IBYzNb0MZlb32tbWiLwFQyP5brO6nuAaTSip15I7Y1Cxx1yf12R73g22loKilI6k x/KHmCS31jLPtNK9Hr7WryJuHDKIxm6q2WOLGkiG5fMGzRtlsnnoJp8VHOweZL3UgOqD Yp9Mej0ineJbJd+Wi2XOTkv2cIRVDaAPQ/UYZ3EeECrXCdsjT+0w6xENB8dXSy+e3Hm8 SWEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=IlW0xgIWcuqR7siElpNgCs7PK36ClUeHwwWlG1M+QvA=; b=kuDiod5fS2eBgBw3FVn+yJZkqSncWUD6F3MAzn6+pirUyP3APkUpRWTUiuzcLF5toH x/dhY5rwEfrPb4KVpDa95cTW7gEZhF2d/ym4/BlWE72rb5IaEfnxButEoFqqNwTtU3W/ 4Nwe6NHaawXUmn0Ge2NTiYvEcudUdx+yPtBsJ+ze+WYWVWw1af+ETtGcgxGEwEuCmIil HlLF8iqhTLn7nDzB5uDsaf82QJjvPWHtdMXLi51diywIiV2KHztUvWmWEM2TLgIRy3Se 8zbbs2nlmPQsoDrCGzuguG/L3a3I6vX6UnpeBOXsWCNYPTVxSIPhAM4/T7+ZIN2G6xk9 ZgHQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=aLBeiqT9; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b13-20020ac87fcd000000b003f57ce75042si263041qtk.732.2023.05.16.12.42.57 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:42:57 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=aLBeiqT9; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yb-0006dY-3O; Tue, 16 May 2023 15:42:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0YZ-0006cR-GX for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:59 -0400 Received: from mail-pf1-x42e.google.com ([2607:f8b0:4864:20::42e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0YW-0002kn-18 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:59 -0400 Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-643a1656b79so10540543b3a.3 for ; Tue, 16 May 2023 12:41:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266115; x=1686858115; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IlW0xgIWcuqR7siElpNgCs7PK36ClUeHwwWlG1M+QvA=; b=aLBeiqT9QE8lRZ/g9Rp8hyQ+ppzofDGP04yuoXBPOcrvs9LUBynSRnWZk80uPgqC5q Udu+5cjJzYF1XTPMsVXeyoK5eJxQAcXXsR8CXSTbA+/y27gNfm8u83Vd/1bgg/RzDbff Q2X6iNuqbEUEanAGc63+3ZBW+2SFXI1troyCVT0qerM8OJmLVwFpUa4gvSBLOsI1J+AQ Da9zP7RA3MxQE411L5qo6X8aUj2jYo49KcaKcyfS2n9xRYDI94mwpWxdjxut9TzkWpfL FJ1MFsG8xdqcw/J7YecyNXDIgcLtdWvRqNeVgnzP+ItH4+RUPBD3ru7SoSRK4GGJurmI tn3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266115; x=1686858115; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IlW0xgIWcuqR7siElpNgCs7PK36ClUeHwwWlG1M+QvA=; b=DmTiQ1H6BsjL/m5fs42nosbg69LAJJN+brcAk5Fyki23IF9oqdS0Nqk6/IPIJ4acFK HunadMBPt2Ni8JqoiKRNREa2wQUsn74zdXYF8qHSxcMYXQFA8GWMdjsailen9ZGfdSmP JPKjDmAK8IuyqYICDGGBqCFBbqnPjqL14cQnPu6G6KaMTcNSsbSNoW5T0T3yxLfGuoNv fNsVtQ6ulAo553ybIdsahBhAXXBTpem8fxK2mlRO8bWMb/6ojLEbSdZ2w3oVY74yPw0P bbI5EU7u27bgWtf9Rz+aNCEBdEMLwNV8uXUyPWvUAw8hCYmfpnmhkh8Tv6t472QfekMJ wyzQ== X-Gm-Message-State: AC+VfDwI3W73IcKP9EkS/yb0taPnhynMlxXHhSaSaVp7zS2+pGM5bbDi Y6bFQ9cXoBrUqpT8/nUFF8ifZU6rlgyta2ufuC4= X-Received: by 2002:a05:6a00:179b:b0:63b:62d1:d868 with SMTP id s27-20020a056a00179b00b0063b62d1d868mr58776361pfg.8.1684266114511; Tue, 16 May 2023 12:41:54 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.41.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:41:54 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 08/80] tcg: Add 128-bit guest memory primitives Date: Tue, 16 May 2023 12:40:33 -0700 Message-Id: <20230516194145.1749305-9-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42e; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- accel/tcg/tcg-runtime.h | 3 + include/tcg/tcg-ldst.h | 4 + accel/tcg/cputlb.c | 399 +++++++++++++++++++++++++-------- accel/tcg/user-exec.c | 94 ++++++-- tcg/tcg-op.c | 173 +++++++++----- accel/tcg/ldst_atomicity.c.inc | 184 +++++++++++++++ 6 files changed, 679 insertions(+), 178 deletions(-) diff --git a/accel/tcg/tcg-runtime.h b/accel/tcg/tcg-runtime.h index b8e6421c8a..d9adc646c1 100644 --- a/accel/tcg/tcg-runtime.h +++ b/accel/tcg/tcg-runtime.h @@ -39,6 +39,9 @@ DEF_HELPER_FLAGS_1(exit_atomic, TCG_CALL_NO_WG, noreturn, env) DEF_HELPER_FLAGS_3(memset, TCG_CALL_NO_RWG, ptr, ptr, int, ptr) #endif /* IN_HELPER_PROTO */ +DEF_HELPER_FLAGS_3(ld_i128, TCG_CALL_NO_WG, i128, env, tl, i32) +DEF_HELPER_FLAGS_4(st_i128, TCG_CALL_NO_WG, void, env, tl, i128, i32) + DEF_HELPER_FLAGS_5(atomic_cmpxchgb, TCG_CALL_NO_WG, i32, env, tl, i32, i32, i32) DEF_HELPER_FLAGS_5(atomic_cmpxchgw_be, TCG_CALL_NO_WG, diff --git a/include/tcg/tcg-ldst.h b/include/tcg/tcg-ldst.h index 57fafa14b1..64f48e6990 100644 --- a/include/tcg/tcg-ldst.h +++ b/include/tcg/tcg-ldst.h @@ -34,6 +34,8 @@ tcg_target_ulong helper_ldul_mmu(CPUArchState *env, target_ulong addr, MemOpIdx oi, uintptr_t retaddr); uint64_t helper_ldq_mmu(CPUArchState *env, target_ulong addr, MemOpIdx oi, uintptr_t retaddr); +Int128 helper_ld16_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t retaddr); /* Value sign-extended to tcg register size. */ tcg_target_ulong helper_ldsb_mmu(CPUArchState *env, target_ulong addr, @@ -55,6 +57,8 @@ void helper_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, MemOpIdx oi, uintptr_t retaddr); void helper_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, MemOpIdx oi, uintptr_t retaddr); +void helper_st16_mmu(CPUArchState *env, target_ulong addr, Int128 val, + MemOpIdx oi, uintptr_t retaddr); #ifdef CONFIG_USER_ONLY diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 34796ef568..49e49f75a4 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -40,6 +40,7 @@ #include "qemu/plugin-memory.h" #endif #include "tcg/tcg-ldst.h" +#include "exec/helper-proto.h" /* DEBUG defines, enable DEBUG_TLB_LOG to log to the CPU_LOG_MMU target */ /* #define DEBUG_TLB */ @@ -2162,6 +2163,31 @@ static uint64_t do_ld_whole_be8(CPUArchState *env, uintptr_t ra, return (ret_be << (p->size * 8)) | x; } +/** + * do_ld_parts_be16 + * @p: translation parameters + * @ret_be: accumulated data + * + * As do_ld_bytes_beN, but with one atomic load. + * 16 aligned bytes are guaranteed to cover the load. + */ +static Int128 do_ld_whole_be16(CPUArchState *env, uintptr_t ra, + MMULookupPageData *p, uint64_t ret_be) +{ + int o = p->addr & 15; + Int128 x, y = load_atomic16_or_exit(env, ra, p->haddr - o); + int size = p->size; + + if (!HOST_BIG_ENDIAN) { + y = bswap128(y); + } + y = int128_lshift(y, o * 8); + y = int128_urshift(y, (16 - size) * 8); + x = int128_make64(ret_be); + x = int128_lshift(x, size * 8); + return int128_or(x, y); +} + /* * Wrapper for the above. */ @@ -2211,6 +2237,63 @@ static uint64_t do_ld_beN(CPUArchState *env, MMULookupPageData *p, } } +/* + * Wrapper for the above, for 8 < size < 16. + */ +static Int128 do_ld16_beN(CPUArchState *env, MMULookupPageData *p, + uint64_t a, int mmu_idx, MemOp mop, uintptr_t ra) +{ + int size = p->size; + uint64_t b; + MemOp atom; + + if (unlikely(p->flags & TLB_MMIO)) { + p->size = size - 8; + a = do_ld_mmio_beN(env, p, a, mmu_idx, MMU_DATA_LOAD, ra); + p->addr += p->size; + p->size = 8; + b = do_ld_mmio_beN(env, p, 0, mmu_idx, MMU_DATA_LOAD, ra); + return int128_make128(b, a); + } + + /* + * It is a given that we cross a page and therefore there is no + * atomicity for the load as a whole, but subobjects may need attention. + */ + atom = mop & MO_ATOM_MASK; + switch (atom) { + case MO_ATOM_SUBALIGN: + p->size = size - 8; + a = do_ld_parts_beN(p, a); + p->haddr += size - 8; + p->size = 8; + b = do_ld_parts_beN(p, 0); + break; + + case MO_ATOM_WITHIN16_PAIR: + /* Since size > 8, this is the half that must be atomic. */ + return do_ld_whole_be16(env, ra, p, a); + + case MO_ATOM_IFALIGN_PAIR: + /* + * Since size > 8, both halves are misaligned, + * and so neither is atomic. + */ + case MO_ATOM_IFALIGN: + case MO_ATOM_WITHIN16: + case MO_ATOM_NONE: + p->size = size - 8; + a = do_ld_bytes_beN(p, a); + b = ldq_be_p(p->haddr + size - 8); + break; + + default: + g_assert_not_reached(); + } + + return int128_make128(b, a); +} + static uint8_t do_ld_1(CPUArchState *env, MMULookupPageData *p, int mmu_idx, MMUAccessType type, uintptr_t ra) { @@ -2399,6 +2482,80 @@ tcg_target_ulong helper_ldsl_mmu(CPUArchState *env, target_ulong addr, return (int32_t)helper_ldul_mmu(env, addr, oi, retaddr); } +static Int128 do_ld16_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t ra) +{ + MMULookupLocals l; + bool crosspage; + uint64_t a, b; + Int128 ret; + int first; + + crosspage = mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD, &l); + if (likely(!crosspage)) { + /* Perform the load host endian. */ + if (unlikely(l.page[0].flags & TLB_MMIO)) { + QEMU_IOTHREAD_LOCK_GUARD(); + a = io_readx(env, l.page[0].full, l.mmu_idx, addr, + ra, MMU_DATA_LOAD, MO_64); + b = io_readx(env, l.page[0].full, l.mmu_idx, addr + 8, + ra, MMU_DATA_LOAD, MO_64); + ret = int128_make128(HOST_BIG_ENDIAN ? b : a, + HOST_BIG_ENDIAN ? a : b); + } else { + ret = load_atom_16(env, ra, l.page[0].haddr, l.memop); + } + if (l.memop & MO_BSWAP) { + ret = bswap128(ret); + } + return ret; + } + + first = l.page[0].size; + if (first == 8) { + MemOp mop8 = (l.memop & ~MO_SIZE) | MO_64; + + a = do_ld_8(env, &l.page[0], l.mmu_idx, MMU_DATA_LOAD, mop8, ra); + b = do_ld_8(env, &l.page[1], l.mmu_idx, MMU_DATA_LOAD, mop8, ra); + if ((mop8 & MO_BSWAP) == MO_LE) { + ret = int128_make128(a, b); + } else { + ret = int128_make128(b, a); + } + return ret; + } + + if (first < 8) { + a = do_ld_beN(env, &l.page[0], 0, l.mmu_idx, + MMU_DATA_LOAD, l.memop, ra); + ret = do_ld16_beN(env, &l.page[1], a, l.mmu_idx, l.memop, ra); + } else { + ret = do_ld16_beN(env, &l.page[0], 0, l.mmu_idx, l.memop, ra); + b = int128_getlo(ret); + ret = int128_lshift(ret, l.page[1].size * 8); + a = int128_gethi(ret); + b = do_ld_beN(env, &l.page[1], b, l.mmu_idx, + MMU_DATA_LOAD, l.memop, ra); + ret = int128_make128(b, a); + } + if ((l.memop & MO_BSWAP) == MO_LE) { + ret = bswap128(ret); + } + return ret; +} + +Int128 helper_ld16_mmu(CPUArchState *env, target_ulong addr, + uint32_t oi, uintptr_t retaddr) +{ + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_128); + return do_ld16_mmu(env, addr, oi, retaddr); +} + +Int128 helper_ld_i128(CPUArchState *env, target_ulong addr, uint32_t oi) +{ + return helper_ld16_mmu(env, addr, oi, GETPC()); +} + /* * Load helpers for cpu_ldst.h. */ @@ -2487,59 +2644,23 @@ uint64_t cpu_ldq_le_mmu(CPUArchState *env, abi_ptr addr, Int128 cpu_ld16_be_mmu(CPUArchState *env, abi_ptr addr, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - int mmu_idx = get_mmuidx(oi); - MemOpIdx new_oi; - unsigned a_bits; - uint64_t h, l; + Int128 ret; - tcg_debug_assert((mop & (MO_BSWAP|MO_SSIZE)) == (MO_BE|MO_128)); - a_bits = get_alignment_bits(mop); - - /* Handle CPU specific unaligned behaviour */ - if (addr & ((1 << a_bits) - 1)) { - cpu_unaligned_access(env_cpu(env), addr, MMU_DATA_LOAD, - mmu_idx, ra); - } - - /* Construct an unaligned 64-bit replacement MemOpIdx. */ - mop = (mop & ~(MO_SIZE | MO_AMASK)) | MO_64 | MO_UNALN; - new_oi = make_memop_idx(mop, mmu_idx); - - h = helper_ldq_mmu(env, addr, new_oi, ra); - l = helper_ldq_mmu(env, addr + 8, new_oi, ra); - - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return int128_make128(l, h); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP|MO_SIZE)) == (MO_BE|MO_128)); + ret = do_ld16_mmu(env, addr, oi, ra); + plugin_load_cb(env, addr, oi); + return ret; } Int128 cpu_ld16_le_mmu(CPUArchState *env, abi_ptr addr, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - int mmu_idx = get_mmuidx(oi); - MemOpIdx new_oi; - unsigned a_bits; - uint64_t h, l; + Int128 ret; - tcg_debug_assert((mop & (MO_BSWAP|MO_SSIZE)) == (MO_LE|MO_128)); - a_bits = get_alignment_bits(mop); - - /* Handle CPU specific unaligned behaviour */ - if (addr & ((1 << a_bits) - 1)) { - cpu_unaligned_access(env_cpu(env), addr, MMU_DATA_LOAD, - mmu_idx, ra); - } - - /* Construct an unaligned 64-bit replacement MemOpIdx. */ - mop = (mop & ~(MO_SIZE | MO_AMASK)) | MO_64 | MO_UNALN; - new_oi = make_memop_idx(mop, mmu_idx); - - l = helper_ldq_mmu(env, addr, new_oi, ra); - h = helper_ldq_mmu(env, addr + 8, new_oi, ra); - - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return int128_make128(l, h); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP|MO_SIZE)) == (MO_LE|MO_128)); + ret = do_ld16_mmu(env, addr, oi, ra); + plugin_load_cb(env, addr, oi); + return ret; } /* @@ -2625,6 +2746,60 @@ static uint64_t do_st_leN(CPUArchState *env, MMULookupPageData *p, } } +/* + * Wrapper for the above, for 8 < size < 16. + */ +static uint64_t do_st16_leN(CPUArchState *env, MMULookupPageData *p, + Int128 val_le, int mmu_idx, + MemOp mop, uintptr_t ra) +{ + int size = p->size; + MemOp atom; + + if (unlikely(p->flags & TLB_MMIO)) { + p->size = 8; + do_st_mmio_leN(env, p, int128_getlo(val_le), mmu_idx, ra); + p->size = size - 8; + p->addr += 8; + return do_st_mmio_leN(env, p, int128_gethi(val_le), mmu_idx, ra); + } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { + return int128_gethi(val_le) >> ((size - 8) * 8); + } + + /* + * It is a given that we cross a page and therefore there is no atomicity + * for the store as a whole, but subobjects may need attention. + */ + atom = mop & MO_ATOM_MASK; + switch (atom) { + case MO_ATOM_SUBALIGN: + store_parts_leN(p->haddr, 8, int128_getlo(val_le)); + return store_parts_leN(p->haddr + 8, p->size - 8, + int128_gethi(val_le)); + + case MO_ATOM_WITHIN16_PAIR: + /* Since size > 8, this is the half that must be atomic. */ + if (!HAVE_al16) { + cpu_loop_exit_atomic(env_cpu(env), ra); + } + return store_whole_le16(p->haddr, p->size, val_le); + + case MO_ATOM_IFALIGN_PAIR: + /* + * Since size > 8, both halves are misaligned, + * and so neither is atomic. + */ + case MO_ATOM_IFALIGN: + case MO_ATOM_NONE: + stq_le_p(p->haddr, int128_getlo(val_le)); + return store_bytes_leN(p->haddr + 8, p->size - 8, + int128_gethi(val_le)); + + default: + g_assert_not_reached(); + } +} + static void do_st_1(CPUArchState *env, MMULookupPageData *p, uint8_t val, int mmu_idx, uintptr_t ra) { @@ -2781,6 +2956,80 @@ void helper_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, do_st8_mmu(env, addr, val, oi, retaddr); } +static void do_st16_mmu(CPUArchState *env, target_ulong addr, Int128 val, + MemOpIdx oi, uintptr_t ra) +{ + MMULookupLocals l; + bool crosspage; + uint64_t a, b; + int first; + + crosspage = mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE, &l); + if (likely(!crosspage)) { + /* Swap to host endian if necessary, then store. */ + if (l.memop & MO_BSWAP) { + val = bswap128(val); + } + if (unlikely(l.page[0].flags & TLB_MMIO)) { + QEMU_IOTHREAD_LOCK_GUARD(); + if (HOST_BIG_ENDIAN) { + b = int128_getlo(val), a = int128_gethi(val); + } else { + a = int128_getlo(val), b = int128_gethi(val); + } + io_writex(env, l.page[0].full, l.mmu_idx, a, addr, ra, MO_64); + io_writex(env, l.page[0].full, l.mmu_idx, b, addr + 8, ra, MO_64); + } else if (unlikely(l.page[0].flags & TLB_DISCARD_WRITE)) { + /* nothing */ + } else { + store_atom_16(env, ra, l.page[0].haddr, l.memop, val); + } + return; + } + + first = l.page[0].size; + if (first == 8) { + MemOp mop8 = (l.memop & ~(MO_SIZE | MO_BSWAP)) | MO_64; + + if (l.memop & MO_BSWAP) { + val = bswap128(val); + } + if (HOST_BIG_ENDIAN) { + b = int128_getlo(val), a = int128_gethi(val); + } else { + a = int128_getlo(val), b = int128_gethi(val); + } + do_st_8(env, &l.page[0], a, l.mmu_idx, mop8, ra); + do_st_8(env, &l.page[1], b, l.mmu_idx, mop8, ra); + return; + } + + if ((l.memop & MO_BSWAP) != MO_LE) { + val = bswap128(val); + } + if (first < 8) { + do_st_leN(env, &l.page[0], int128_getlo(val), l.mmu_idx, l.memop, ra); + val = int128_urshift(val, first * 8); + do_st16_leN(env, &l.page[1], val, l.mmu_idx, l.memop, ra); + } else { + b = do_st16_leN(env, &l.page[0], val, l.mmu_idx, l.memop, ra); + do_st_leN(env, &l.page[1], b, l.mmu_idx, l.memop, ra); + } +} + +void helper_st16_mmu(CPUArchState *env, target_ulong addr, Int128 val, + MemOpIdx oi, uintptr_t retaddr) +{ + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_128); + do_st16_mmu(env, addr, val, oi, retaddr); +} + +void helper_st_i128(CPUArchState *env, target_ulong addr, Int128 val, + MemOpIdx oi) +{ + helper_st16_mmu(env, addr, val, oi, GETPC()); +} + /* * Store Helpers for cpu_ldst.h */ @@ -2845,58 +3094,20 @@ void cpu_stq_le_mmu(CPUArchState *env, target_ulong addr, uint64_t val, plugin_store_cb(env, addr, oi); } -void cpu_st16_be_mmu(CPUArchState *env, abi_ptr addr, Int128 val, - MemOpIdx oi, uintptr_t ra) +void cpu_st16_be_mmu(CPUArchState *env, target_ulong addr, Int128 val, + MemOpIdx oi, uintptr_t retaddr) { - MemOp mop = get_memop(oi); - int mmu_idx = get_mmuidx(oi); - MemOpIdx new_oi; - unsigned a_bits; - - tcg_debug_assert((mop & (MO_BSWAP|MO_SSIZE)) == (MO_BE|MO_128)); - a_bits = get_alignment_bits(mop); - - /* Handle CPU specific unaligned behaviour */ - if (addr & ((1 << a_bits) - 1)) { - cpu_unaligned_access(env_cpu(env), addr, MMU_DATA_STORE, - mmu_idx, ra); - } - - /* Construct an unaligned 64-bit replacement MemOpIdx. */ - mop = (mop & ~(MO_SIZE | MO_AMASK)) | MO_64 | MO_UNALN; - new_oi = make_memop_idx(mop, mmu_idx); - - helper_stq_mmu(env, addr, int128_gethi(val), new_oi, ra); - helper_stq_mmu(env, addr + 8, int128_getlo(val), new_oi, ra); - - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP|MO_SIZE)) == (MO_BE|MO_128)); + do_st16_mmu(env, addr, val, oi, retaddr); + plugin_store_cb(env, addr, oi); } -void cpu_st16_le_mmu(CPUArchState *env, abi_ptr addr, Int128 val, - MemOpIdx oi, uintptr_t ra) +void cpu_st16_le_mmu(CPUArchState *env, target_ulong addr, Int128 val, + MemOpIdx oi, uintptr_t retaddr) { - MemOp mop = get_memop(oi); - int mmu_idx = get_mmuidx(oi); - MemOpIdx new_oi; - unsigned a_bits; - - tcg_debug_assert((mop & (MO_BSWAP|MO_SSIZE)) == (MO_LE|MO_128)); - a_bits = get_alignment_bits(mop); - - /* Handle CPU specific unaligned behaviour */ - if (addr & ((1 << a_bits) - 1)) { - cpu_unaligned_access(env_cpu(env), addr, MMU_DATA_STORE, - mmu_idx, ra); - } - - /* Construct an unaligned 64-bit replacement MemOpIdx. */ - mop = (mop & ~(MO_SIZE | MO_AMASK)) | MO_64 | MO_UNALN; - new_oi = make_memop_idx(mop, mmu_idx); - - helper_stq_mmu(env, addr, int128_getlo(val), new_oi, ra); - helper_stq_mmu(env, addr + 8, int128_gethi(val), new_oi, ra); - - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); + tcg_debug_assert((get_memop(oi) & (MO_BSWAP|MO_SIZE)) == (MO_LE|MO_128)); + do_st16_mmu(env, addr, val, oi, retaddr); + plugin_store_cb(env, addr, oi); } #include "ldst_common.c.inc" diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index d9f9766b7f..8f86254eb4 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -1121,18 +1121,45 @@ uint64_t cpu_ldq_le_mmu(CPUArchState *env, abi_ptr addr, return cpu_to_le64(ret); } -Int128 cpu_ld16_be_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) +static Int128 do_ld16_he_mmu(CPUArchState *env, abi_ptr addr, + MemOp mop, uintptr_t ra) { void *haddr; Int128 ret; - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == (MO_128 | MO_BE)); - haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD); - memcpy(&ret, haddr, 16); + tcg_debug_assert((mop & MO_SIZE) == MO_128); + haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_LOAD); + ret = load_atom_16(env, ra, haddr, mop); clear_helper_retaddr(); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); + return ret; +} +Int128 helper_ld16_mmu(CPUArchState *env, target_ulong addr, + MemOpIdx oi, uintptr_t ra) +{ + MemOp mop = get_memop(oi); + Int128 ret = do_ld16_he_mmu(env, addr, mop, ra); + + if (mop & MO_BSWAP) { + ret = bswap128(ret); + } + return ret; +} + +Int128 helper_ld_i128(CPUArchState *env, target_ulong addr, MemOpIdx oi) +{ + return helper_ld16_mmu(env, addr, oi, GETPC()); +} + +Int128 cpu_ld16_be_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) +{ + MemOp mop = get_memop(oi); + Int128 ret; + + tcg_debug_assert((mop & MO_BSWAP) == MO_BE); + ret = do_ld16_he_mmu(env, addr, mop, ra); + qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); if (!HOST_BIG_ENDIAN) { ret = bswap128(ret); } @@ -1142,15 +1169,12 @@ Int128 cpu_ld16_be_mmu(CPUArchState *env, abi_ptr addr, Int128 cpu_ld16_le_mmu(CPUArchState *env, abi_ptr addr, MemOpIdx oi, uintptr_t ra) { - void *haddr; + MemOp mop = get_memop(oi); Int128 ret; - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == (MO_128 | MO_LE)); - haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD); - memcpy(&ret, haddr, 16); - clear_helper_retaddr(); + tcg_debug_assert((mop & MO_BSWAP) == MO_LE); + ret = do_ld16_he_mmu(env, addr, mop, ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - if (HOST_BIG_ENDIAN) { ret = bswap128(ret); } @@ -1307,33 +1331,57 @@ void cpu_stq_le_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } -void cpu_st16_be_mmu(CPUArchState *env, abi_ptr addr, - Int128 val, MemOpIdx oi, uintptr_t ra) +static void do_st16_he_mmu(CPUArchState *env, abi_ptr addr, Int128 val, + MemOp mop, uintptr_t ra) { void *haddr; - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == (MO_128 | MO_BE)); - haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE); + tcg_debug_assert((mop & MO_SIZE) == MO_128); + haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_STORE); + store_atom_16(env, ra, haddr, mop, val); + clear_helper_retaddr(); +} + +void helper_st16_mmu(CPUArchState *env, target_ulong addr, Int128 val, + MemOpIdx oi, uintptr_t ra) +{ + MemOp mop = get_memop(oi); + + if (mop & MO_BSWAP) { + val = bswap128(val); + } + do_st16_he_mmu(env, addr, val, mop, ra); +} + +void helper_st_i128(CPUArchState *env, target_ulong addr, + Int128 val, MemOpIdx oi) +{ + helper_st16_mmu(env, addr, val, oi, GETPC()); +} + +void cpu_st16_be_mmu(CPUArchState *env, abi_ptr addr, + Int128 val, MemOpIdx oi, uintptr_t ra) +{ + MemOp mop = get_memop(oi); + + tcg_debug_assert((mop & MO_BSWAP) == MO_BE); if (!HOST_BIG_ENDIAN) { val = bswap128(val); } - memcpy(haddr, &val, 16); - clear_helper_retaddr(); + do_st16_he_mmu(env, addr, val, mop, ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } void cpu_st16_le_mmu(CPUArchState *env, abi_ptr addr, Int128 val, MemOpIdx oi, uintptr_t ra) { - void *haddr; + MemOp mop = get_memop(oi); - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == (MO_128 | MO_LE)); - haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE); + tcg_debug_assert((mop & MO_BSWAP) == MO_LE); if (HOST_BIG_ENDIAN) { val = bswap128(val); } - memcpy(haddr, &val, 16); - clear_helper_retaddr(); + do_st16_he_mmu(env, addr, val, mop, ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } diff --git a/tcg/tcg-op.c b/tcg/tcg-op.c index 3136cef81a..22481a344c 100644 --- a/tcg/tcg-op.c +++ b/tcg/tcg-op.c @@ -3119,6 +3119,37 @@ void tcg_gen_qemu_st_i64(TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop) } } +/* + * Return true if @mop, without knowledge of the pointer alignment, + * does not require 16-byte atomicity, and it would be adventagous + * to avoid a call to a helper function. + */ +static bool use_two_i64_for_i128(MemOp mop) +{ +#ifdef CONFIG_SOFTMMU + /* Two softmmu tlb lookups is larger than one function call. */ + return false; +#else + /* + * For user-only, two 64-bit operations may well be smaller than a call. + * Determine if that would be legal for the requested atomicity. + */ + switch (mop & MO_ATOM_MASK) { + case MO_ATOM_NONE: + case MO_ATOM_IFALIGN_PAIR: + return true; + case MO_ATOM_IFALIGN: + case MO_ATOM_SUBALIGN: + case MO_ATOM_WITHIN16: + case MO_ATOM_WITHIN16_PAIR: + /* In a serialized context, no atomicity is required. */ + return !(tcg_ctx->gen_tb->cflags & CF_PARALLEL); + default: + g_assert_not_reached(); + } +#endif +} + static void canonicalize_memop_i128_as_i64(MemOp ret[2], MemOp orig) { MemOp mop_1 = orig, mop_2; @@ -3164,93 +3195,113 @@ static void canonicalize_memop_i128_as_i64(MemOp ret[2], MemOp orig) ret[1] = mop_2; } +#if TARGET_LONG_BITS == 64 +#define tcg_temp_ebb_new tcg_temp_ebb_new_i64 +#else +#define tcg_temp_ebb_new tcg_temp_ebb_new_i32 +#endif + void tcg_gen_qemu_ld_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) { - MemOp mop[2]; - TCGv addr_p8; - TCGv_i64 x, y; + MemOpIdx oi = make_memop_idx(memop, idx); - canonicalize_memop_i128_as_i64(mop, memop); + tcg_debug_assert((memop & MO_SIZE) == MO_128); + tcg_debug_assert((memop & MO_SIGN) == 0); tcg_gen_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); addr = plugin_prep_mem_callbacks(addr); - /* TODO: respect atomicity of the operation. */ /* TODO: allow the tcg backend to see the whole operation. */ - /* - * Since there are no global TCGv_i128, there is no visible state - * changed if the second load faults. Load directly into the two - * subwords. - */ - if ((memop & MO_BSWAP) == MO_LE) { - x = TCGV128_LOW(val); - y = TCGV128_HIGH(val); + if (use_two_i64_for_i128(memop)) { + MemOp mop[2]; + TCGv addr_p8; + TCGv_i64 x, y; + + canonicalize_memop_i128_as_i64(mop, memop); + + /* + * Since there are no global TCGv_i128, there is no visible state + * changed if the second load faults. Load directly into the two + * subwords. + */ + if ((memop & MO_BSWAP) == MO_LE) { + x = TCGV128_LOW(val); + y = TCGV128_HIGH(val); + } else { + x = TCGV128_HIGH(val); + y = TCGV128_LOW(val); + } + + gen_ldst_i64(INDEX_op_qemu_ld_i64, x, addr, mop[0], idx); + + if ((mop[0] ^ memop) & MO_BSWAP) { + tcg_gen_bswap64_i64(x, x); + } + + addr_p8 = tcg_temp_ebb_new(); + tcg_gen_addi_tl(addr_p8, addr, 8); + gen_ldst_i64(INDEX_op_qemu_ld_i64, y, addr_p8, mop[1], idx); + tcg_temp_free(addr_p8); + + if ((mop[0] ^ memop) & MO_BSWAP) { + tcg_gen_bswap64_i64(y, y); + } } else { - x = TCGV128_HIGH(val); - y = TCGV128_LOW(val); + gen_helper_ld_i128(val, cpu_env, addr, tcg_constant_i32(oi)); } - gen_ldst_i64(INDEX_op_qemu_ld_i64, x, addr, mop[0], idx); - - if ((mop[0] ^ memop) & MO_BSWAP) { - tcg_gen_bswap64_i64(x, x); - } - - addr_p8 = tcg_temp_new(); - tcg_gen_addi_tl(addr_p8, addr, 8); - gen_ldst_i64(INDEX_op_qemu_ld_i64, y, addr_p8, mop[1], idx); - tcg_temp_free(addr_p8); - - if ((mop[0] ^ memop) & MO_BSWAP) { - tcg_gen_bswap64_i64(y, y); - } - - plugin_gen_mem_callbacks(addr, make_memop_idx(memop, idx), - QEMU_PLUGIN_MEM_R); + plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_R); } void tcg_gen_qemu_st_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) { - MemOp mop[2]; - TCGv addr_p8; - TCGv_i64 x, y; + MemOpIdx oi = make_memop_idx(memop, idx); - canonicalize_memop_i128_as_i64(mop, memop); + tcg_debug_assert((memop & MO_SIZE) == MO_128); + tcg_debug_assert((memop & MO_SIGN) == 0); tcg_gen_req_mo(TCG_MO_ST_LD | TCG_MO_ST_ST); addr = plugin_prep_mem_callbacks(addr); - /* TODO: respect atomicity of the operation. */ /* TODO: allow the tcg backend to see the whole operation. */ - if ((memop & MO_BSWAP) == MO_LE) { - x = TCGV128_LOW(val); - y = TCGV128_HIGH(val); + if (use_two_i64_for_i128(memop)) { + MemOp mop[2]; + TCGv addr_p8; + TCGv_i64 x, y; + + canonicalize_memop_i128_as_i64(mop, memop); + + if ((memop & MO_BSWAP) == MO_LE) { + x = TCGV128_LOW(val); + y = TCGV128_HIGH(val); + } else { + x = TCGV128_HIGH(val); + y = TCGV128_LOW(val); + } + + addr_p8 = tcg_temp_ebb_new(); + if ((mop[0] ^ memop) & MO_BSWAP) { + TCGv_i64 t = tcg_temp_ebb_new_i64(); + + tcg_gen_bswap64_i64(t, x); + gen_ldst_i64(INDEX_op_qemu_st_i64, t, addr, mop[0], idx); + tcg_gen_bswap64_i64(t, y); + tcg_gen_addi_tl(addr_p8, addr, 8); + gen_ldst_i64(INDEX_op_qemu_st_i64, t, addr_p8, mop[1], idx); + tcg_temp_free_i64(t); + } else { + gen_ldst_i64(INDEX_op_qemu_st_i64, x, addr, mop[0], idx); + tcg_gen_addi_tl(addr_p8, addr, 8); + gen_ldst_i64(INDEX_op_qemu_st_i64, y, addr_p8, mop[1], idx); + } + tcg_temp_free(addr_p8); } else { - x = TCGV128_HIGH(val); - y = TCGV128_LOW(val); + gen_helper_st_i128(cpu_env, addr, val, tcg_constant_i32(oi)); } - addr_p8 = tcg_temp_new(); - if ((mop[0] ^ memop) & MO_BSWAP) { - TCGv_i64 t = tcg_temp_ebb_new_i64(); - - tcg_gen_bswap64_i64(t, x); - gen_ldst_i64(INDEX_op_qemu_st_i64, t, addr, mop[0], idx); - tcg_gen_bswap64_i64(t, y); - tcg_gen_addi_tl(addr_p8, addr, 8); - gen_ldst_i64(INDEX_op_qemu_st_i64, t, addr_p8, mop[1], idx); - tcg_temp_free_i64(t); - } else { - gen_ldst_i64(INDEX_op_qemu_st_i64, x, addr, mop[0], idx); - tcg_gen_addi_tl(addr_p8, addr, 8); - gen_ldst_i64(INDEX_op_qemu_st_i64, y, addr_p8, mop[1], idx); - } - tcg_temp_free(addr_p8); - - plugin_gen_mem_callbacks(addr, make_memop_idx(memop, idx), - QEMU_PLUGIN_MEM_W); + plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_W); } static void tcg_gen_ext_i32(TCGv_i32 ret, TCGv_i32 val, MemOp opc) diff --git a/accel/tcg/ldst_atomicity.c.inc b/accel/tcg/ldst_atomicity.c.inc index 1f39e43896..ce73b32def 100644 --- a/accel/tcg/ldst_atomicity.c.inc +++ b/accel/tcg/ldst_atomicity.c.inc @@ -439,6 +439,21 @@ static inline uint64_t load_atom_8_by_4(void *pv) } } +/** + * load_atom_8_by_8_or_4: + * @pv: host address + * + * Load 8 bytes from aligned @pv, with at least 4-byte atomicity. + */ +static inline uint64_t load_atom_8_by_8_or_4(void *pv) +{ + if (HAVE_al8_fast) { + return load_atomic8(pv); + } else { + return load_atom_8_by_4(pv); + } +} + /** * load_atom_2: * @p: host address @@ -571,6 +586,64 @@ static uint64_t load_atom_8(CPUArchState *env, uintptr_t ra, } } +/** + * load_atom_16: + * @p: host address + * @memop: the full memory op + * + * Load 16 bytes from @p, honoring the atomicity of @memop. + */ +static Int128 load_atom_16(CPUArchState *env, uintptr_t ra, + void *pv, MemOp memop) +{ + uintptr_t pi = (uintptr_t)pv; + int atmax; + Int128 r; + uint64_t a, b; + + /* + * If the host does not support 16-byte atomics, wait until we have + * examined the atomicity parameters below. + */ + if (HAVE_al16_fast && likely((pi & 15) == 0)) { + return load_atomic16(pv); + } + + atmax = required_atomicity(env, pi, memop); + switch (atmax) { + case MO_8: + memcpy(&r, pv, 16); + return r; + case MO_16: + a = load_atom_8_by_2(pv); + b = load_atom_8_by_2(pv + 8); + break; + case MO_32: + a = load_atom_8_by_4(pv); + b = load_atom_8_by_4(pv + 8); + break; + case MO_64: + if (!HAVE_al8) { + cpu_loop_exit_atomic(env_cpu(env), ra); + } + a = load_atomic8(pv); + b = load_atomic8(pv + 8); + break; + case -MO_64: + if (!HAVE_al8) { + cpu_loop_exit_atomic(env_cpu(env), ra); + } + a = load_atom_extract_al8x2(pv); + b = load_atom_extract_al8x2(pv + 8); + break; + case MO_128: + return load_atomic16_or_exit(env, ra, pv); + default: + g_assert_not_reached(); + } + return int128_make128(HOST_BIG_ENDIAN ? b : a, HOST_BIG_ENDIAN ? a : b); +} + /** * store_atomic2: * @pv: host address @@ -612,6 +685,35 @@ static inline void store_atomic8(void *pv, uint64_t val) qatomic_set__nocheck(p, val); } +/** + * store_atomic16: + * @pv: host address + * @val: value to store + * + * Atomically store 16 aligned bytes to @pv. + */ +static inline void store_atomic16(void *pv, Int128Alias val) +{ +#if defined(CONFIG_ATOMIC128) + __uint128_t *pu = __builtin_assume_aligned(pv, 16); + qatomic_set__nocheck(pu, val.u); +#elif defined(CONFIG_CMPXCHG128) + __uint128_t *pu = __builtin_assume_aligned(pv, 16); + __uint128_t o; + + /* + * Without CONFIG_ATOMIC128, __atomic_compare_exchange_n will always + * defer to libatomic, so we must use __sync_*_compare_and_swap_16 + * and accept the sequential consistency that comes with it. + */ + do { + o = *pu; + } while (!__sync_bool_compare_and_swap_16(pu, o, val.u)); +#else + qemu_build_not_reached(); +#endif +} + /** * store_atom_4x2 */ @@ -1055,3 +1157,85 @@ static void store_atom_8(CPUArchState *env, uintptr_t ra, } cpu_loop_exit_atomic(env_cpu(env), ra); } + +/** + * store_atom_16: + * @p: host address + * @val: the value to store + * @memop: the full memory op + * + * Store 16 bytes to @p, honoring the atomicity of @memop. + */ +static void store_atom_16(CPUArchState *env, uintptr_t ra, + void *pv, MemOp memop, Int128 val) +{ + uintptr_t pi = (uintptr_t)pv; + uint64_t a, b; + int atmax; + + if (HAVE_al16_fast && likely((pi & 15) == 0)) { + store_atomic16(pv, val); + return; + } + + atmax = required_atomicity(env, pi, memop); + + a = HOST_BIG_ENDIAN ? int128_gethi(val) : int128_getlo(val); + b = HOST_BIG_ENDIAN ? int128_getlo(val) : int128_gethi(val); + switch (atmax) { + case MO_8: + memcpy(pv, &val, 16); + return; + case MO_16: + store_atom_8_by_2(pv, a); + store_atom_8_by_2(pv + 8, b); + return; + case MO_32: + store_atom_8_by_4(pv, a); + store_atom_8_by_4(pv + 8, b); + return; + case MO_64: + if (HAVE_al8) { + store_atomic8(pv, a); + store_atomic8(pv + 8, b); + return; + } + break; + case -MO_64: + if (HAVE_al16) { + uint64_t val_le; + int s2 = pi & 15; + int s1 = 16 - s2; + + if (HOST_BIG_ENDIAN) { + val = bswap128(val); + } + switch (s2) { + case 1 ... 7: + val_le = store_whole_le16(pv, s1, val); + store_bytes_leN(pv + s1, s2, val_le); + break; + case 9 ... 15: + store_bytes_leN(pv, s1, int128_getlo(val)); + val = int128_urshift(val, s1 * 8); + store_whole_le16(pv + s1, s2, val); + break; + case 0: /* aligned */ + case 8: /* atmax MO_64 */ + default: + g_assert_not_reached(); + } + return; + } + break; + case MO_128: + if (HAVE_al16) { + store_atomic16(pv, val); + return; + } + break; + default: + g_assert_not_reached(); + } + cpu_loop_exit_atomic(env_cpu(env), ra); +} From patchwork Tue May 16 19:40:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682422 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp21074wrt; Tue, 16 May 2023 12:59:30 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6mwW29IkMRgPKWD+9Q010db0LIOA/jgRbmeqD24UYFO1ebx78DxlkgsYhvVp3934pZvh06 X-Received: by 2002:a1c:7407:0:b0:3f5:39:240c with SMTP id p7-20020a1c7407000000b003f50039240cmr8173263wmc.27.1684267170506; Tue, 16 May 2023 12:59:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267170; cv=none; d=google.com; s=arc-20160816; b=INmR16E3C6Ck0fagS6rE3UG0jJR7808SoYCAK0x1Y02Cwvfj7KNt6hLbb5URuxZf5u CMnpIoS97fU4qYssyic0B+DcoSTE+sNR05A7he4ARpdCizKqlXeMqSuPQgQZFmn54hYT sPrAI7CBUIT3MzfCe3cPUOQp9NZwBXo9zhOeOV7bix5WBZzP+4oITCFAnp8U1gET8UVQ o5ry3lhMjF/mqQNnk+Qt60COj2lMeC5NX40zHjIK/fp0b9FWiVJLGX8pbdLNxQvq2YIe FciwrTf5R39fmekmXXBiOoNgSgqXITcra1qj4hF2XOKZoXKVGaIv7s0wdvH27K7Tw1j3 HloQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=dl5gN58AGH2j6NZb2E6EW9hvdv0wzfEKuUUoszXGJnk=; b=zrdR2Jo09C5bmMfMoSazzwD0SmiKcolytVqKStOeZAFIYyqAAjgxdup5t8V7G0gUGT 0PV4H4CxJbVf5AqY6myh4AuMmEFZp3ui60P91iehb86sQ3gTYPYWTfqKeUAFv9GZWukV 8iVch7lKGtkP6Mtb7y536aT02GeEeNwCrffeFexC/7R/zyKRn9o48j+OcLblZ8PF+hEZ tzE1HOVBY6cjmW6lhZEjK+p2AfM8Tb76tpQKOf/q2MPMgkPHIlOo4WRzCioo6EQyuAkx sRR8JAVX+hYC0pFC2fZjdDlY2lQek8YHQZIOdpBE9jJTQqV5u/h+y1Q/PkWD8HKv4jWB iZhA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=nqAPnAAB; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d11-20020a05600c3acb00b003f41d2da92csi136998wms.56.2023.05.16.12.59.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:59:30 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=nqAPnAAB; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0YZ-0006cD-Ea; Tue, 16 May 2023 15:41:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0YY-0006bz-Ll for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:58 -0400 Received: from mail-pf1-x435.google.com ([2607:f8b0:4864:20::435]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0YW-0002l2-Nz for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:58 -0400 Received: by mail-pf1-x435.google.com with SMTP id d2e1a72fcca58-6439bbc93b6so10316258b3a.1 for ; Tue, 16 May 2023 12:41:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266115; x=1686858115; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dl5gN58AGH2j6NZb2E6EW9hvdv0wzfEKuUUoszXGJnk=; b=nqAPnAAB9EatO41ZwqwL5PCtp4JVCBOXAwaFLdsO0hwINzOtAoXfXH1wFJk+QihQ/r WpH79ygBDBGKvDgj9t7HRk9Cka0WOjeT082DWaDItvdwC0kPGJMQa97QgazP4YBuxUyA tstI0JwbW+A3a+9BselFzYfYwn2f6Dr2esYTLz0Kwrd+mzNnAkVS2l/jOrrpO0AHDurp larVHjFFduZYOBN+pdl7KbIqmGthiN4iKEsC+aAttGAAx45S67hkezc7UlEbspSLavJT z+9ZNkWeJylmuWiWSiNke41JxHhXT9QcAQ/XaGynqB8UgMX5WhzH7IPmvSIVD21LjrvE auFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266115; x=1686858115; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dl5gN58AGH2j6NZb2E6EW9hvdv0wzfEKuUUoszXGJnk=; b=B3ti985oQM7u+N6g5ZkIkiv0DimeJumo08JzLNp2a6iKRkIXciLgknLqU91H29iUTc uHDH4Eyr/xoLJRiYXTiAzaTOj9bIL8FQN1tjZV+QKAGhkuWaAnBEN2dYAVFhWTEPQM1Z pQ7RKtfbCSym5N3TD55YheYKcaXeQMVj94YQLBgRltvIFYGv3lBjqenQQjGf81aPY0/L xrzHBN4foOiRuvQRoqAVJrqLDLTZ9iLl2GeCMo9x96BxZAaXYar+L3OjEOTtijXICCaV WnhpVbev7VtmoDFzkHb7z/bb4H4uzs/S+d/rYoKiADD4T9rGdJs9AhDRh6MQCXzvg7pp XpLQ== X-Gm-Message-State: AC+VfDx75eo8HNP6ae+uagAhLJ25ccnnci3V5HCktmgkG8tcoCbcih23 CaKpewPxjT+4Mz/hgAp0h1f01RsOTB6squGco0w= X-Received: by 2002:a05:6a20:429a:b0:100:a785:4a86 with SMTP id o26-20020a056a20429a00b00100a7854a86mr37578140pzj.7.1684266115403; Tue, 16 May 2023 12:41:55 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.41.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:41:55 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 09/80] meson: Detect atomic128 support with optimization Date: Tue, 16 May 2023 12:40:34 -0700 Message-Id: <20230516194145.1749305-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::435; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x435.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org There is an edge condition prior to gcc13 for which optimization is required to generate 16-byte atomic sequences. Detect this. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- meson.build | 52 ++++++++++++++++++++++------------ accel/tcg/ldst_atomicity.c.inc | 29 ++++++++++++++++--- 2 files changed, 59 insertions(+), 22 deletions(-) diff --git a/meson.build b/meson.build index d3cf48960b..61de8199cb 100644 --- a/meson.build +++ b/meson.build @@ -2249,23 +2249,21 @@ config_host_data.set('HAVE_BROKEN_SIZE_MAX', not cc.compiles(''' return printf("%zu", SIZE_MAX); }''', args: ['-Werror'])) -atomic_test = ''' +# See if 64-bit atomic operations are supported. +# Note that without __atomic builtins, we can only +# assume atomic loads/stores max at pointer size. +config_host_data.set('CONFIG_ATOMIC64', cc.links(''' #include int main(void) { - @0@ x = 0, y = 0; + uint64_t x = 0, y = 0; y = __atomic_load_n(&x, __ATOMIC_RELAXED); __atomic_store_n(&x, y, __ATOMIC_RELAXED); __atomic_compare_exchange_n(&x, &y, x, 0, __ATOMIC_RELAXED, __ATOMIC_RELAXED); __atomic_exchange_n(&x, y, __ATOMIC_RELAXED); __atomic_fetch_add(&x, y, __ATOMIC_RELAXED); return 0; - }''' - -# See if 64-bit atomic operations are supported. -# Note that without __atomic builtins, we can only -# assume atomic loads/stores max at pointer size. -config_host_data.set('CONFIG_ATOMIC64', cc.links(atomic_test.format('uint64_t'))) + }''')) has_int128 = cc.links(''' __int128_t a; @@ -2283,21 +2281,39 @@ if has_int128 # "do we have 128-bit atomics which are handled inline and specifically not # via libatomic". The reason we can't use libatomic is documented in the # comment starting "GCC is a house divided" in include/qemu/atomic128.h. - has_atomic128 = cc.links(atomic_test.format('unsigned __int128')) + # We only care about these operations on 16-byte aligned pointers, so + # force 16-byte alignment of the pointer, which may be greater than + # __alignof(unsigned __int128) for the host. + atomic_test_128 = ''' + int main(int ac, char **av) { + unsigned __int128 *p = __builtin_assume_aligned(av[ac - 1], sizeof(16)); + p[1] = __atomic_load_n(&p[0], __ATOMIC_RELAXED); + __atomic_store_n(&p[2], p[3], __ATOMIC_RELAXED); + __atomic_compare_exchange_n(&p[4], &p[5], p[6], 0, __ATOMIC_RELAXED, __ATOMIC_RELAXED); + return 0; + }''' + has_atomic128 = cc.links(atomic_test_128) config_host_data.set('CONFIG_ATOMIC128', has_atomic128) if not has_atomic128 - has_cmpxchg128 = cc.links(''' - int main(void) - { - unsigned __int128 x = 0, y = 0; - __sync_val_compare_and_swap_16(&x, y, x); - return 0; - } - ''') + # Even with __builtin_assume_aligned, the above test may have failed + # without optimization enabled. Try again with optimizations locally + # enabled for the function. See + # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107389 + has_atomic128_opt = cc.links('__attribute__((optimize("O1")))' + atomic_test_128) + config_host_data.set('CONFIG_ATOMIC128_OPT', has_atomic128_opt) - config_host_data.set('CONFIG_CMPXCHG128', has_cmpxchg128) + if not has_atomic128_opt + config_host_data.set('CONFIG_CMPXCHG128', cc.links(''' + int main(void) + { + unsigned __int128 x = 0, y = 0; + __sync_val_compare_and_swap_16(&x, y, x); + return 0; + } + ''')) + endif endif endif diff --git a/accel/tcg/ldst_atomicity.c.inc b/accel/tcg/ldst_atomicity.c.inc index ce73b32def..ba5db7c366 100644 --- a/accel/tcg/ldst_atomicity.c.inc +++ b/accel/tcg/ldst_atomicity.c.inc @@ -16,6 +16,23 @@ #endif #define HAVE_al8_fast (ATOMIC_REG_SIZE >= 8) +/* + * If __alignof(unsigned __int128) < 16, GCC may refuse to inline atomics + * that are supported by the host, e.g. s390x. We can force the pointer to + * have our known alignment with __builtin_assume_aligned, however prior to + * GCC 13 that was only reliable with optimization enabled. See + * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107389 + */ +#if defined(CONFIG_ATOMIC128_OPT) +# if !defined(__OPTIMIZE__) +# define ATTRIBUTE_ATOMIC128_OPT __attribute__((optimize("O1"))) +# endif +# define CONFIG_ATOMIC128 +#endif +#ifndef ATTRIBUTE_ATOMIC128_OPT +# define ATTRIBUTE_ATOMIC128_OPT +#endif + #if defined(CONFIG_ATOMIC128) # define HAVE_al16_fast true #else @@ -152,7 +169,8 @@ static inline uint64_t load_atomic8(void *pv) * * Atomically load 16 aligned bytes from @pv. */ -static inline Int128 load_atomic16(void *pv) +static inline Int128 ATTRIBUTE_ATOMIC128_OPT +load_atomic16(void *pv) { #ifdef CONFIG_ATOMIC128 __uint128_t *p = __builtin_assume_aligned(pv, 16); @@ -356,7 +374,8 @@ static uint64_t load_atom_extract_al16_or_exit(CPUArchState *env, uintptr_t ra, * cross an 16-byte boundary then the access must be 16-byte atomic, * otherwise the access must be 8-byte atomic. */ -static inline uint64_t load_atom_extract_al16_or_al8(void *pv, int s) +static inline uint64_t ATTRIBUTE_ATOMIC128_OPT +load_atom_extract_al16_or_al8(void *pv, int s) { #if defined(CONFIG_ATOMIC128) uintptr_t pi = (uintptr_t)pv; @@ -692,7 +711,8 @@ static inline void store_atomic8(void *pv, uint64_t val) * * Atomically store 16 aligned bytes to @pv. */ -static inline void store_atomic16(void *pv, Int128Alias val) +static inline void ATTRIBUTE_ATOMIC128_OPT +store_atomic16(void *pv, Int128Alias val) { #if defined(CONFIG_ATOMIC128) __uint128_t *pu = __builtin_assume_aligned(pv, 16); @@ -790,7 +810,8 @@ static void store_atom_insert_al8(uint64_t *p, uint64_t val, uint64_t msk) * * Atomically store @val to @p masked by @msk. */ -static void store_atom_insert_al16(Int128 *ps, Int128Alias val, Int128Alias msk) +static void ATTRIBUTE_ATOMIC128_OPT +store_atom_insert_al16(Int128 *ps, Int128Alias val, Int128Alias msk) { #if defined(CONFIG_ATOMIC128) __uint128_t *pu, old, new; From patchwork Tue May 16 19:40:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682418 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp21048wrt; Tue, 16 May 2023 12:59:26 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5kbxfJ6n/24PaitmTMgXfBsZsfXWyE583zZ58I0gEfjmRkNU8rdesAnHu5m5N/EZMTr/m4 X-Received: by 2002:a7b:ca45:0:b0:3f4:2300:e00d with SMTP id m5-20020a7bca45000000b003f42300e00dmr26746208wml.3.1684267166569; Tue, 16 May 2023 12:59:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267166; cv=none; d=google.com; s=arc-20160816; b=Gciek3wbZgoTxRjiUAmTSqmmZ2lK4+NzQQMuYjHoWPPvqq4BULNxY7KeeSxWPyt3Gr C9MEyncQAiwBBcTfCmc4JhvkVvnqXL2ESlrPwWllbaHOSMedRjQHFrWhjeZtB1lImBI0 ILUqjFb5O/L/9I5sJpzVIUrWwOkkR8PDJM6gHBaL1gC4EUgrQGB/P3MDutzIcF1w6E+M nQweWmiUcv+RbBY4MHwea3FhGh+InTn0aRAIHlpIOMfSrCtBmprHJp7gUa1hxTcDsvCU z1iCqa0JJCovAy9TPYlO5MVUeeeRRLOQDHnYgbHdHfd+OEhXh6yDmbVe/5f/JfnPMEaG jRMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=dcVdAagIvm7EA0iM5EE3QZ/b7E6C622n8vck9DZLSDI=; b=nDmWkuJuHMK9E6EojrW/Wuso+I44p7vKkL7LQsAM+FBrmC1eFF58T/gqtsyEiqjhV4 djv9U7Xo6YTlb+wboDIVVQyTWL5gxhQqFLeBvmCDePg1d5uZ0R6AHbj1EdsGfIOnjA1O ZxaDo5S7RtH79tkwc5ZCY1oqSihhbwfRsGEENogkR10p0HKCjwn2TYHjHqTBgKyTzz9W 93CnoiV4hRbsFRMxNUvK/vytdJxyz54i0v9NyzLKtGlemO9m+vj7xGPZmcN9irUeNd7x nJOA4DTIA/MDo7qxJHTeKraZ7ZgfN6XKav2jHBqAm/nl9MBcD2re0/+o7CqZY0FuijZS e1Aw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=y+WXfnEW; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id c13-20020a7bc00d000000b003f42cc3261fsi110396wmb.119.2023.05.16.12.59.26 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:59:26 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=y+WXfnEW; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yc-0006ds-9h; Tue, 16 May 2023 15:42:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0YZ-0006cZ-ML for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:59 -0400 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0YX-0002lO-Nb for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:59 -0400 Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-643557840e4so15613710b3a.2 for ; Tue, 16 May 2023 12:41:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266116; x=1686858116; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dcVdAagIvm7EA0iM5EE3QZ/b7E6C622n8vck9DZLSDI=; b=y+WXfnEWugejtCia7Yy+U2z9Dr5N/Tp+HHtMTxgbY0GZA6aN/cGw8nvpsjIYFim+SS jwmg+HulbmO+2+mJZlcJaBrsfEjaC9ddH36s6YZKv52wd2y8BXibLe78y1bfO2pKxAWO VCHDUTwahcgeRsITC10KH7Jq3sWm7KdGhpwZQtLIwDl27hPcDfqzXunfQh3mPVB9HMFk GCOJuDIyJCel0WswpKRT7nwk+60gWLyhUkFGn+5PTg2azbxQCzvN5T+H6Tv/P3OFkn7d LQASSFxfaTgsCX9pyZwGPn+NpBSwzrM9gmrl1UHMb/7LdmYmvukB6bHWQjL/uknRqGMT 57+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266116; x=1686858116; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dcVdAagIvm7EA0iM5EE3QZ/b7E6C622n8vck9DZLSDI=; b=LNq7wOiXMCXK6VX5zAfDilgM2dOC0aX5a27nmK4SCInfir/33f3e5T7O+91HvnH2Jd wXpwxrukQPp6RZBgQLHrg7rMrKrZ0NU5+KFc9JE81vUJzgGEWj8SvnajCGcD1LAduJB5 8miR3TYK/5Q4Vol8/J1Hokd8IbENlI0opt+V1yM7OARjfJ8vChSnpVT0km8FSng6/UOy GB3psV/vlehIgKs3tx8ePZazxZ7UtLh7m6K9LHDOhq0opqIqmONC3IbF8nlxVQIeic22 jf3J938JgaOvdMEesDfAiL0rzALpwkE28zQD7jZwvIvU55sUhgMA+JQ8HgxPU2CjhwO1 LQCg== X-Gm-Message-State: AC+VfDyB4jQsfpzO9GkrpBNpA9ZWJwzuhi47ZYBXBbeJs5zgujcdp866 hBnpwOgCrWjT4UPi5zOF6cnik9fWNyvbTiCLJDI= X-Received: by 2002:a05:6a00:17a7:b0:645:454c:286d with SMTP id s39-20020a056a0017a700b00645454c286dmr45541830pfg.3.1684266116428; Tue, 16 May 2023 12:41:56 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.41.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:41:56 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 10/80] tcg/i386: Add have_atomic16 Date: Tue, 16 May 2023 12:40:35 -0700 Message-Id: <20230516194145.1749305-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42c; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Notice when Intel or AMD have guaranteed that vmovdqa is atomic. The new variable will also be used in generated code. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- include/qemu/cpuid.h | 18 ++++++++++++++++++ tcg/i386/tcg-target.h | 1 + tcg/i386/tcg-target.c.inc | 27 +++++++++++++++++++++++++++ 3 files changed, 46 insertions(+) diff --git a/include/qemu/cpuid.h b/include/qemu/cpuid.h index 1451e8ef2f..35325f1995 100644 --- a/include/qemu/cpuid.h +++ b/include/qemu/cpuid.h @@ -71,6 +71,24 @@ #define bit_LZCNT (1 << 5) #endif +/* + * Signatures for different CPU implementations as returned from Leaf 0. + */ + +#ifndef signature_INTEL_ecx +/* "Genu" "ineI" "ntel" */ +#define signature_INTEL_ebx 0x756e6547 +#define signature_INTEL_edx 0x49656e69 +#define signature_INTEL_ecx 0x6c65746e +#endif + +#ifndef signature_AMD_ecx +/* "Auth" "enti" "cAMD" */ +#define signature_AMD_ebx 0x68747541 +#define signature_AMD_edx 0x69746e65 +#define signature_AMD_ecx 0x444d4163 +#endif + static inline unsigned xgetbv_low(unsigned c) { unsigned a, d; diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index d4f2a6f8c2..0421776cb8 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -120,6 +120,7 @@ extern bool have_avx512dq; extern bool have_avx512vbmi2; extern bool have_avx512vl; extern bool have_movbe; +extern bool have_atomic16; /* optional instructions */ #define TCG_TARGET_HAS_div2_i32 1 diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index 826f7764c9..911123cfa8 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -185,6 +185,7 @@ bool have_avx512dq; bool have_avx512vbmi2; bool have_avx512vl; bool have_movbe; +bool have_atomic16; #ifdef CONFIG_CPUID_H static bool have_bmi2; @@ -4026,6 +4027,32 @@ static void tcg_target_init(TCGContext *s) have_avx512dq = (b7 & bit_AVX512DQ) != 0; have_avx512vbmi2 = (c7 & bit_AVX512VBMI2) != 0; } + + /* + * The Intel SDM has added: + * Processors that enumerate support for Intel® AVX + * (by setting the feature flag CPUID.01H:ECX.AVX[bit 28]) + * guarantee that the 16-byte memory operations performed + * by the following instructions will always be carried + * out atomically: + * - MOVAPD, MOVAPS, and MOVDQA. + * - VMOVAPD, VMOVAPS, and VMOVDQA when encoded with VEX.128. + * - VMOVAPD, VMOVAPS, VMOVDQA32, and VMOVDQA64 when encoded + * with EVEX.128 and k0 (masking disabled). + * Note that these instructions require the linear addresses + * of their memory operands to be 16-byte aligned. + * + * AMD has provided an even stronger guarantee that processors + * with AVX provide 16-byte atomicity for all cachable, + * naturally aligned single loads and stores, e.g. MOVDQU. + * + * See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104688 + */ + if (have_avx1) { + __cpuid(0, a, b, c, d); + have_atomic16 = (c == signature_INTEL_ecx || + c == signature_AMD_ecx); + } } } } From patchwork Tue May 16 19:40:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682426 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp21124wrt; Tue, 16 May 2023 12:59:42 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6BWQUv2Bm8hrLWqDmllpd2hy/yOQyDUrhW5h0kFoZRLI8AuCUK4o7tzRE41CMvIDmZWyq4 X-Received: by 2002:adf:ee0c:0:b0:2fd:c43a:b713 with SMTP id y12-20020adfee0c000000b002fdc43ab713mr28553466wrn.7.1684267182539; Tue, 16 May 2023 12:59:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267182; cv=none; d=google.com; s=arc-20160816; b=k3pMCPaaOoyCR/WD0JHJe4DJU6pnOrvo/gv2lEJoTLBZONRykcl0WzW2vt+fTz41kA F6V5JMMmT467yh1C0CEl1n/+2OM70cEaSefCGjCGJFqGDiiE87Dcn8JNee95/kcamGyY O7MQdAG55i/Hu59QTR7oErFBD6LH1JWFAwCswZCIxN55w1IIKxl45PtxlBOVgiCe11lA k8oQ3LxcDBZOFWVZbm/YJNNqS9oOsxFa2jsbUxcqcVqlXTq8SDf2UB20+Y5kbMLbc6/E BH0hFT4jGRe+TI0scOLc8bkmHB7XUtCLhb82UxXWA8nqs38Fp09qUiDWrOE5IDSo40qu o2oQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=WjHj7SEbww3DXTRNkJQIziiBJKKRW1RlTjQnuos+9yc=; b=URaPQJMZsvoEaIqKxd6y7lq5r76idkaPzMLINSYqfBI3g32siyuLD+LxrzMyCcp22H Cp2pK6k+UxRf682DGytE1nKLhM+1J/QHJv5jPlOR9sD7AWe/PHasrYDzergCCZpq3hLJ XOtkiG7bNAaMCSqsPO+2PgjtRIOmpNNjZsVvmTtemW6nHlX8lsr2lUHQRR9CxMez9xw4 N9L0+t66k3YELvkzXcQ3+78Xy0kPxevV2gnhpLySiXVuBpKf4Q8JigK3HuUVtF8A5nRQ odbUX3ZG6PwiEQDZg6wsau6XonPVQ059l6QXiNJhQrj3jwcVVllnzyCvndK1zHW1jLJD pWtg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KsiVpDJY; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z3-20020a5d44c3000000b00307acec2584si177709wrr.805.2023.05.16.12.59.42 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:59:42 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KsiVpDJY; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yc-0006e0-RP; Tue, 16 May 2023 15:42:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Ya-0006cz-1v for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:00 -0400 Received: from mail-pf1-x431.google.com ([2607:f8b0:4864:20::431]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0YY-0002jf-40 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:41:59 -0400 Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-643a1fed360so9087809b3a.3 for ; Tue, 16 May 2023 12:41:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266117; x=1686858117; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WjHj7SEbww3DXTRNkJQIziiBJKKRW1RlTjQnuos+9yc=; b=KsiVpDJYD8d/RF/XrK5LWLpzgDuo7ljc4I+V7F2FZvX8E55ib6eiM61CFVukCliHUK qvtOimiXk25FcsAtWR3f8t7B/+aJzObJzNieC/HmodK6vJ0Ze7dZql9uce9PGr0n+Wyi qbr1eURBiJyEBAGaFkRwOsm62VkREDSP/6i/nGvXNlZxdc01xl3VUvcx2DWIKYJ4gFm+ ulc3NRLrqP9j1KZeqJ7/Ew2rgrCSVTRK+eFj3dSioTBxenqv3Ael+UMVcNgB2ZrQmMmq HzSrAkXdXFllKmqABWniSuOfEluWPqguzlB2MhbE4qnYovgCfGtrKf36ojmyO50ueYyU QoFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266117; x=1686858117; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WjHj7SEbww3DXTRNkJQIziiBJKKRW1RlTjQnuos+9yc=; b=PhYHQeCMw9WCcYuHiQ+ZkaYVt5bt0CAybPpGzZ41lciLpKhzN8ReQM7F16M+RE4Fgy QuN78ZHB6Z8XHcpmycW6zznYUp5tbRXwuG4XgvtTH0KFr8ThLnzWalit/6SgROEj1UII 4N0izG0vybqPdh496Ey4hheF1KSV5eVvwN/OtCSwUwyjpBoqk94NvYTTe4QkdYFB+KbH HdR8aPF0PpRajD/PGvzwBRQHiVWT33dZo4ozE7yKaFM3z8fmY+6uLPICx2p/IeOjSqQ4 jcsOiyzZ0O9HsyfwJODuRoiGawocgisheXMaM+vh/YNSWTGpScdVwluN8h2+mYnWPrVk xeIA== X-Gm-Message-State: AC+VfDy2My1aaC0RU602QN/d8yHRMwzyxJYiPso6eexoPXnMNz9bcZ6H 7yEqFhTH1uxjeQW7VxsO1bv5sLN5LvGu4Y4WjSs= X-Received: by 2002:a05:6a00:1390:b0:646:3162:31a2 with SMTP id t16-20020a056a00139000b00646316231a2mr41795260pfg.17.1684266117268; Tue, 16 May 2023 12:41:57 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.41.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:41:56 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell , =?utf-8?q?Philippe_Mathieu-Dau?= =?utf-8?q?d=C3=A9?= Subject: [PULL 11/80] tcg/aarch64: Detect have_lse, have_lse2 for linux Date: Tue, 16 May 2023 12:40:36 -0700 Message-Id: <20230516194145.1749305-12-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::431; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x431.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Notice when the host has additional atomic instructions. The new variables will also be used in generated code. Reviewed-by: Peter Maydell Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.h | 3 +++ tcg/aarch64/tcg-target.c.inc | 12 ++++++++++++ 2 files changed, 15 insertions(+) diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index c0b0f614ba..3c0b0d312d 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -57,6 +57,9 @@ typedef enum { #define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_EVEN #define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_NORMAL +extern bool have_lse; +extern bool have_lse2; + /* optional instructions */ #define TCG_TARGET_HAS_div_i32 1 #define TCG_TARGET_HAS_rem_i32 1 diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index e6636c1f8b..fc551a3d10 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -13,6 +13,9 @@ #include "../tcg-ldst.c.inc" #include "../tcg-pool.c.inc" #include "qemu/bitops.h" +#ifdef __linux__ +#include +#endif /* We're going to re-use TCGType in setting of the SF bit, which controls the size of the operation performed. If we know the values match, it @@ -71,6 +74,9 @@ static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) return TCG_REG_X0 + slot; } +bool have_lse; +bool have_lse2; + #define TCG_REG_TMP TCG_REG_X30 #define TCG_VEC_TMP TCG_REG_V31 @@ -2899,6 +2905,12 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) static void tcg_target_init(TCGContext *s) { +#ifdef __linux__ + unsigned long hwcap = qemu_getauxval(AT_HWCAP); + have_lse = hwcap & HWCAP_ATOMICS; + have_lse2 = hwcap & HWCAP_USCAT; +#endif + tcg_target_available_regs[TCG_TYPE_I32] = 0xffffffffu; tcg_target_available_regs[TCG_TYPE_I64] = 0xffffffffu; tcg_target_available_regs[TCG_TYPE_V64] = 0xffffffff00000000ull; From patchwork Tue May 16 19:40:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682428 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp21144wrt; Tue, 16 May 2023 12:59:45 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6EYrs+Z2B4XO17fGFux7Gldr/bI1xezuupRtg8a+ABMCsSmqpiTWSh2cBtoRkuMQGcf2Ry X-Received: by 2002:a7b:c454:0:b0:3f4:2297:630a with SMTP id l20-20020a7bc454000000b003f42297630amr22878945wmi.20.1684267185613; Tue, 16 May 2023 12:59:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267185; cv=none; d=google.com; s=arc-20160816; b=zCtSWnwAIuI/Iuy5kPBvO7X0nZ9PO256o1EWwSr2HbKtVzhr/q67VQNdpVc1nPhY98 /VdgGa4LyLt9tWe15uY/ofTH/S5ilVx7tC1EJXPNl7z2XNEHqnyQcaQFrppBuU4UYQs9 lY5wi9SLzi7MnFEWl6btQHQWBkl4o62POoQwUk2IMzRvd5waXKSZlB/8cdU9xOODWk+t CbZSbC5WyA77/eSq6af29aQtcVEmN+5HZ9ab49ngM/9kcaM/hRLc+f3EwzobmoInuOHw 3pRrX4+fgDW2DRlkyO3iiascOlyQA9HLGQ1/z4FglEhaWz4HOXhbAR6uzADwlPL1DB5J ko+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=nP6uwN475NI4kEPgZuacjyw/HnU7r6xvn2JKlD8Wx/Y=; b=BfHvASdY5oGb89QIjjfSzl2nNEHbXSgIQsOWrm9H77byuLhCISBcwcyK0yZKaX4Kr6 XWgR/vFQrT8DmW74C4JY8WcYHZ2sr6IUTau0pI6xHMPOA8JDdVGTj1Ho6pP0JeSe0VBQ sDc4cXBURCvP+bRPjPNrB6fDrPwbtjh9rlba/4NY0CQrS/LjRvj62T41YaAwG92u8kHf fxHku+Yx3J1GOpX0fbpNWSHgLga6dKC82OmleINj5DuB/8vohj3EEAWysgZTf2pAvoS5 xXi6X/2vHhWQTlEU0+RMfwWe0awhzLi4xDyvrv89DzhrvVCz8+hOYF/3hKWTr1Y+VT6F I7Pw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fwWcOFw0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a22-20020a05600c349600b003f419da97dbsi161006wmq.30.2023.05.16.12.59.45 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:59:45 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fwWcOFw0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yc-0006e4-Up; Tue, 16 May 2023 15:42:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yb-0006di-BZ for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:01 -0400 Received: from mail-pg1-x535.google.com ([2607:f8b0:4864:20::535]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0YZ-0002lw-Ke for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:01 -0400 Received: by mail-pg1-x535.google.com with SMTP id 41be03b00d2f7-52caed90d17so9528070a12.0 for ; Tue, 16 May 2023 12:41:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266118; x=1686858118; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nP6uwN475NI4kEPgZuacjyw/HnU7r6xvn2JKlD8Wx/Y=; b=fwWcOFw0g3RNCfIsNcyJc89wsK0h5ZExv7lKDKVHJ4oy5iOuGeT4aUJq0rOw3IRM2C 1mLpyCICrHPRWEf5w0n3RDOL8uDG6kxroY7sVGSTZE+GTiX9dQubAwDRc4xBdB5muy2z XXrCSuR+fEDc+UuXJaeNYGwHLdaUuo1UL5YBKDJ6w45EwRDzFDVIiaiVUgcWaQqGI5Vr YLF6sTTd5uq3SRPDI+xdc7BK4S8uykD3jJ1sqpNf4Kgi/O5xMEoQdHw2v/H6yR4aDYZH Ij9NTVPjr9Qwb6Wox7PDLry4kwacBNERNyHK9R53lgCiVcehCqqTo0ll5d06eesHKAWB Syzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266118; x=1686858118; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nP6uwN475NI4kEPgZuacjyw/HnU7r6xvn2JKlD8Wx/Y=; b=J6By+6AdDTDeAKRImkS98ZN85yEG60dERKojMIaHTHQ4bwKD6FpOSSsgz/9lSTj63t quLK2ZyDeoS11lmLDZYLoHGy0SQYAbtj29PDLig5bSAHq8u/+CVzZcxTFiEekzlNqYYK pX9Dy8oiGkXUpSwKPh8Kv9ZO6Re58kRIzFJJZhyUjIETJdPNc1VMWEpwmTqf7QrswrJg 7FYdvlzT2lr3abLvmHyu9YtOFogI2IdnRZsHAo4T/Od0cyHEWdaJknqMUPHbCjU3ugao H35E7+SWK9wptMW2yTviZrntmA7eK290YifADvQp+Cuw0ggbAP3luZyML8iu5F5CLlwB xh7g== X-Gm-Message-State: AC+VfDzTuHUbOlYdpErf6lPfDzEARY/Sp6trMpwwVktzfBMR1Pdjr/8I d0G9l/klxsI6d6jWCH0sfEGH6zC1GRA/WyoyRHc= X-Received: by 2002:a05:6a20:8e06:b0:101:ab2:eaa8 with SMTP id y6-20020a056a208e0600b001010ab2eaa8mr37857010pzj.18.1684266118248; Tue, 16 May 2023 12:41:58 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.41.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:41:57 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Peter Maydell Subject: [PULL 12/80] tcg/aarch64: Detect have_lse, have_lse2 for darwin Date: Tue, 16 May 2023 12:40:37 -0700 Message-Id: <20230516194145.1749305-13-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::535; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x535.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org These features are present for Apple M1. Tested-by: Philippe Mathieu-Daudé Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.c.inc | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index fc551a3d10..c64606af5b 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -16,6 +16,9 @@ #ifdef __linux__ #include #endif +#ifdef CONFIG_DARWIN +#include +#endif /* We're going to re-use TCGType in setting of the SF bit, which controls the size of the operation performed. If we know the values match, it @@ -2903,6 +2906,27 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) } } +#ifdef CONFIG_DARWIN +static bool sysctl_for_bool(const char *name) +{ + int val = 0; + size_t len = sizeof(val); + + if (sysctlbyname(name, &val, &len, NULL, 0) == 0) { + return val != 0; + } + + /* + * We might in the future ask for properties not present in older kernels, + * but we're only asking about static properties, all of which should be + * 'int'. So we shouln't see ENOMEM (val too small), or any of the other + * more exotic errors. + */ + assert(errno == ENOENT); + return false; +} +#endif + static void tcg_target_init(TCGContext *s) { #ifdef __linux__ @@ -2910,6 +2934,10 @@ static void tcg_target_init(TCGContext *s) have_lse = hwcap & HWCAP_ATOMICS; have_lse2 = hwcap & HWCAP_USCAT; #endif +#ifdef CONFIG_DARWIN + have_lse = sysctl_for_bool("hw.optional.arm.FEAT_LSE"); + have_lse2 = sysctl_for_bool("hw.optional.arm.FEAT_LSE2"); +#endif tcg_target_available_regs[TCG_TYPE_I32] = 0xffffffffu; tcg_target_available_regs[TCG_TYPE_I64] = 0xffffffffu; From patchwork Tue May 16 19:40:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682385 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp18025wrt; Tue, 16 May 2023 12:48:53 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7pNvpUTYRuOWDbDUCni3amMV1RizQ9OfFuC8hoYq3MGQGfl+uHJVlDkYC1lxEaxkILb7pI X-Received: by 2002:ac8:5b92:0:b0:3f3:9b12:3a80 with SMTP id a18-20020ac85b92000000b003f39b123a80mr46309037qta.54.1684266533207; Tue, 16 May 2023 12:48:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266533; cv=none; d=google.com; s=arc-20160816; b=hAk6LLzEbP2HvxH/n9Isi3jMbJQdp8aUwY7qCH5ffKmMcm4NV/FO7ojagduvcz7vk1 l5Mie4q/RfVaSMZwX29IoRiB9ZiT+dSbcQAUMmcnJF3m9Q8odz7gH0G2mBnZi1lwbpsy zQdTLPfZFk08Kq6s6iFYPW0pxtgXPwewSd6O/07KfGO+vpQ1Rxq2VMq5Wv+IE+XKjcby Ftp+j998FfxjjkXTpSM4xsDXWzN9NKihzmc9TlSu/mRebBcj01chsxFFggNRauDI9I3D yRSrGTb+tk+12jEsVOHYakK469B5NPMbKwIUpSBNI/yJKoQ+7nEwD7HQ2IkqrrIDXj2u 0tZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=0BFtx9KYq0Th2HWZb/BY+/P+rVcQ7G1nOi7Bio8e4oI=; b=IdGYvffVDD+YAwG7qNI2F/4EZctQHR6aZhjUvCo5MwQjQqK9/XVVM6JnTfXw9PZ0U1 J9JTwjZJ1tDJiWpoI2cM+0j3yixaTnZ5Ut+XgVwAZVGY0+n3PHMQkO9Zz/eEcS20NXQP Ai6H1ueoNuvOAQ8At/SmmZ7EQhUQjRnnbACNIB3rM3VMaDjSMbuuAbKNTf6MvKU0M/PS FHHxUOuijPm9YH390vRtm6S+N0ojKYGU3H18elDNFFIe3jswtrTeq3JCRZlyDdw1yK9j ZSsSgBHoA8qhrTZsOrkaDW5f1aFuv38+oZzaELUZd2ckSVz8cqIfQDEog0nlBHDa16Tc Yc0A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EslWbfjB; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j17-20020ae9c211000000b00759383cdccasi213563qkg.425.2023.05.16.12.48.53 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:48:53 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EslWbfjB; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yd-0006eW-HC; Tue, 16 May 2023 15:42:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yc-0006dr-8I for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:02 -0400 Received: from mail-pg1-x536.google.com ([2607:f8b0:4864:20::536]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Ya-0002na-Id for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:01 -0400 Received: by mail-pg1-x536.google.com with SMTP id 41be03b00d2f7-5304913530fso8068792a12.0 for ; Tue, 16 May 2023 12:42:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266119; x=1686858119; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0BFtx9KYq0Th2HWZb/BY+/P+rVcQ7G1nOi7Bio8e4oI=; b=EslWbfjB90BUadvpR2qxu4N6UhJY1ngA1MjB399KaCev/ocB8t23VOXlOwUWWc+rPc y4zWiBARwVSWSAFP2wT5Fk+gIoTIkkkxCZT5srFu8cyNS1j0O9hTAooz/TLuqFg3Yixw WAa4RMGwbVrFK/vcVqd1rzG5uTUfQOlORhiguOErt2lCYsjRHDoFCuncr6+I+26bxWmb yRRvE2BYIAFYDNNWW1p2CUUtvWGvU55tppSFT4HrwUBAMw7vixkn6obuHrPjOBFRZrzp v73rjlTOvfTKgZbyUH4pJcUwr0Angjmh/EVW7SN2pZcFcCpfGNxMGezXcbPA9RIh1kPf RsLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266119; x=1686858119; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0BFtx9KYq0Th2HWZb/BY+/P+rVcQ7G1nOi7Bio8e4oI=; b=OdsuqniOEynJj+a3k0FWg4he/GVTHCg9QJD5H1ndjW/+qwnFpKjR+snjJP/1rhLFe0 +kzbhePM41UZTOKL39Lz+K4ux07Cwgk+3ZzSIFeDL7mwCb/8/v8tysBOtpItrTWY2xKL NtS6jKmiG1P0W7HDq0eOhAZAQWYgipLNjjF75iDLyrfW+X0jg2UnlMZyUU+bnWtrS9yh cmxYH0f8SYtRKTLDoYIcce4qECYX8g64orYWG3Z3Rm/BcLzbqTDm0EZTdJaDIc0P0Ip3 iZXorlJ4SCJmJqDcxLXcKm1spiReikWe8QuDXEaOwclMseRxifqSHl2KSstOM0tespKF dVYw== X-Gm-Message-State: AC+VfDxuzp12oKJ4G2gy/H4dGl820C33ksMHFQ+4mEWrNEKHQLC1jBoQ L6W9fcGAw3sjOE/cEuum5hsDGRliIAvg2lHHeKA= X-Received: by 2002:a05:6a20:394d:b0:104:8824:74eb with SMTP id r13-20020a056a20394d00b00104882474ebmr19735544pzg.51.1684266119092; Tue, 16 May 2023 12:41:59 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.41.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:41:58 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 13/80] tcg/i386: Use full load/store helpers in user-only mode Date: Tue, 16 May 2023 12:40:38 -0700 Message-Id: <20230516194145.1749305-14-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::536; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x536.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Instead of using helper_unaligned_{ld,st}, use the full load/store helpers. This will allow the fast path to increase alignment to implement atomicity while not immediately raising an alignment exception. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.c.inc | 52 +++------------------------------------ 1 file changed, 4 insertions(+), 48 deletions(-) diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index 911123cfa8..21553f3c39 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -1776,7 +1776,6 @@ typedef struct { int seg; } HostAddress; -#if defined(CONFIG_SOFTMMU) /* * Because i686 has no register parameters and because x86_64 has xchg * to handle addr/data register overlap, we have placed all input arguments @@ -1812,7 +1811,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) /* resolve label address */ tcg_patch32(label_ptr[0], s->code_ptr - label_ptr[0] - 4); - if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { + if (label_ptr[1]) { tcg_patch32(label_ptr[1], s->code_ptr - label_ptr[1] - 4); } @@ -1834,7 +1833,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) /* resolve label address */ tcg_patch32(label_ptr[0], s->code_ptr - label_ptr[0] - 4); - if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { + if (label_ptr[1]) { tcg_patch32(label_ptr[1], s->code_ptr - label_ptr[1] - 4); } @@ -1844,51 +1843,8 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) tcg_out_jmp(s, l->raddr); return true; } -#else -static bool tcg_out_fail_alignment(TCGContext *s, TCGLabelQemuLdst *l) -{ - /* resolve label address */ - tcg_patch32(l->label_ptr[0], s->code_ptr - l->label_ptr[0] - 4); - - if (TCG_TARGET_REG_BITS == 32) { - int ofs = 0; - - tcg_out_st(s, TCG_TYPE_PTR, TCG_AREG0, TCG_REG_ESP, ofs); - ofs += 4; - - tcg_out_st(s, TCG_TYPE_I32, l->addrlo_reg, TCG_REG_ESP, ofs); - ofs += 4; - if (TARGET_LONG_BITS == 64) { - tcg_out_st(s, TCG_TYPE_I32, l->addrhi_reg, TCG_REG_ESP, ofs); - ofs += 4; - } - - tcg_out_pushi(s, (uintptr_t)l->raddr); - } else { - tcg_out_mov(s, TCG_TYPE_TL, tcg_target_call_iarg_regs[1], - l->addrlo_reg); - tcg_out_mov(s, TCG_TYPE_PTR, tcg_target_call_iarg_regs[0], TCG_AREG0); - - tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_RAX, (uintptr_t)l->raddr); - tcg_out_push(s, TCG_REG_RAX); - } - - /* "Tail call" to the helper, with the return address back inline. */ - tcg_out_jmp(s, (const void *)(l->is_ld ? helper_unaligned_ld - : helper_unaligned_st)); - return true; -} - -static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) -{ - return tcg_out_fail_alignment(s, l); -} - -static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) -{ - return tcg_out_fail_alignment(s, l); -} +#ifndef CONFIG_SOFTMMU static HostAddress x86_guest_base = { .index = -1 }; @@ -1920,7 +1876,7 @@ static inline int setup_guest_base_seg(void) return 0; } #endif /* setup_guest_base_seg */ -#endif /* SOFTMMU */ +#endif /* !SOFTMMU */ /* * For softmmu, perform the TLB load and compare. From patchwork Tue May 16 19:40:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682373 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp640220ltp; Tue, 16 May 2023 12:46:32 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5VPxXNBL85F02wJQNmvMgN9bvlCKoV3ZZL3SPTK7WKhUqDH6vI5yuJy/qndIFPowaaekBt X-Received: by 2002:a05:622a:594:b0:3f5:6af9:d535 with SMTP id c20-20020a05622a059400b003f56af9d535mr1221654qtb.43.1684266392785; Tue, 16 May 2023 12:46:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266392; cv=none; d=google.com; s=arc-20160816; b=t7qDxh1Ykhkfb5QIFOI0fs8wlmN4zIZ7hZoxIGAJGqC9JwO6Sv3GOjb6+VGxCbkIU8 BdEyTfqirlc5fx78s2JZTJMyMxwHViH+XvnWO/QK17ShZDxG/TFXCJDTXydXjRqFnMga LkOHEANl5WkV33fxCDf2/K/KqCARAMpoF7OnC2UwGwZrsuHY3k3wShtkyEJ02AWqVOT1 8yHc9N+Uj1+02QbW+TgpQkTOGdT10lO4HZmyMGv1/wwQteaxwGBNaRSyh1/kcSveVjVj HvYSfjqpcUM1yBuCV4m7AFqMHYhUYgRkg+o3BAgkOCKD+GdsFa+kkJ/RK4ubSbf0zUe0 GsfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=jxeAK+0nBaqYYTLaoVSvaQrIxJ38Tit6zinqkGACFIA=; b=w9NhgFMRGQ1NOJvOhUU4ZOtFMpl1yIIbffxFUiDQLFjgOddWCEe2T5FpLtWkNQa38e jFs6ueR7TlXXiIotgluxHfr4z4kI7rowAuR2FJzG7HdhNS+hZfJcbnOVMmRAtPe1i7uS FcMIkSkJqc+sTe0TCkKwHaNN62p0js/93TvrYcvhl90G63v7wNgt+gSDhf6YsxJ0tG7Q 1hhukp8OL8W9WDhkXQTuO4rfnURT2GCOjvD0r1417YOIh27Spl0GlT3hCdSGSfOoQsCJ ooGjMHOkL2qqlRWRRaYrIiNMFbq0hFhpWwZ4dLMJcPBf0WyoqaFF2csI7Dr8TqCyoXl+ bKYQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NFm9K4dg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id u14-20020a05622a14ce00b003f0b0263994si11635444qtx.743.2023.05.16.12.46.32 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:46:32 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NFm9K4dg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yf-0006fv-Cp; Tue, 16 May 2023 15:42:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yd-0006em-TD for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:03 -0400 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yc-0002o7-9Q for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:03 -0400 Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-64384c6797eso11570414b3a.2 for ; Tue, 16 May 2023 12:42:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266120; x=1686858120; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jxeAK+0nBaqYYTLaoVSvaQrIxJ38Tit6zinqkGACFIA=; b=NFm9K4dgx5Kc5XkcZQuKi1605ciD7QY8rkhN9cCutaGeWpq5uotC5Xh1VNTNwGwL5S XlPJjROa6UV1Uzpwm+ouJ7RYNEfvie9mkyTgetgHIdtEJns1Oa+fspOdjo6lGR8vl53y mPm1iLDwNC8uKn4L8opL1G86lDwsiEyERG4JnMUDcnsznF7D/gQmvrOiDPDQFAhYAoeg Sl9UkAhPCULWI7Gp0P94w7OG0IgtA5+H+sUIUU/Pw/xB4zd7+xZfkO5dm9Keiz3O6H68 CkaNkHZW/u/FAktX0QpdnufmOSf8OthnfrGfS6WH2efC5Br2K9WxWSXMD4cCdq+TTYhh YEaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266120; x=1686858120; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jxeAK+0nBaqYYTLaoVSvaQrIxJ38Tit6zinqkGACFIA=; b=JOE3qdu9d9ft5Dcu5t8MbW+qVX8GzBLK6kXo0J5tl8fgcSRzCjuSycoO/qqlNz5bC9 crcqengzmT9MWIlhx6r6qGAfGVJN/334mMDN4Ap4msj1QxiQxqFPwZrjWzsxcydJ/ni8 6H28n+o4TW8UdV2FHrwicx6WUqmVXr8YPJhO3egVV+F7lmxbaW6+wfEbvXnu7HmyFB0/ FfVWkfhigB1jOWGVNF1hsJpHf1zugZkexAWmqju3vC/w3MrCd4/gIuB20vkWIJNfppQY Y/vmgzLI/01nb2wZzs/mogckIP112mSwS8818SWrDvqQDK9FeZ5bj8JjXHj7s7h3UpIg y4iw== X-Gm-Message-State: AC+VfDwYxF7swwygZ5RZwUjuptN16skfgu9A1+7LVPTB0RmjpZ+RY3aX rxI9cMWF/IAkThlJkSrMPJ/5osZwTqJ4lK6wHj8= X-Received: by 2002:a05:6a00:2194:b0:643:b653:3aa with SMTP id h20-20020a056a00219400b00643b65303aamr46908229pfi.32.1684266119894; Tue, 16 May 2023 12:41:59 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.41.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:41:59 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 14/80] tcg/aarch64: Use full load/store helpers in user-only mode Date: Tue, 16 May 2023 12:40:39 -0700 Message-Id: <20230516194145.1749305-15-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42c; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Instead of using helper_unaligned_{ld,st}, use the full load/store helpers. This will allow the fast path to increase alignment to implement atomicity while not immediately raising an alignment exception. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.c.inc | 35 ----------------------------------- 1 file changed, 35 deletions(-) diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index c64606af5b..36d8798bca 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -1595,7 +1595,6 @@ typedef struct { TCGType index_ext; } HostAddress; -#ifdef CONFIG_SOFTMMU static const TCGLdstHelperParam ldst_helper_param = { .ntmp = 1, .tmp = { TCG_REG_TMP } }; @@ -1628,40 +1627,6 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) tcg_out_goto(s, lb->raddr); return true; } -#else -static void tcg_out_adr(TCGContext *s, TCGReg rd, const void *target) -{ - ptrdiff_t offset = tcg_pcrel_diff(s, target); - tcg_debug_assert(offset == sextract64(offset, 0, 21)); - tcg_out_insn(s, 3406, ADR, rd, offset); -} - -static bool tcg_out_fail_alignment(TCGContext *s, TCGLabelQemuLdst *l) -{ - if (!reloc_pc19(l->label_ptr[0], tcg_splitwx_to_rx(s->code_ptr))) { - return false; - } - - tcg_out_mov(s, TCG_TYPE_TL, TCG_REG_X1, l->addrlo_reg); - tcg_out_mov(s, TCG_TYPE_PTR, TCG_REG_X0, TCG_AREG0); - - /* "Tail call" to the helper, with the return address back inline. */ - tcg_out_adr(s, TCG_REG_LR, l->raddr); - tcg_out_goto_long(s, (const void *)(l->is_ld ? helper_unaligned_ld - : helper_unaligned_st)); - return true; -} - -static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) -{ - return tcg_out_fail_alignment(s, l); -} - -static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) -{ - return tcg_out_fail_alignment(s, l); -} -#endif /* CONFIG_SOFTMMU */ /* * For softmmu, perform the TLB load and compare. From patchwork Tue May 16 19:40:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682383 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp18013wrt; Tue, 16 May 2023 12:48:52 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5fIQDNTuFgOf023gcxLM6qPpxLT+c5dhSp4TQm+1kE7sa0GF1spvrNYtXqlhDrOSkqLVDy X-Received: by 2002:ac8:5a85:0:b0:3f0:ab4e:df6b with SMTP id c5-20020ac85a85000000b003f0ab4edf6bmr56095471qtc.67.1684266532302; Tue, 16 May 2023 12:48:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266532; cv=none; d=google.com; s=arc-20160816; b=iqxKzHTEbx0Ty/BfjYYlNveXvDrZ9yzakW6mOf9c4X9uqfdOf+fHb1JJ3Z17DMUGwn W76+3x+QFEtVZV7CPt1zd3BpIQ163HpJ6/ijdVlZmB1j8euytg+CC7e7ZAPueO7buTPC mIuun7VIOiEB1xhyWmh8puY2F2tq4LEXgk6Aj/6l3uabjmPQ1hU1n9XKrIMjzWAIHLGQ 5njuQtadyoUAd923eJBn+QqgDkw9wKHuqYGWqlBo5WpF2dW5nfuwS41v9pZHgXTJnPbW t4pwDehS4a+YlDqOSGwQdUXUEEzqMXceSPmGX+LZdL11n95UoDZIGa9VWRkbXf3N4fsc SA+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=EAy5XbfXYPrm/J1tRB5DUCT9JP3KNW8vRC1FRgDgvtM=; b=w9ovZzNgzLelMwBArc7kiWu8DusTBI63xPVGJdAJSm/WZCBMTmB/WSsjmzdXCUxppt m0uklv7LIliHsaTKVV8s8sfDtnCMSM6UTL1B6a5IP4IAbkdlOyei3btOhahqJaHFmmH3 X88istfQXyHWARaPEw8hSqAiuCrkn/953wYwsUAXN+JBgwyGHvHzVI66m1IjFMeXtRpJ p4ypbf8dpeEZf9p5crACwRntR537lKH4Zsf/8JCDHfIs2gZk9RQNBnlB1VLfJvZMNHiU UicfgvxmktdF5FnxK7AYWSfUQPGWtDeRCoRwkOvfbPsJWZ17zpKSM57ojrNCjerKhbGz EWbA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=l9G11aPY; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d18-20020ac85d92000000b003f4f44d9676si8142768qtx.615.2023.05.16.12.48.52 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:48:52 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=l9G11aPY; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yg-0006gG-FS; Tue, 16 May 2023 15:42:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yd-0006eY-Lx for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:03 -0400 Received: from mail-pf1-x42e.google.com ([2607:f8b0:4864:20::42e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yc-0002oP-2K for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:03 -0400 Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-643b60855c8so12646824b3a.2 for ; Tue, 16 May 2023 12:42:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266121; x=1686858121; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EAy5XbfXYPrm/J1tRB5DUCT9JP3KNW8vRC1FRgDgvtM=; b=l9G11aPY7zvMYkC060Xfq9uCYZYAWU5DHnoZ6Ukg7WkDOeUdrWLGibcXcPZo53PeJI 8lgXx4wsfDmbn2DLeyuNN7h34ovZBJEgRHeUTELQx/hA9QqGBVr8AynYixl3nWtJK2Du gwfI8e7bKO0w/3UDGYNvIzdRdrxAu1m6Jgk3KAD7/R19QFZHHye2ARpHNo/VGgg9eXn6 ohLNGPHHBfBS7a64CIn9a4TM7VWrDNFXhS+mtl/AfAFQ55sLRWdeoidkasGDggcke6df zUV0seCX4bz1FQB22Nl0vRQ5HnnUofL8JqE6ZXkKqT1TXguDtUhTIssX5Uc2zGElTnp/ KcuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266121; x=1686858121; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EAy5XbfXYPrm/J1tRB5DUCT9JP3KNW8vRC1FRgDgvtM=; b=hsCfaIcHXaGdmwbD2hNH7Xp6IV7AYyFS0GGTxE6UmHx99+dqVD9TdmmGYAEVC2IS7R KmLaEuxgVPHRvvEjN9CPaZ1ldlI4piZ7h5hWn2zvF/e4h8ircU7CskTgbCxdMNMDrx8f rcamsoCVUQ7qZB7a64FO8Ux0Z75r3B7t7QbdBjm/EG8LX7i3jCUipSLSwscq6Aam1KUb hkt5hD2rPpKfAVbBIjCDLqEE5jtmmnZ4b77UmSZDBlFu+Scxbuzcc+xI57EQPW9ktqGV V99eqTvjvVlM0NT0XngSz3QFAVvAerDTL9cvXRA/zVNlMVmyb6SZDLx2+fdW62zyN3Jp 9Oxw== X-Gm-Message-State: AC+VfDyA7aElx6Ogzs1oLLftHDNzb9K3mUVgGPaAMbtcShzumNiXuuvm +/oWoymI2rtNQyG8ioNecHp9c3B+Bu8Y1YB46Qs= X-Received: by 2002:a05:6a00:b55:b0:648:c1be:496 with SMTP id p21-20020a056a000b5500b00648c1be0496mr32881741pfo.22.1684266120845; Tue, 16 May 2023 12:42:00 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:00 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 15/80] tcg/ppc: Use full load/store helpers in user-only mode Date: Tue, 16 May 2023 12:40:40 -0700 Message-Id: <20230516194145.1749305-16-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42e; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Instead of using helper_unaligned_{ld,st}, use the full load/store helpers. This will allow the fast path to increase alignment to implement atomicity while not immediately raising an alignment exception. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/ppc/tcg-target.c.inc | 44 ---------------------------------------- 1 file changed, 44 deletions(-) diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc index 6a81916e64..218602c10c 100644 --- a/tcg/ppc/tcg-target.c.inc +++ b/tcg/ppc/tcg-target.c.inc @@ -1962,7 +1962,6 @@ static const uint32_t qemu_stx_opc[(MO_SIZE + MO_BSWAP) + 1] = { [MO_BSWAP | MO_UQ] = STDBRX, }; -#if defined (CONFIG_SOFTMMU) static TCGReg ldst_ra_gen(TCGContext *s, const TCGLabelQemuLdst *l, int arg) { if (arg < 0) { @@ -2012,49 +2011,6 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) tcg_out_b(s, 0, lb->raddr); return true; } -#else -static bool tcg_out_fail_alignment(TCGContext *s, TCGLabelQemuLdst *l) -{ - if (!reloc_pc14(l->label_ptr[0], tcg_splitwx_to_rx(s->code_ptr))) { - return false; - } - - if (TCG_TARGET_REG_BITS < TARGET_LONG_BITS) { - TCGReg arg = TCG_REG_R4; - - arg |= (TCG_TARGET_CALL_ARG_I64 == TCG_CALL_ARG_EVEN); - if (l->addrlo_reg != arg) { - tcg_out_mov(s, TCG_TYPE_I32, arg, l->addrhi_reg); - tcg_out_mov(s, TCG_TYPE_I32, arg + 1, l->addrlo_reg); - } else if (l->addrhi_reg != arg + 1) { - tcg_out_mov(s, TCG_TYPE_I32, arg + 1, l->addrlo_reg); - tcg_out_mov(s, TCG_TYPE_I32, arg, l->addrhi_reg); - } else { - tcg_out_mov(s, TCG_TYPE_I32, TCG_REG_R0, arg); - tcg_out_mov(s, TCG_TYPE_I32, arg, arg + 1); - tcg_out_mov(s, TCG_TYPE_I32, arg + 1, TCG_REG_R0); - } - } else { - tcg_out_mov(s, TCG_TYPE_TL, TCG_REG_R4, l->addrlo_reg); - } - tcg_out_mov(s, TCG_TYPE_TL, TCG_REG_R3, TCG_AREG0); - - /* "Tail call" to the helper, with the return address back inline. */ - tcg_out_call_int(s, 0, (const void *)(l->is_ld ? helper_unaligned_ld - : helper_unaligned_st)); - return true; -} - -static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) -{ - return tcg_out_fail_alignment(s, l); -} - -static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) -{ - return tcg_out_fail_alignment(s, l); -} -#endif /* SOFTMMU */ typedef struct { TCGReg base; From patchwork Tue May 16 19:40:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682380 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp18000wrt; Tue, 16 May 2023 12:48:51 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ44/NQ9z/DMmtj2G4wJqL51F/eZGc2GgCVJAUOSLtxGaNWuou/KeOJyMKB8OnByTwIfiK6U X-Received: by 2002:a05:622a:14d0:b0:3f5:3ed1:79c7 with SMTP id u16-20020a05622a14d000b003f53ed179c7mr6036575qtx.38.1684266531007; Tue, 16 May 2023 12:48:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266530; cv=none; d=google.com; s=arc-20160816; b=aXmu+5pFu+w9WoBLN3lNe4VUHmp12xskvru7zMGRoAEdcXjNT7kNVZFdvsnetKqznF AFRdikDkYmySsUhsQWIruzpmertg3Z9OCMCdpvU4RMG9MlkwY2ectAfyekMLEIFLdx3C p33kbR7T8ZEFTjYzRXyfxPGhbvCYsskgVHt4yGGe5taWIH1fZoSDfga6Bclgu6GdszXk WserAkzBeHVYfYgZjyba1mGwyzYQEA9OmxDq/AkX7I0MZJLke8pjNy7JAj7Tc8mw64LS PdtgcWdTtaBlOgENvRdAZ52qtU72BV/P5jn+t7n9Xt5BPwREz5RUR4xQMBeHXqc8OrJ6 Av7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=c2X0vEktAU9CIzOCMmP9cn2Oif9GXs/gZcvny2bfDX8=; b=NOdvk629l35pRWOdtvpw07W41z0z8Hf6BV0uyTGzmSJZBjw4Ib7umUmI44PRqZA/s6 N2VGJ/6sZUeYVZEPR6NVpOs5XPT9uwfot4gxUT1rNxUoyv9a5gaxyrd/6G8fBI4SQtGt tLZ8Nvres5qkiVNa4MfGV/RCtZaL7exy+Dlbwwzs17FWpLBDg8bMsP78Kfo9ovxXnhPi knhmPE5bdTq0sNKYwp83hpWdN2Ijff9RckaoR9QsF1yldwFBov1IELZEiYLLuY6dB/FE t5UR32Jeg25e0lYX4KBguER11ICwjZmXRmjsICvTjwnsLaNeNh/0w7/woxRwN0rBrpg8 /0Dw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uvl0ZopT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id p19-20020a05620a15f300b00757799b6e23si213677qkm.405.2023.05.16.12.48.50 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:48:50 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uvl0ZopT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yh-0006gr-3Y; Tue, 16 May 2023 15:42:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Ye-0006f8-GO for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:04 -0400 Received: from mail-pf1-x433.google.com ([2607:f8b0:4864:20::433]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yc-0002og-Rw for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:04 -0400 Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-643aad3bc41so13989099b3a.0 for ; Tue, 16 May 2023 12:42:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266121; x=1686858121; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=c2X0vEktAU9CIzOCMmP9cn2Oif9GXs/gZcvny2bfDX8=; b=uvl0ZopTb8oIbfZ5g+2Qo3kJXWessNouX2EwNEfvea1iYehrUVBtUpim9RYYqYGEzW AT8229jFrj1s6YDlhNnmwi3G6/ChKAt8BJvsbYPAZed0/STx0vU21ZlvIpUhvsZydzS4 zumYBD31G7H3k8bS6ebFhNTF/if7K4ruxTt1jZXgkQonnwOBwaod7xEFTwfCD54kyURF PRNmPMMVg88RRVEifNVZWWKldjwxOVflfQWcTdZpdrKIoqjI/6VhWmvS8NuSUj8LJOkA uPm/DCXL7cy1yXQa1Uxhgha6cDIso6ZUu/kj6wqtVle2k3/BKCV2NAobWT67xPqdeRA6 iyxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266121; x=1686858121; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=c2X0vEktAU9CIzOCMmP9cn2Oif9GXs/gZcvny2bfDX8=; b=Cdfxj4QrWCtg3VTYYLeFjfuGp4aDvO7dLBvUC/CkiIk7KGd3rpH0Q1zYKszaOZqp/W 6rEE5xwEeDftJqnlHRnTSS76IqTMmv+ARh7GmQKT1CX2j7SCXNfu3TFEK4oG8oqapbsu MZVvkvDQA7G1R9qlzDDVuRWMcHWMvvbC1vmLx7P7uBk6k5ZOfLQGJYKhkfvqh4ARQ+gm HFWNy98kT1hcT5R1RA9z9mwCVM09gP/O7F09EHlU6LMQdg9GEbaSzkjF1LE9993U9k/7 Ox4gnCgk23A8Vye3jrC+woc0hA4BjGhYUw/YhVwA4xzSr4r9yVBT0Bb6nbeClnk1mKNa WS3w== X-Gm-Message-State: AC+VfDxB7ShNF56xmaTcrGHaRgnkxLPzQkEWMiSbu/KLSlt30cudvrA1 k33BEgtZeFGU+QdVTRGqYRE8wbqu0wNuwX4Kh9U= X-Received: by 2002:a05:6a00:1749:b0:64a:f730:154b with SMTP id j9-20020a056a00174900b0064af730154bmr18049143pfc.5.1684266121556; Tue, 16 May 2023 12:42:01 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:01 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 16/80] tcg/loongarch64: Use full load/store helpers in user-only mode Date: Tue, 16 May 2023 12:40:41 -0700 Message-Id: <20230516194145.1749305-17-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::433; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x433.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Instead of using helper_unaligned_{ld,st}, use the full load/store helpers. This will allow the fast path to increase alignment to implement atomicity while not immediately raising an alignment exception. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/loongarch64/tcg-target.c.inc | 30 ------------------------------ 1 file changed, 30 deletions(-) diff --git a/tcg/loongarch64/tcg-target.c.inc b/tcg/loongarch64/tcg-target.c.inc index d1bc29826f..e651ec5c71 100644 --- a/tcg/loongarch64/tcg-target.c.inc +++ b/tcg/loongarch64/tcg-target.c.inc @@ -783,7 +783,6 @@ static bool tcg_out_sti(TCGContext *s, TCGType type, TCGArg val, * Load/store helpers for SoftMMU, and qemu_ld/st implementations */ -#if defined(CONFIG_SOFTMMU) static bool tcg_out_goto(TCGContext *s, const tcg_insn_unit *target) { tcg_out_opc_b(s, 0); @@ -822,35 +821,6 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) tcg_out_call_int(s, qemu_st_helpers[opc & MO_SIZE], false); return tcg_out_goto(s, l->raddr); } -#else -static bool tcg_out_fail_alignment(TCGContext *s, TCGLabelQemuLdst *l) -{ - /* resolve label address */ - if (!reloc_br_sk16(l->label_ptr[0], tcg_splitwx_to_rx(s->code_ptr))) { - return false; - } - - tcg_out_mov(s, TCG_TYPE_TL, TCG_REG_A1, l->addrlo_reg); - tcg_out_mov(s, TCG_TYPE_PTR, TCG_REG_A0, TCG_AREG0); - - /* tail call, with the return address back inline. */ - tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_RA, (uintptr_t)l->raddr); - tcg_out_call_int(s, (const void *)(l->is_ld ? helper_unaligned_ld - : helper_unaligned_st), true); - return true; -} - -static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) -{ - return tcg_out_fail_alignment(s, l); -} - -static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) -{ - return tcg_out_fail_alignment(s, l); -} - -#endif /* CONFIG_SOFTMMU */ typedef struct { TCGReg base; From patchwork Tue May 16 19:40:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682357 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp638729ltp; Tue, 16 May 2023 12:42:50 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6H+nfI1EneO7RgrHUULNBv4YiC435E58Gc2v64F6epNEzHMspuN8BDEXSrlgkD/FXxuQqE X-Received: by 2002:ad4:5bcd:0:b0:61b:5b4b:2a4 with SMTP id t13-20020ad45bcd000000b0061b5b4b02a4mr12844437qvt.29.1684266169756; Tue, 16 May 2023 12:42:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266169; cv=none; d=google.com; s=arc-20160816; b=gmOylqkdWrSDTu4b8rANwEN0oFE3ndrEZ5U/vSYLMzuKrfwIiG0rpYvRwjEw2u5iq4 ErX0zOFgjkNnTFwMR3ZZAmSJHPlaFi4ay9XbzP4BD3Ol6RqauTj0Ro/UVPa0wIsIN+LF 7QgTIcLpCaOGWaNk28ihiYNFRvpfLEo8An8WMEpDIFgpX3v+O+0QwL38glLGloyGvHzY Kv4CwPMJs5tRqDq7mJWrXDle0L1YApG+/UT7CJfaaLrWfMdxC9zaMSvXU8YWnANWo9tI qJh6INtzvXEdRK/YFpCqnfphhB25sF7DI+HtDXkvTnGc6wVPa23LDuZel7ILh404mUF1 Yoow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=W19ruQQxWDDIeEDzZGJivenEm5AmU9KSchMTkoB+WqU=; b=S4dzuu2BlhOjMOp9NHiROEE2mULeBHvng+yQvZW9W8jNFQDmEKRPmmyoY18QvvmktY Mc0M63u2x/8Q5fGJBwHu0ZcNn2z9i1bpHTBuNTcSBa7U4+2u6JHtmKqEMEwDnp1OljC+ hE0i9WIMNC/9CYWvlC4pHRWvPkrlUpL3wSWpNoFdD68DR79VPKfRw8AhyZBaSuG4j5Xp //Fi49Jl4CFJB4oc60O4NniFy1Kus6dWe24E77jLUQn6MsPmlYchKiP9J1qvY2rzKRbG lhbgNe1izig20dbYyTVrtJySL002ueTroqR2K4Sg/2asT1D4R6CoLHqdpvqlfG2DOe21 BAag== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ZeIhiwuV; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x24-20020a05620a12b800b0074e19a5795dsi231839qki.340.2023.05.16.12.42.49 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:42:49 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ZeIhiwuV; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yi-0006h1-Lr; Tue, 16 May 2023 15:42:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Ye-0006fM-VW for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:04 -0400 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yd-0002lO-44 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:04 -0400 Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-643557840e4so15613786b3a.2 for ; Tue, 16 May 2023 12:42:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266122; x=1686858122; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=W19ruQQxWDDIeEDzZGJivenEm5AmU9KSchMTkoB+WqU=; b=ZeIhiwuV0DyAsO/BKOZrb56VSlvN9ZtwUXAvDVoOwMDeNrGh0BXfqvb5nxJtBAs5CI FMkNnyfSt1DXd9rcHG5T+R6CPWHj6f9m3sd7v+uCG4uLe0QlVXdia7seXfcZnjzT1jUP jwb0IrKBYVqTOuGJucsOPKP60ROzrNztJ/K4qEVdNEGdpLwMB3UYkE7Gpu221vqiB0Lk Iv7uEzzGfZEPHEO3CWu0jWmCFEfcVXhkA8uAU3DbJGr3X+4vTdV5e5ZbOJlYulJ2NR3k WinCqN5imZexdyKVD1BqYoOPqiTkpxXwy+Ro0sMiLFjpQ6jC2ltMx+OOzx/q96lqtfsT AVag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266122; x=1686858122; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=W19ruQQxWDDIeEDzZGJivenEm5AmU9KSchMTkoB+WqU=; b=NSxmKoy2z5lRhtEXXQFgcpE6QooWUIb2mcEdNyBfRRJa/w7oOAc1+xDcIFE93OLjsy aW8bEuHV33004ErFmdE+JW0p3stSw2uhMRlkVZX0Ntx8Na9lSV59Kcpa8/JIT8renDOD NsQpebhxe2as0qefmOTfaepXvN6GxSEV4/ri/Q86Q9CbIZNll/ca529VRLBIZRzdCw6h 5ZI3DdentpldQjlIPIY9oVunLfPdto66wNTLBmQqa8xHxT3flaU2babW+gYFSKlhIzOW AnFlbjzjByvv6H3wy8oglbG4e0eSvYHp6Xk/2u83t5MpXxIclsCpHCp6Id1Cbbt2/+u2 Q4Jg== X-Gm-Message-State: AC+VfDzN6Tl1exiI7zSK1sKjAyunJvEVDxc5wXMVby+0qAgS9rr+ezRf qpEYwkcxBR9j/gsIn/tQa/CHpXYqrRIznr1OXe0= X-Received: by 2002:a05:6a00:801:b0:63d:2260:f7d with SMTP id m1-20020a056a00080100b0063d22600f7dmr54301260pfk.8.1684266122352; Tue, 16 May 2023 12:42:02 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:01 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 17/80] tcg/riscv: Use full load/store helpers in user-only mode Date: Tue, 16 May 2023 12:40:42 -0700 Message-Id: <20230516194145.1749305-18-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42c; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Instead of using helper_unaligned_{ld,st}, use the full load/store helpers. This will allow the fast path to increase alignment to implement atomicity while not immediately raising an alignment exception. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/riscv/tcg-target.c.inc | 29 ----------------------------- 1 file changed, 29 deletions(-) diff --git a/tcg/riscv/tcg-target.c.inc b/tcg/riscv/tcg-target.c.inc index 8ed0e2f210..19cd4507fb 100644 --- a/tcg/riscv/tcg-target.c.inc +++ b/tcg/riscv/tcg-target.c.inc @@ -846,7 +846,6 @@ static void tcg_out_mb(TCGContext *s, TCGArg a0) * Load/store and TLB */ -#if defined(CONFIG_SOFTMMU) static void tcg_out_goto(TCGContext *s, const tcg_insn_unit *target) { tcg_out_opc_jump(s, OPC_JAL, TCG_REG_ZERO, 0); @@ -893,34 +892,6 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) tcg_out_goto(s, l->raddr); return true; } -#else -static bool tcg_out_fail_alignment(TCGContext *s, TCGLabelQemuLdst *l) -{ - /* resolve label address */ - if (!reloc_sbimm12(l->label_ptr[0], tcg_splitwx_to_rx(s->code_ptr))) { - return false; - } - - tcg_out_mov(s, TCG_TYPE_TL, TCG_REG_A1, l->addrlo_reg); - tcg_out_mov(s, TCG_TYPE_PTR, TCG_REG_A0, TCG_AREG0); - - /* tail call, with the return address back inline. */ - tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_RA, (uintptr_t)l->raddr); - tcg_out_call_int(s, (const void *)(l->is_ld ? helper_unaligned_ld - : helper_unaligned_st), true); - return true; -} - -static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) -{ - return tcg_out_fail_alignment(s, l); -} - -static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) -{ - return tcg_out_fail_alignment(s, l); -} -#endif /* CONFIG_SOFTMMU */ /* * For softmmu, perform the TLB load and compare. From patchwork Tue May 16 19:40:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682370 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp639639ltp; Tue, 16 May 2023 12:45:04 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5SBMuBeOzomKZebYS6tlpjOMang8f42WanrRWgGYGOXXkRGaXLcL7Ig/pTzyKTecmka8KJ X-Received: by 2002:a05:622a:118f:b0:3f4:c9b:24ca with SMTP id m15-20020a05622a118f00b003f40c9b24camr40314436qtk.53.1684266304663; Tue, 16 May 2023 12:45:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266304; cv=none; d=google.com; s=arc-20160816; b=k91VG4nrkOymM7qErPtuc2dRarWh7rIdi+NrN7WI8DNd5mmVZwqKhNuOeKEyQ/C1+Y CamUdA1ZVG2IpmacXLqCq8sP9hfs0mXLiKd4+C8dpwtY0XdePnWdAeily1ZzG4inDgQc ahIPSFkLqwb71Np0uq/gMBcj4wlltUBfVFjEZ7R34n3sV4ueYpGpWv0m9wsa7s0i1npK hmMpXeBBOmfnmccCKpkcBabzfeCbuGPiKgrLuePdJUx/Bwo12P5xAtMjH2iZYdTrPY5o d4cPJ2DNRdXMJomZMJEDhXRNZydLQs5s4WPgaaIPmHGCsBfNtfOXCFehWfMTY5AD5lrn 7XEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=JU4vdDs/6qpsKlmOEFRbKkCiPMY/9YXw9xFKAMp8gMg=; b=MZwIVQQPk+FXFG7MBQvsGZapXb0+zpWhPKvU35QuwtGeQR2fI+zFBQpLUtnhyjDg2c /VYgnP0sJ44ghYjNlQ9yNehlOvWN4w5lDTxDZX4kNNNZQ9lla5spqNLWIPr31y74cYwi Tb+V5Fv8ivKEppMbgoygJdOUZOS+MF+F22gi9+BtMsxfIJpL46Xo4dlMRBHrPFmsmfcD qaWD1K8288FTUXP15ELt1a9ld5Km9Ww+CYhvHwuyl8PUPdm+C6+pI3TU9AUqW/sdM9KJ A3E9/nRqMbspF4Ca5bP4jn+/PNOoClsOvcaxuOmxlJXU4mJlOmNbPVgicxLGs28bCSt+ y+Qw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fhACcL9M; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id s4-20020a05620a16a400b0075789213edesi216070qkj.384.2023.05.16.12.45.04 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:45:04 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fhACcL9M; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yj-0006hg-8M; Tue, 16 May 2023 15:42:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yg-0006gJ-Ey for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:06 -0400 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Ye-0002pV-Oa for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:06 -0400 Received: by mail-pg1-x530.google.com with SMTP id 41be03b00d2f7-52867360efcso10426275a12.2 for ; Tue, 16 May 2023 12:42:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266123; x=1686858123; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JU4vdDs/6qpsKlmOEFRbKkCiPMY/9YXw9xFKAMp8gMg=; b=fhACcL9MpK0F5hlkhWB9aQBYdkFEEQuq+jknL40Xx/Jbacn3+lmeSW0o89rkL73WJF zX18DIc8feyZLxdEb+aj5kgbgweFbNxjWE+A7RVYd+3gX8B6dAHPlIbOx60+08UGPVCk q8DEg65wfw7CTNNR3yxLzxV9obHdqbe2FrOgBQ425CNqDwTqZRncvwc/TuNPtDH6SpLX kHUh4w6tnUBJHbU3ttwFWLEhzfj1f2mEJ7qpXPm3ARVtLmr/BCflF/et/azvNCi3EZ9d nUtZGC7r41xnSp4AyovHUvgMmh57HK2viBarBOdmLuLJn1IQjSPICR/rtES0Q1/IDLiz tPKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266123; x=1686858123; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JU4vdDs/6qpsKlmOEFRbKkCiPMY/9YXw9xFKAMp8gMg=; b=GnVmmmArx4A+1qGkOjc/HVbO7/2PyD62J+Lssk9WtVh3UUWmzT4CL/6TshORgBNKxN 0AJYJvbyAqYRD/bLFoZDvaWy7Y8JWtwieTbjhy/oxmlnjqyfyTG2/B2YC4k1qoLthZWZ Aab9+DkxGaQccA+c9slkvtPzFSBExrW7BWm4fr/3fMbz8x5qBxxsJY24e+gwxAH5VyU3 ajP5OW3sgkjdOOxdJPR6lROJbc3fEedTnbS3DcKVVBXyb8xNzID3kd6UmijRHTAZyG6Z YPDZOCll1qle3Wp3etc1SwjfvZr5OaGGiwQ4ZWe3NIADYJdYFLneJsR1ANYjHJKVFDOr adBA== X-Gm-Message-State: AC+VfDynVC3xPswtG7tiTRzI9FQvN8X8MY7aMZASxuLLF7J9tGGL12cN mBo1PjMQvL+wgy+XRhelURDb6CNiav/cH3AHgeY= X-Received: by 2002:a05:6a20:4299:b0:100:bda9:4b39 with SMTP id o25-20020a056a20429900b00100bda94b39mr38837991pzj.34.1684266123356; Tue, 16 May 2023 12:42:03 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:03 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 18/80] tcg/arm: Adjust constraints on qemu_ld/st Date: Tue, 16 May 2023 12:40:43 -0700 Message-Id: <20230516194145.1749305-19-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::530; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x530.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Always reserve r3 for tlb softmmu lookup. Fix a bug in user-only ALL_QLDST_REGS, in that r14 is clobbered by the BLNE that leads to the misaligned trap. Remove r0+r1 from user-only ALL_QLDST_REGS; I believe these had been reserved for bswap, which we no longer perform during qemu_st. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/arm/tcg-target-con-set.h | 16 ++++++++-------- tcg/arm/tcg-target-con-str.h | 5 ++--- tcg/arm/tcg-target.c.inc | 23 ++++++++--------------- 3 files changed, 18 insertions(+), 26 deletions(-) diff --git a/tcg/arm/tcg-target-con-set.h b/tcg/arm/tcg-target-con-set.h index b8849b2478..229ae258ac 100644 --- a/tcg/arm/tcg-target-con-set.h +++ b/tcg/arm/tcg-target-con-set.h @@ -12,19 +12,19 @@ C_O0_I1(r) C_O0_I2(r, r) C_O0_I2(r, rIN) -C_O0_I2(s, s) +C_O0_I2(q, q) C_O0_I2(w, r) -C_O0_I3(s, s, s) -C_O0_I3(S, p, s) +C_O0_I3(q, q, q) +C_O0_I3(Q, p, q) C_O0_I4(r, r, rI, rI) -C_O0_I4(S, p, s, s) -C_O1_I1(r, l) +C_O0_I4(Q, p, q, q) +C_O1_I1(r, q) C_O1_I1(r, r) C_O1_I1(w, r) C_O1_I1(w, w) C_O1_I1(w, wr) C_O1_I2(r, 0, rZ) -C_O1_I2(r, l, l) +C_O1_I2(r, q, q) C_O1_I2(r, r, r) C_O1_I2(r, r, rI) C_O1_I2(r, r, rIK) @@ -39,8 +39,8 @@ C_O1_I2(w, w, wZ) C_O1_I3(w, w, w, w) C_O1_I4(r, r, r, rI, rI) C_O1_I4(r, r, rIN, rIK, 0) -C_O2_I1(e, p, l) -C_O2_I2(e, p, l, l) +C_O2_I1(e, p, q) +C_O2_I2(e, p, q, q) C_O2_I2(r, r, r, r) C_O2_I4(r, r, r, r, rIN, rIK) C_O2_I4(r, r, rI, rI, rIN, rIK) diff --git a/tcg/arm/tcg-target-con-str.h b/tcg/arm/tcg-target-con-str.h index 24b4b59feb..f83f1d3919 100644 --- a/tcg/arm/tcg-target-con-str.h +++ b/tcg/arm/tcg-target-con-str.h @@ -10,9 +10,8 @@ */ REGS('e', ALL_GENERAL_REGS & 0x5555) /* even regs */ REGS('r', ALL_GENERAL_REGS) -REGS('l', ALL_QLOAD_REGS) -REGS('s', ALL_QSTORE_REGS) -REGS('S', ALL_QSTORE_REGS & 0x5555) /* even qstore */ +REGS('q', ALL_QLDST_REGS) +REGS('Q', ALL_QLDST_REGS & 0x5555) /* even qldst */ REGS('w', ALL_VECTOR_REGS) /* diff --git a/tcg/arm/tcg-target.c.inc b/tcg/arm/tcg-target.c.inc index 8b0d526659..a02804dd69 100644 --- a/tcg/arm/tcg-target.c.inc +++ b/tcg/arm/tcg-target.c.inc @@ -353,23 +353,16 @@ static bool patch_reloc(tcg_insn_unit *code_ptr, int type, #define ALL_VECTOR_REGS 0xffff0000u /* - * r0-r2 will be overwritten when reading the tlb entry (softmmu only) - * and r0-r1 doing the byte swapping, so don't use these. - * r3 is removed for softmmu to avoid clashes with helper arguments. + * r0-r3 will be overwritten when reading the tlb entry (softmmu only); + * r14 will be overwritten by the BLNE branching to the slow path. */ #ifdef CONFIG_SOFTMMU -#define ALL_QLOAD_REGS \ +#define ALL_QLDST_REGS \ (ALL_GENERAL_REGS & ~((1 << TCG_REG_R0) | (1 << TCG_REG_R1) | \ (1 << TCG_REG_R2) | (1 << TCG_REG_R3) | \ (1 << TCG_REG_R14))) -#define ALL_QSTORE_REGS \ - (ALL_GENERAL_REGS & ~((1 << TCG_REG_R0) | (1 << TCG_REG_R1) | \ - (1 << TCG_REG_R2) | (1 << TCG_REG_R14) | \ - ((TARGET_LONG_BITS == 64) << TCG_REG_R3))) #else -#define ALL_QLOAD_REGS ALL_GENERAL_REGS -#define ALL_QSTORE_REGS \ - (ALL_GENERAL_REGS & ~((1 << TCG_REG_R0) | (1 << TCG_REG_R1))) +#define ALL_QLDST_REGS (ALL_GENERAL_REGS & ~(1 << TCG_REG_R14)) #endif /* @@ -2203,13 +2196,13 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) return C_O1_I4(r, r, r, rI, rI); case INDEX_op_qemu_ld_i32: - return TARGET_LONG_BITS == 32 ? C_O1_I1(r, l) : C_O1_I2(r, l, l); + return TARGET_LONG_BITS == 32 ? C_O1_I1(r, q) : C_O1_I2(r, q, q); case INDEX_op_qemu_ld_i64: - return TARGET_LONG_BITS == 32 ? C_O2_I1(e, p, l) : C_O2_I2(e, p, l, l); + return TARGET_LONG_BITS == 32 ? C_O2_I1(e, p, q) : C_O2_I2(e, p, q, q); case INDEX_op_qemu_st_i32: - return TARGET_LONG_BITS == 32 ? C_O0_I2(s, s) : C_O0_I3(s, s, s); + return TARGET_LONG_BITS == 32 ? C_O0_I2(q, q) : C_O0_I3(q, q, q); case INDEX_op_qemu_st_i64: - return TARGET_LONG_BITS == 32 ? C_O0_I3(S, p, s) : C_O0_I4(S, p, s, s); + return TARGET_LONG_BITS == 32 ? C_O0_I3(Q, p, q) : C_O0_I4(Q, p, q, q); case INDEX_op_st_vec: return C_O0_I2(w, r); From patchwork Tue May 16 19:40:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682365 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp639151ltp; Tue, 16 May 2023 12:43:52 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ62Zwcq0uNf8oCWSWxlrAVplPKyJ3hxCsQ/Wnq80/qHuopeHKNfHjRdW4GqshBc3BxClKGu X-Received: by 2002:a05:6214:29ec:b0:621:2bff:ae81 with SMTP id jv12-20020a05621429ec00b006212bffae81mr49109658qvb.30.1684266231956; Tue, 16 May 2023 12:43:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266231; cv=none; d=google.com; s=arc-20160816; b=pCMpO/F5W4vQbXl7VvK+ZoBat1pFKCpVBws+fIIdi2lTE8Cf/ThzTE/9onMKMFWxLF 6ctl5VvjYzXsUWLrc474hk/sgryNvmelX+sF08osq3T2k5dt/j3heujaQqN9GUwYG15d UNrt4X9hpMc9FgsNlF9G8wevvYzjQGyGDxBQxHw6EDRCF6hBguadyYmkZRce43/f4eiw /WdsZFmI3yCgYKba1zMlWdbnZyji8Zmq7tk+vBld7K8Y4cZdf7an9V/ykRwswUDSHyAX TYWtgqawvP+IgJ9vCcbcYDzPLURFUPfGOhic3hsvD4aCxBMspyPu55WlBwDtc+BVkcft bwXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=gXvb5M+QQ4GlJC4+AWTZXZCLTvkRCJufT4wwLa/Umbo=; b=NgCVhlkbKqaogEnCCfgbSKzdO2RfJoj4T388DMSqZRghg8amxuFoWnroCrsv5YCXNx I2HrjiVb4HbUvxcDSjgr9l2IDrslyEjz9wdWQpof1vywDh4a7XvVjoXmdifeu47kOL8T 3ha+BG5g0MxgXiNHQhSLV47jm2WFxPuPxQNMMkUoBbmJ+HHLtJmLPlEL0FKacSBSpWZg APUdWBHnCSZpFh6/DgeuqZdVNn4ThhXrVHw7RWGzNi4T6iw9Pfc6jKy+vF8gtjMB6v9l 31gHRIFFlhhKTTIPd/GTtFXXJxqISPgIeV6N8S9YTGCQCNeSNCFjMy3b57WAXf4oL56p B56g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="mbB7mZS/"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i5-20020a05620a150500b007590d551ca4si197597qkk.550.2023.05.16.12.43.51 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:43:51 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="mbB7mZS/"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yi-0006h3-5D; Tue, 16 May 2023 15:42:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yh-0006gt-7q for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:07 -0400 Received: from mail-pf1-x42b.google.com ([2607:f8b0:4864:20::42b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yf-0002pt-JK for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:06 -0400 Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-6439d505274so9270805b3a.0 for ; Tue, 16 May 2023 12:42:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266124; x=1686858124; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gXvb5M+QQ4GlJC4+AWTZXZCLTvkRCJufT4wwLa/Umbo=; b=mbB7mZS/rh8KtNwrnd06tNhkE2dAL4nJHTfzdCTgA6g/Qcip/h5kxAyGwS6pn2Xx1w SLb9L0DhuoO7VFlSgpluK70wKup+ouVICxvRtWLN9/MJyUYKKWNkPQoLiV7oc74WHyE2 x+xcClk/2dwk3bOyYcFob/m4J9j/pX8QF130Ej6FK3+wyP+g7WDIPwwcbassyezGipAP BKtX8fTHA2Zl0BK2lRMO3QLmPGwWnFAj7ZVqJH40S3ONuGtk822VrPBnOVHLnQX9E2wY TVwOlFgl16+4PjXMjFvD6AP+T+tCV7TxImkFhFXLAELiTKYt2WiVzAbCFKZL3592PSe0 K+zA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266124; x=1686858124; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gXvb5M+QQ4GlJC4+AWTZXZCLTvkRCJufT4wwLa/Umbo=; b=PMK8py22SoAzG6DMXOq+wzTNdIA4snZnw+EA7moLYSeJP9v3494OvbH0m/zwYb5l0y zdJgpgXuXnhienmenl1esaBeIqoD4HBsdPfobWbc3RD7bi7f9EmJXfqbOV20ssG2YAk9 iG1kU4nciIslre6IXSgu8oBDUt8cUZH9Obe55JRNUUMybnWcZ1JWhdvu7c/lR3k1j1Zj K4Qm3Oi26ev8v/ojuYjaUFvvgEo1gTxGnwtD18T/WYXmbM/1LxL7luuE1bGK5+o4k3+b Vje+3uUl9GueYx+J+PWWGy9loCZY5N2XsZDJY2r6xHNCBiJzLUsvN8F2SKQnaUzYecRv 2ypA== X-Gm-Message-State: AC+VfDyOr0QQbo3lxMvHgEKXs5pR10sR0J6kNboHFzL/V456rHI189jj HPde4vxtaZP83NkjhzUm7rF2miWvNLU63cYdcg0= X-Received: by 2002:a05:6a00:a8b:b0:643:b27f:6c43 with SMTP id b11-20020a056a000a8b00b00643b27f6c43mr48193547pfl.27.1684266124208; Tue, 16 May 2023 12:42:04 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:03 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 19/80] tcg/arm: Use full load/store helpers in user-only mode Date: Tue, 16 May 2023 12:40:44 -0700 Message-Id: <20230516194145.1749305-20-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42b; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Instead of using helper_unaligned_{ld,st}, use the full load/store helpers. This will allow the fast path to increase alignment to implement atomicity while not immediately raising an alignment exception. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/arm/tcg-target.c.inc | 45 ---------------------------------------- 1 file changed, 45 deletions(-) diff --git a/tcg/arm/tcg-target.c.inc b/tcg/arm/tcg-target.c.inc index a02804dd69..eb0542f32e 100644 --- a/tcg/arm/tcg-target.c.inc +++ b/tcg/arm/tcg-target.c.inc @@ -1325,7 +1325,6 @@ typedef struct { bool index_scratch; } HostAddress; -#ifdef CONFIG_SOFTMMU static TCGReg ldst_ra_gen(TCGContext *s, const TCGLabelQemuLdst *l, int arg) { /* We arrive at the slow path via "BLNE", so R14 contains l->raddr. */ @@ -1368,50 +1367,6 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) tcg_out_goto(s, COND_AL, qemu_st_helpers[opc & MO_SIZE]); return true; } -#else -static bool tcg_out_fail_alignment(TCGContext *s, TCGLabelQemuLdst *l) -{ - if (!reloc_pc24(l->label_ptr[0], tcg_splitwx_to_rx(s->code_ptr))) { - return false; - } - - if (TARGET_LONG_BITS == 64) { - /* 64-bit target address is aligned into R2:R3. */ - TCGMovExtend ext[2] = { - { .dst = TCG_REG_R2, .dst_type = TCG_TYPE_I32, - .src = l->addrlo_reg, - .src_type = TCG_TYPE_I32, .src_ext = MO_UL }, - { .dst = TCG_REG_R3, .dst_type = TCG_TYPE_I32, - .src = l->addrhi_reg, - .src_type = TCG_TYPE_I32, .src_ext = MO_UL }, - }; - tcg_out_movext2(s, &ext[0], &ext[1], TCG_REG_TMP); - } else { - tcg_out_mov(s, TCG_TYPE_I32, TCG_REG_R1, l->addrlo_reg); - } - tcg_out_mov(s, TCG_TYPE_PTR, TCG_REG_R0, TCG_AREG0); - - /* - * Tail call to the helper, with the return address back inline, - * just for the clarity of the debugging traceback -- the helper - * cannot return. We have used BLNE to arrive here, so LR is - * already set. - */ - tcg_out_goto(s, COND_AL, (const void *) - (l->is_ld ? helper_unaligned_ld : helper_unaligned_st)); - return true; -} - -static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) -{ - return tcg_out_fail_alignment(s, l); -} - -static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) -{ - return tcg_out_fail_alignment(s, l); -} -#endif /* SOFTMMU */ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, TCGReg addrlo, TCGReg addrhi, From patchwork Tue May 16 19:40:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682356 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp638616ltp; Tue, 16 May 2023 12:42:31 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4fqiKD+xBcUA3I+/VoRez7E1o+o/wPlnpXddxComrishxiHXKtpQwVNKDqC6SoNiSRKQ6/ X-Received: by 2002:ac8:5b4a:0:b0:3f5:41d9:fde8 with SMTP id n10-20020ac85b4a000000b003f541d9fde8mr5691094qtw.57.1684266151280; Tue, 16 May 2023 12:42:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266151; cv=none; d=google.com; s=arc-20160816; b=d+UtV7Ahze91KAGoGfdXZ0j41ruMkR4TNt5vXSEqNUZWkmPAXUi0eEMuXva8G2i9vS zECv+A4rHX/8ITNbTEAQDqCCq5txe1fL9Vn3sj6m0lRImmVYGeFh9qJ+Vur1PyEPboP9 LM1d/dPeEDWqIFLhTKyS14WpYjS4eBp4QicXe4Cu90eZyJ/q9g3GduV3cSFe2Y/O/aUE Kvizj40ZGmRsOKh/xHqbuEYJVgYUUmFz823UPcf2eaEwLM8aC+vUqcwgKmaKHGrELITl HHnV6hUD5HrB9t7vrDOqfrMxrL5ki3SjS55p52KJwbAarqc61DmqCaLrWeYl9dM8I2G0 LLBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=K2D+W3Frhm1HnR66jCW7Uv+/eEEehjSzUkt7BPDzA4s=; b=eF8IAs7fIatayPPHorBUURsj5HufHkj0mcBGHllhXnkkeLtsXn3eEJeJY9XbEbmEpT lz/dWyDkFivarXNMuzail1suD6fgHlb4wQ2aaTbXGDdnwVfAAnPuiciY/KO306NVlvw1 zi8XWYUqjqbDgMl3020agnqetFs+psBQlJzpfkbBxTScv7gOOvmYhdQRPHxSGYUqhDGX +I8u5yLDqzUWZ0PFN4utZ3x3vcy+pCFWfnf1aRE2TWQe8+psO8GoGrLwdljHAPqB+PNV 0C34874xkLgWF0sfkvbG87efSjiKOFOSIAqc0suXqp2WCGM5TWLJKEaiPk0Z0TeIkhEw 4Cyg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TTKN56KE; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id q5-20020a05620a038500b007590571f3c5si186388qkm.663.2023.05.16.12.42.31 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:42:31 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TTKN56KE; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yj-0006iI-VZ; Tue, 16 May 2023 15:42:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yi-0006h7-8H for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:08 -0400 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yg-0002qM-Hn for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:08 -0400 Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-64359d9c531so10855218b3a.3 for ; Tue, 16 May 2023 12:42:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266125; x=1686858125; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=K2D+W3Frhm1HnR66jCW7Uv+/eEEehjSzUkt7BPDzA4s=; b=TTKN56KE0YQPNgRxxpnHIaBG1HjMfRK70FxkKxZLzubfgPh0SGvRhoa8BinspPrvUz xX8fau9il95dLxKo9Maxf7eERP+BYb/QB7I3AdQhZ96ad7fLYcpZ3whmSHrnU1sFU8eo 61Q/AzedOVpjfKLPJeg4s6Dd+ku6APPmrASvgd3ytKlzNG62DSJ1OZ6WG5LS5D++d2rc wKX2nMpmUlRHYUABWnHPEZC4AGgVsZ8x8IT735MO1gwgr1DY7H5PYabtBvf8JT0rSxKp TtlFZIFXaVOTQEIXJYD2/9Gim+lOXrenuInIQQ2miR3MuCLoIIdHumMakkN23d/mMNfD MMZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266125; x=1686858125; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=K2D+W3Frhm1HnR66jCW7Uv+/eEEehjSzUkt7BPDzA4s=; b=E6U0BQvdqZ0+AMZt/yjLULNUkOSkkWfibrAOn9p0wGZ1x0/oPe/iiUpSaZEikptMzu dKpeiK7w44cHhQdT8sXunm+ze413D3kIpoPMhgZjdEsvvvFzYP6owBSbexGlV8sR8tpI 64YaPKqoUV2w+eAk0lNmyBrFxCYHkHPYBXZ4zUyyT2s5Q8++eEtjkNd/zNbCTRp+2D+o beUzowOum1iihPjDYHbDo0G1XGFq5f8sYgHyHhgURiJkOvKAVJdsq7Rq9B8sLgt6vfdP /NR2X0I4YhqEm40MR1EUPpi/SmRvaues0leIkj/cq997/kE8ZsthOBlqmpllSgw4KMxc x/LA== X-Gm-Message-State: AC+VfDz0j/1ZLUIh6snvEGUc4TJNDt5E/Q6X6oMfiCsAPdR6a3CAQzWE pokBXsxxVCldc1AP4gkGZKQWH3P1BVcZCUZdXXA= X-Received: by 2002:a05:6a00:23d3:b0:63d:4752:4da3 with SMTP id g19-20020a056a0023d300b0063d47524da3mr50326109pfc.25.1684266125278; Tue, 16 May 2023 12:42:05 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:04 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 20/80] tcg/mips: Use full load/store helpers in user-only mode Date: Tue, 16 May 2023 12:40:45 -0700 Message-Id: <20230516194145.1749305-21-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::429; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x429.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Instead of using helper_unaligned_{ld,st}, use the full load/store helpers. This will allow the fast path to increase alignment to implement atomicity while not immediately raising an alignment exception. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/mips/tcg-target.c.inc | 57 ++------------------------------------- 1 file changed, 2 insertions(+), 55 deletions(-) diff --git a/tcg/mips/tcg-target.c.inc b/tcg/mips/tcg-target.c.inc index 7770ef46bd..fa0f334e8d 100644 --- a/tcg/mips/tcg-target.c.inc +++ b/tcg/mips/tcg-target.c.inc @@ -1075,7 +1075,6 @@ static void tcg_out_call(TCGContext *s, const tcg_insn_unit *arg, tcg_out_nop(s); } -#if defined(CONFIG_SOFTMMU) /* We have four temps, we might as well expose three of them. */ static const TCGLdstHelperParam ldst_helper_param = { .ntmp = 3, .tmp = { TCG_TMP0, TCG_TMP1, TCG_TMP2 } @@ -1088,8 +1087,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) /* resolve label address */ if (!reloc_pc16(l->label_ptr[0], tgt_rx) - || (TCG_TARGET_REG_BITS < TARGET_LONG_BITS - && !reloc_pc16(l->label_ptr[1], tgt_rx))) { + || (l->label_ptr[1] && !reloc_pc16(l->label_ptr[1], tgt_rx))) { return false; } @@ -1118,8 +1116,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) /* resolve label address */ if (!reloc_pc16(l->label_ptr[0], tgt_rx) - || (TCG_TARGET_REG_BITS < TARGET_LONG_BITS - && !reloc_pc16(l->label_ptr[1], tgt_rx))) { + || (l->label_ptr[1] && !reloc_pc16(l->label_ptr[1], tgt_rx))) { return false; } @@ -1139,56 +1136,6 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) return true; } -#else -static bool tcg_out_fail_alignment(TCGContext *s, TCGLabelQemuLdst *l) -{ - void *target; - - if (!reloc_pc16(l->label_ptr[0], tcg_splitwx_to_rx(s->code_ptr))) { - return false; - } - - if (TCG_TARGET_REG_BITS < TARGET_LONG_BITS) { - /* A0 is env, A1 is skipped, A2:A3 is the uint64_t address. */ - TCGReg a2 = MIPS_BE ? l->addrhi_reg : l->addrlo_reg; - TCGReg a3 = MIPS_BE ? l->addrlo_reg : l->addrhi_reg; - - if (a3 != TCG_REG_A2) { - tcg_out_mov(s, TCG_TYPE_I32, TCG_REG_A2, a2); - tcg_out_mov(s, TCG_TYPE_I32, TCG_REG_A3, a3); - } else if (a2 != TCG_REG_A3) { - tcg_out_mov(s, TCG_TYPE_I32, TCG_REG_A3, a3); - tcg_out_mov(s, TCG_TYPE_I32, TCG_REG_A2, a2); - } else { - tcg_out_mov(s, TCG_TYPE_I32, TCG_TMP0, TCG_REG_A2); - tcg_out_mov(s, TCG_TYPE_I32, TCG_REG_A2, TCG_REG_A3); - tcg_out_mov(s, TCG_TYPE_I32, TCG_REG_A3, TCG_TMP0); - } - } else { - tcg_out_mov(s, TCG_TYPE_TL, TCG_REG_A1, l->addrlo_reg); - } - tcg_out_mov(s, TCG_TYPE_PTR, TCG_REG_A0, TCG_AREG0); - - /* - * Tail call to the helper, with the return address back inline. - * We have arrived here via BNEL, so $31 is already set. - */ - target = (l->is_ld ? helper_unaligned_ld : helper_unaligned_st); - tcg_out_call_int(s, target, true); - return true; -} - -static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) -{ - return tcg_out_fail_alignment(s, l); -} - -static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) -{ - return tcg_out_fail_alignment(s, l); -} -#endif /* SOFTMMU */ - typedef struct { TCGReg base; MemOp align; From patchwork Tue May 16 19:40:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682367 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp639278ltp; Tue, 16 May 2023 12:44:12 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7+f3AeNWq2G9QBsJnhZRD707DzSp/trnZJwP21aBs/EF3DhkqZ+wktFCUQqel4DiTJyIk3 X-Received: by 2002:ac8:7c4d:0:b0:3f2:384:9292 with SMTP id o13-20020ac87c4d000000b003f203849292mr59154722qtv.34.1684266251822; Tue, 16 May 2023 12:44:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266251; cv=none; d=google.com; s=arc-20160816; b=LVTvzKGVtB2KaVYHjaLk0Z6TUrSzXL5YLDatbvURgMcBVEudGphHWnVcxqyMbE5+Ti 2966OQiKVvb7Do49yeDP89rREeCdNHsHCZ1aMfHghk7RrbVqHB31xV1vaU+RL214Ckkg SWJPf7WVlCgccPjK9TNEFRMzIaVaGXDwAkY22We/ky92SV7cjigeGcS4l7o9gBiTbduA vqqL9VSvIVZXfog2vW9FPoy+FYY93hv0YjIskArDl+CVIol6i48X/A+0C/O/lZ/5sw7p KVoVxmwCOD0VmD1oMgBf+RxMTfw3OWgvfRnJ7YriWtHaNs1arKOUwwpEXNJ7gdfHzgxA dINw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=PEsWAFpnCMYNqvb5yhVEEwYFWaIpTnBv29asu3C8fXA=; b=JAuGF+WnJ27nS5fqGjXR88/2NYofUoo9518ZCGJeNf+Chh8AVWUVlA6iQ+18ZXSfQ/ knjF+VVBYNXQTOoBfq0YMMOE4oe5bGSvs20L3KYn0p9JEAKl867/PjHTprhJJKt/VYzc 6P8mjQyEsGiGxl0YvRXyFad03AxswRaPxMAlW/55wcvkR8gxW2CrRqmA9xFzsDCCRzWK G9KGcs++9vWj5ckKMQUfBcEF4GEIiQWwj0ODWTqpj+CuSm/VLARU6CBU7VqU2DNldv4r HBQ785u7sPSqrcv50Mr7wEathw7iZPUzGAS/pRHlsWMAFZZ2LrkOMBZtc5GfKa2qNwss Fa/g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=BDwoJPT3; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f22-20020a05622a105600b003b689c70396si11664179qte.86.2023.05.16.12.44.11 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:44:11 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=BDwoJPT3; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yk-0006iU-ON; Tue, 16 May 2023 15:42:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yj-0006hp-GN for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:09 -0400 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yg-0002jW-UI for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:09 -0400 Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-643a6f993a7so9345404b3a.1 for ; Tue, 16 May 2023 12:42:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266126; x=1686858126; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PEsWAFpnCMYNqvb5yhVEEwYFWaIpTnBv29asu3C8fXA=; b=BDwoJPT3tJA8LuDQMGq7qGKQjQ5oLt3zrE1kSuzeEbI8De+ckLxl9kASJIzQSMVm7b wQ++4sOO5VpmuI4fez5wyQBXmdy0e7NOCfakw6hhMQNmzIAPmAWeFCTSi9+eKESFWsR/ CTjMrdaSgG8/iARFPSQVJo7TbDMSorliKJUwFX3x22uamqsHHmQBFfCq/6vt+XGkcIJF rXGp+w4iIhVXOrqUpANJvEt7VDZ0mgT+FdS0EzVo9F0KY328wLiorpAtuoZDiA4Fao7N 7oNQizi5/vAr7kmXG8GfEsMtSO7ESxiKZ8KVJ1XXpMlR8shZUZa2hTfvz2oCLgZsl+eZ nPew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266126; x=1686858126; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PEsWAFpnCMYNqvb5yhVEEwYFWaIpTnBv29asu3C8fXA=; b=Et9Aa05dwT0ZK5xYiGAoXQAkdJQgi0fJSwNl3HO40/douB2FUh9IOu16zQHp+RU3lW Z7tmYBQXp5XC4IvsGYCJ7Lpiwxlak3HpLylYGb8cv/IwT+H/SGg2LMUaSf8SHdfNcqXV n2HmASe3C3b7pgJhDLP9g759FsdNfp0QXytHl2DRk0cIBHThsYqRcRRqRvsy5zGQgs+5 BfLJTWhEe1DQ6hHKEkbwfrInTjP0q15o0hM5YqSITU6exDyWWu2zGBrMQRYQfe/SyNpM t3+tnuzhJzRtnBPwe5+EgqyUxAKPReuSB6TIIrx35axRT8b+ANie8PIaVNlmJt/AWCA0 kGwQ== X-Gm-Message-State: AC+VfDy27Gr7kJ9AF4EAzzfiMy+33vLQCL5O/sNhtqrA70OIui0cBhxq 4wjp1wclvQHRa4qpmPhozHbE3TqQHrSq8zwuN2M= X-Received: by 2002:a05:6a00:194b:b0:647:f128:c4f5 with SMTP id s11-20020a056a00194b00b00647f128c4f5mr31552345pfk.22.1684266126154; Tue, 16 May 2023 12:42:06 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:05 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 21/80] tcg/s390x: Use full load/store helpers in user-only mode Date: Tue, 16 May 2023 12:40:46 -0700 Message-Id: <20230516194145.1749305-22-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::432; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x432.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Instead of using helper_unaligned_{ld,st}, use the full load/store helpers. This will allow the fast path to increase alignment to implement atomicity while not immediately raising an alignment exception. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/s390x/tcg-target.c.inc | 29 ----------------------------- 1 file changed, 29 deletions(-) diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc index 968977be98..de8aed5f77 100644 --- a/tcg/s390x/tcg-target.c.inc +++ b/tcg/s390x/tcg-target.c.inc @@ -1679,7 +1679,6 @@ static void tcg_out_qemu_st_direct(TCGContext *s, MemOp opc, TCGReg data, } } -#if defined(CONFIG_SOFTMMU) static const TCGLdstHelperParam ldst_helper_param = { .ntmp = 1, .tmp = { TCG_TMP0 } }; @@ -1716,34 +1715,6 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) tgen_gotoi(s, S390_CC_ALWAYS, lb->raddr); return true; } -#else -static bool tcg_out_fail_alignment(TCGContext *s, TCGLabelQemuLdst *l) -{ - if (!patch_reloc(l->label_ptr[0], R_390_PC16DBL, - (intptr_t)tcg_splitwx_to_rx(s->code_ptr), 2)) { - return false; - } - - tcg_out_mov(s, TCG_TYPE_TL, TCG_REG_R3, l->addrlo_reg); - tcg_out_mov(s, TCG_TYPE_PTR, TCG_REG_R2, TCG_AREG0); - - /* "Tail call" to the helper, with the return address back inline. */ - tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_R14, (uintptr_t)l->raddr); - tgen_gotoi(s, S390_CC_ALWAYS, (const void *)(l->is_ld ? helper_unaligned_ld - : helper_unaligned_st)); - return true; -} - -static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) -{ - return tcg_out_fail_alignment(s, l); -} - -static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) -{ - return tcg_out_fail_alignment(s, l); -} -#endif /* CONFIG_SOFTMMU */ /* * For softmmu, perform the TLB load and compare. From patchwork Tue May 16 19:40:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682366 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp639274ltp; Tue, 16 May 2023 12:44:11 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4oXUwd6Hh9G4Cj2jor0wJ61/hCsZ5avuHHyuhf2QOHyAhOisqGJAybbeIlGeFXBPHV8WYO X-Received: by 2002:a05:6214:21cb:b0:5f1:6a6a:f566 with SMTP id d11-20020a05621421cb00b005f16a6af566mr82894108qvh.19.1684266251626; Tue, 16 May 2023 12:44:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266251; cv=none; d=google.com; s=arc-20160816; b=pwyDvSbaGtBrKkWo38lt66ldOTV47O0JBcj5CKHnNm5lgUpJ2c2M+0QVSaoiYlvTOT anWgGxPxXlxQAjTccyTXkCh5tUOHnjK3X0yomWoNLgykj0YVC4v6ayUzouAIvbRiyxkq bjUzqsC12ZBBk7Mn4BHxGo2AWk4o3L1fTCK86C6PvVwkTPL+4TbTmiLA3tP4CMt+Kgab jyoH/nrORNmAaYnAjvRf5l8mVl/EynNaBwa9abkICOzEA/8ebGQbWKjXhfBismJQPYDm 1BSXEN1X25Iva4x0FSYj4J4+tqWfjasCc50OhKun69VMdoTNIlz20+b37bWTsTaA4Lup 4rnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=kBJ0o8emlRVNnP9LnknwNquHEepyiJr2UVW8S3DFDz4=; b=TAyk3R1F+qVN1c7hA9Zy+CRlcbrqSFWcLrOZ2ykhgA3vyydghQZWnVZTwmw7voZgzi 9dWOaYbRwR79WJC3exySx/rc2+vwYFKImLjyLf/njxPoLp7rdnqewcSfWnTR7lUVXewL ZE+EzMb7VOW4bRRONaRGH+wLLQL0m0HdUCEsiAhrBWQJQvXPEqZpF5w2eTRrUAcOiMNz oKVouTvwA2ckEhydpTBcF3jAYJ47Xj3+sj8w7RCNr+OkLyFiid1G0Gy6tfgpaNVk9eHd tgCS6OgiQEGu+0sdGLHNuMTfU7p1xkQruwZHDYpjxrmqJnUSL0XFQj8V994LZv9yFhdl dong== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xAUYy86i; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 5-20020a0562140dc500b005dda9d02288si12337570qvt.605.2023.05.16.12.44.11 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:44:11 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xAUYy86i; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yk-0006iN-K5; Tue, 16 May 2023 15:42:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yj-0006hi-Bw for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:09 -0400 Received: from mail-pf1-x42b.google.com ([2607:f8b0:4864:20::42b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yh-0002jw-Qz for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:09 -0400 Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-64ab2a37812so8109747b3a.1 for ; Tue, 16 May 2023 12:42:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266127; x=1686858127; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kBJ0o8emlRVNnP9LnknwNquHEepyiJr2UVW8S3DFDz4=; b=xAUYy86iAr1xpNnSaXXKfd3jSPyJZQhmcnFDjTgxFCCsigEPOmwiOSid9L/D7p7264 0VnGcaZmZvRfx4OCcDmOFBmriMw9P3uH6a9B1J1kNpxCQdSRxTm83d0aqffF3g76LEp9 1u6V5pWd3sKnQI0D+78wzgj3cvzLnbbfPeru5xPxrmmLei5L6YVqS/dFlR2+et3htsgD sdGwxDM9gEhkoECh/Ala9u3WFF6gu5GV3qOvavE8CyZXOedO4lujRwVC30hiJzBo23HS UeQw7VvgUCZjeolkpbM2ELcDHbwUOQtJe3K+3G2vRZnQyv42atpetHIdal+s+7ymgUPa hnxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266127; x=1686858127; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kBJ0o8emlRVNnP9LnknwNquHEepyiJr2UVW8S3DFDz4=; b=L8bLRUEvobERWiAPqPhZ3eR21LTQlgnJfXXQarrH5Kvs6whHDhKVrwVXgDSgAnQnQR FwVZeKFC7BTgU7d0tU3W3TLuNkxVYPtIRlvF+cgK+OYXa6O+5BZhXe5s3YMYWHzPtmXO WJd/wEnP8L46eucVErXbM5jpF+71aj1dDqLZi6vJVOm3MXTf5gO4clKP+g2Y3Giuszhl mII2EvK807H25JjxLLq0O68iNoSLIiM7HVSpZt5D1oL8QBubgCjTnvTgcZdE2uDR+aIh YR4R5vfE11Et0HQ/kHDive7Y3w936LWGu/iguMA8BlUApqC+RNrjURteYSHBisRd6RBG C2hQ== X-Gm-Message-State: AC+VfDzo3j2RpS7HeTFkA2txGMJRvuiiEymIQy1WnTDdOfmhfDHeCUPi DRSE5O3YFXqOPtg0zf31zOagU7G1XyBcAS7vhEw= X-Received: by 2002:a05:6a00:1a49:b0:646:24c6:5f9e with SMTP id h9-20020a056a001a4900b0064624c65f9emr35391059pfv.16.1684266127071; Tue, 16 May 2023 12:42:07 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:06 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 22/80] tcg/sparc64: Allocate %g2 as a third temporary Date: Tue, 16 May 2023 12:40:47 -0700 Message-Id: <20230516194145.1749305-23-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42b; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/sparc64/tcg-target.c.inc | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index e997db2645..64464ab363 100644 --- a/tcg/sparc64/tcg-target.c.inc +++ b/tcg/sparc64/tcg-target.c.inc @@ -83,9 +83,10 @@ static const char * const tcg_target_reg_names[TCG_TARGET_NB_REGS] = { #define ALL_GENERAL_REGS MAKE_64BIT_MASK(0, 32) #define ALL_QLDST_REGS (ALL_GENERAL_REGS & ~SOFTMMU_RESERVE_REGS) -/* Define some temporary registers. T2 is used for constant generation. */ +/* Define some temporary registers. T3 is used for constant generation. */ #define TCG_REG_T1 TCG_REG_G1 -#define TCG_REG_T2 TCG_REG_O7 +#define TCG_REG_T2 TCG_REG_G2 +#define TCG_REG_T3 TCG_REG_O7 #ifndef CONFIG_SOFTMMU # define TCG_GUEST_BASE_REG TCG_REG_I5 @@ -110,7 +111,6 @@ static const int tcg_target_reg_alloc_order[] = { TCG_REG_I4, TCG_REG_I5, - TCG_REG_G2, TCG_REG_G3, TCG_REG_G4, TCG_REG_G5, @@ -492,8 +492,8 @@ static void tcg_out_movi_int(TCGContext *s, TCGType type, TCGReg ret, static void tcg_out_movi(TCGContext *s, TCGType type, TCGReg ret, tcg_target_long arg) { - tcg_debug_assert(ret != TCG_REG_T2); - tcg_out_movi_int(s, type, ret, arg, false, TCG_REG_T2); + tcg_debug_assert(ret != TCG_REG_T3); + tcg_out_movi_int(s, type, ret, arg, false, TCG_REG_T3); } static void tcg_out_ext8s(TCGContext *s, TCGType type, TCGReg rd, TCGReg rs) @@ -885,10 +885,8 @@ static void tcg_out_jmpl_const(TCGContext *s, const tcg_insn_unit *dest, { uintptr_t desti = (uintptr_t)dest; - /* Be careful not to clobber %o7 for a tail call. */ tcg_out_movi_int(s, TCG_TYPE_PTR, TCG_REG_T1, - desti & ~0xfff, in_prologue, - tail_call ? TCG_REG_G2 : TCG_REG_O7); + desti & ~0xfff, in_prologue, TCG_REG_T2); tcg_out_arithi(s, tail_call ? TCG_REG_G0 : TCG_REG_O7, TCG_REG_T1, desti & 0xfff, JMPL); } @@ -1856,6 +1854,7 @@ static void tcg_target_init(TCGContext *s) tcg_regset_set_reg(s->reserved_regs, TCG_REG_O6); /* stack pointer */ tcg_regset_set_reg(s->reserved_regs, TCG_REG_T1); /* for internal use */ tcg_regset_set_reg(s->reserved_regs, TCG_REG_T2); /* for internal use */ + tcg_regset_set_reg(s->reserved_regs, TCG_REG_T3); /* for internal use */ } #define ELF_HOST_MACHINE EM_SPARCV9 From patchwork Tue May 16 19:40:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682434 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp23618wrt; Tue, 16 May 2023 13:05:09 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ78bwJzdZdRBEWgISKcsFiOsa54U7biLisY0mCzgXc9NS7QTOBB7o0dBxXmpDxsF6DDgvC2 X-Received: by 2002:adf:fe51:0:b0:306:42e2:5ec3 with SMTP id m17-20020adffe51000000b0030642e25ec3mr27934962wrs.6.1684267509771; Tue, 16 May 2023 13:05:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267509; cv=none; d=google.com; s=arc-20160816; b=lA1VGCxL9EZcMvRbDMvxVZHTPCehfpGyxSk/RpFAw5P6mrd9AGQwQtsiZ6+BqnkKnn 666gPFh61DE/vPO5CxGJfIIsfRjPVwhV1c4m1X007ZhJ+hBdC/7hWuuIhTBA2RPyPH6z pStIaOBCPEcVIvmFKoPBStN5zdHp/375w76wc5Z/KLRKGELj1V6T1I6dp9vvnAKm9TsF m9JQNQqUnN4Nw4d1cloGcla+4xrj3HJqZcXXMS23l2vQ7obZYrnoj0eVC6wPbzBaraJd cxHmRUpgtEMlCOZqRn94asbsvxsyAcdXVQNOY6xoJiInUUcLm92NSuS3oxKTVsN9zeud X62Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=xCSbl8AzjqW4VuvLznNno/mM1J8MuXTXHDdG/XTdZZU=; b=PuY5EaMQQRmtsbfVWB+uDw4cWWPZHfQGObSDjn5Ae1sBKkG/FWwwJXSgLPCEDTMRdG o+PHKHqf5LlXzUh3Nyd0QlJdQDKvFWsKF+LtKUHhrFh3vhF78jkCmrPsNHwSsLl/2/dP ohuz6pLF9ydXQUmr5gGQ6STPX0u0KMxnJOD0NMUzG36sUUU8vxrpQFPHCMjZBIU80MYc j1DYe2A6NXiq5F9phwZB/fvGdyqzohOBuwmHPa7FNyvLRkuSCnxIdduVb/0R7un2rmuo Lm9JkXT/fVi5w5b2IAWEjw8ZL/O0PLM8wEtT4LKaObllxS1DjSSAC6FlvwjDo1AambDg Q3ZA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=DT6ZwjWo; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l5-20020a5d6685000000b00307a83e9bdcsi213338wru.320.2023.05.16.13.05.09 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 13:05:09 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=DT6ZwjWo; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yn-0006jc-3q; Tue, 16 May 2023 15:42:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yk-0006iM-GQ for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:10 -0400 Received: from mail-pg1-x534.google.com ([2607:f8b0:4864:20::534]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yi-0002kQ-L5 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:10 -0400 Received: by mail-pg1-x534.google.com with SMTP id 41be03b00d2f7-517ab9a4a13so13454554a12.1 for ; Tue, 16 May 2023 12:42:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266128; x=1686858128; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xCSbl8AzjqW4VuvLznNno/mM1J8MuXTXHDdG/XTdZZU=; b=DT6ZwjWo5Ezs4rKy9K4/t+Uv+xYRCU2AnNNHb9yjt2TIQzeHd6FRxcxNym+cTDdrX0 W2HTZhuASzf2vOiBSzRR93IUr4D0vsw30aPYL5PU31IhGolb3bnZgloc1T2dk3a4KZ4D 05WbYMnpQzBpz9guMGWUG0QHeXJBYrK5KIsKFFkLa1cc/h8xGka/4zhz0sIs5UOd8fOE dntNlz1RhZ5rsnVNxIa2kp5pbk+1Qf3RRHkuDk/TGgLBdhbg+oKpGyIJJ/Lnw7bs1z3H CckbAyvC2ODS9DpC1z5lMxu/Udw/TJIzfIxyqenCEi6syclLriDUInmljTdAKXAsP0gx UlAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266128; x=1686858128; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xCSbl8AzjqW4VuvLznNno/mM1J8MuXTXHDdG/XTdZZU=; b=hZUOHzLtuZokpo0NzVKJlDD09FQxMueAtdJc5QlFFd7Z2SNOXzhspVkJNhlkCu6p1t 29liI49RYeI9QqTKCaoEi1EnVNOO9K+p6QLkVDWBgFYSaSJR6sBX1LNS/P7+0fz0Fkv8 oFo8pGrtGAuCSZOh3EJtNqpJKiLdsE+ct3zYsVrJ2ZsqqrghYf/8dMy+If1Gp6upgv+l j18nUZQ6VNocYClVEVld7ukGOj5bSHSj8aRxd2YjIa5OyNr9FpM3FqejsT1In0JH+1pi h1ISlEukps+9s3l9ueBg9K8iBJ2v71TA+esabCaZjXR4v6EgxOxeQ7ujdJ/RV5yw0EVP g4PA== X-Gm-Message-State: AC+VfDyvabRQplt/3GnhQ5fOlB31WjaKNaANzothbNzbz4MLPkU3v1Sl XHDy+qhEqp18wUq3b+VFFV7R7OQ0ICQus9OE9Mk= X-Received: by 2002:a05:6a20:7295:b0:f2:ae03:4020 with SMTP id o21-20020a056a20729500b000f2ae034020mr45936061pzk.54.1684266127859; Tue, 16 May 2023 12:42:07 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:07 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 23/80] tcg/sparc64: Rename tcg_out_movi_imm13 to tcg_out_movi_s13 Date: Tue, 16 May 2023 12:40:48 -0700 Message-Id: <20230516194145.1749305-24-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::534; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x534.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Emphasize that the constant is signed. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/sparc64/tcg-target.c.inc | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index 64464ab363..15d6a9fd73 100644 --- a/tcg/sparc64/tcg-target.c.inc +++ b/tcg/sparc64/tcg-target.c.inc @@ -399,7 +399,8 @@ static void tcg_out_sethi(TCGContext *s, TCGReg ret, uint32_t arg) tcg_out32(s, SETHI | INSN_RD(ret) | ((arg & 0xfffffc00) >> 10)); } -static void tcg_out_movi_imm13(TCGContext *s, TCGReg ret, int32_t arg) +/* A 13-bit constant sign-extended to 64 bits. */ +static void tcg_out_movi_s13(TCGContext *s, TCGReg ret, int32_t arg) { tcg_out_arithi(s, ret, TCG_REG_G0, arg, ARITH_OR); } @@ -408,7 +409,7 @@ static void tcg_out_movi_imm32(TCGContext *s, TCGReg ret, int32_t arg) { if (check_fit_i32(arg, 13)) { /* A 13-bit constant sign-extended to 64-bits. */ - tcg_out_movi_imm13(s, ret, arg); + tcg_out_movi_s13(s, ret, arg); } else { /* A 32-bit constant zero-extended to 64 bits. */ tcg_out_sethi(s, ret, arg); @@ -433,7 +434,7 @@ static void tcg_out_movi_int(TCGContext *s, TCGType type, TCGReg ret, /* A 13-bit constant sign-extended to 64-bits. */ if (check_fit_tl(arg, 13)) { - tcg_out_movi_imm13(s, ret, arg); + tcg_out_movi_s13(s, ret, arg); return; } @@ -767,7 +768,7 @@ static void tcg_out_setcond_i32(TCGContext *s, TCGCond cond, TCGReg ret, default: tcg_out_cmp(s, c1, c2, c2const); - tcg_out_movi_imm13(s, ret, 0); + tcg_out_movi_s13(s, ret, 0); tcg_out_movcc(s, cond, MOVCC_ICC, ret, 1, 1); return; } @@ -803,11 +804,11 @@ static void tcg_out_setcond_i64(TCGContext *s, TCGCond cond, TCGReg ret, /* For 64-bit signed comparisons vs zero, we can avoid the compare if the input does not overlap the output. */ if (c2 == 0 && !is_unsigned_cond(cond) && c1 != ret) { - tcg_out_movi_imm13(s, ret, 0); + tcg_out_movi_s13(s, ret, 0); tcg_out_movr(s, cond, ret, c1, 1, 1); } else { tcg_out_cmp(s, c1, c2, c2const); - tcg_out_movi_imm13(s, ret, 0); + tcg_out_movi_s13(s, ret, 0); tcg_out_movcc(s, cond, MOVCC_XCC, ret, 1, 1); } } @@ -844,7 +845,7 @@ static void tcg_out_addsub2_i64(TCGContext *s, TCGReg rl, TCGReg rh, if (use_vis3_instructions && !is_sub) { /* Note that ADDXC doesn't accept immediates. */ if (bhconst && bh != 0) { - tcg_out_movi_imm13(s, TCG_REG_T2, bh); + tcg_out_movi_s13(s, TCG_REG_T2, bh); bh = TCG_REG_T2; } tcg_out_arith(s, rh, ah, bh, ARITH_ADDXC); @@ -866,7 +867,7 @@ static void tcg_out_addsub2_i64(TCGContext *s, TCGReg rl, TCGReg rh, * so the adjustment fits 12 bits. */ if (bhconst) { - tcg_out_movi_imm13(s, TCG_REG_T2, bh + (is_sub ? -1 : 1)); + tcg_out_movi_s13(s, TCG_REG_T2, bh + (is_sub ? -1 : 1)); } else { tcg_out_arithi(s, TCG_REG_T2, bh, 1, is_sub ? ARITH_SUB : ARITH_ADD); @@ -1036,7 +1037,7 @@ static void tcg_target_qemu_prologue(TCGContext *s) tcg_code_gen_epilogue = tcg_splitwx_to_rx(s->code_ptr); tcg_out_arithi(s, TCG_REG_G0, TCG_REG_I7, 8, RETURN); /* delay slot */ - tcg_out_movi_imm13(s, TCG_REG_O0, 0); + tcg_out_movi_s13(s, TCG_REG_O0, 0); build_trampolines(s); } @@ -1430,7 +1431,7 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) { if (check_fit_ptr(a0, 13)) { tcg_out_arithi(s, TCG_REG_G0, TCG_REG_I7, 8, RETURN); - tcg_out_movi_imm13(s, TCG_REG_O0, a0); + tcg_out_movi_s13(s, TCG_REG_O0, a0); return; } else { intptr_t tb_diff = tcg_tbrel_diff(s, (void *)a0); From patchwork Tue May 16 19:40:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682376 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp640739ltp; Tue, 16 May 2023 12:47:50 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5lHfUlQZyqQuP3EO0OT/BiTcieW94IJwqSpl7+LvCiWPLmGHvIMRp0YnAVQPIxSQi4JNUC X-Received: by 2002:a05:6214:2462:b0:5df:47e2:8df5 with SMTP id im2-20020a056214246200b005df47e28df5mr66735929qvb.31.1684266469923; Tue, 16 May 2023 12:47:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266469; cv=none; d=google.com; s=arc-20160816; b=enMnfJSHvNkHBXUSW6n9usUsNB8rP64C2v42H5qRmfun4q4xn6mRAGwzyEVwaaDFQf rjXsLK3867hr7J/CdUt/i7IflkAZQRH4ujyRhazJs/N3AWDMwJwufkMEwPW/15t7pnyj n9y4BYdhXqdJdu+aNbPs7ZFPPPHq0I1sxgmwPaOlXec3yvWIEqxkv+7oI5DxDSE8wu1k ku84ynjQ4DJH1UT5FxZGkbHuVu9BkMRdrb/YAb3IHzs5P3fWZzLVLCE5H4ccJfyieu4F aMqN7tOpKZDTfBftWX6g4yf6glgWlSmYm05uzJEXCmDV06KhjO/ZG7yW4tH7pez/uwXq jXKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=03Saz/Ae1+wLFRqHrvpgcnIMM8UibULuCXSb3tQDXYE=; b=upD0ovlvJ0tgy/wMRyi8kF+GWw8yMBimXMp6rUwQjHktCdXBbSOimoP5XxBVPe0EW7 e6O8E5iJ3PggtVcZfX1CONWRA/7O3kLFfwPQmrlPyO4XmvC9CfxkceApPEjILWThSTUc DzfUXmTEBPRs+eRXVrAkf/QTKfa/ofbAbSbTru3LO07fqgt6Xh5GbKV4+hth4wQQ7Hvw cV/i7rRJxrfDYbA4fiTftmq2X1UpIaj27l7b7Hsuam+o28p4zr9VexgSrDMIVLi3WDMD JdRLstfKNUVWYrdkBFJOeY9yZDXSx25fw/NTbi6RoTQ6sQvxYS9Yd9l2t4MdudJ2cu7k yUHQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qtT+Pdeq; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m19-20020ae9e713000000b007578ee2138csi207771qka.380.2023.05.16.12.47.49 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:47:49 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qtT+Pdeq; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yo-0006jf-6J; Tue, 16 May 2023 15:42:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yl-0006il-CH for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:11 -0400 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yj-0002rN-R7 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:11 -0400 Received: by mail-pg1-x530.google.com with SMTP id 41be03b00d2f7-52079a12451so10504194a12.3 for ; Tue, 16 May 2023 12:42:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266128; x=1686858128; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=03Saz/Ae1+wLFRqHrvpgcnIMM8UibULuCXSb3tQDXYE=; b=qtT+PdeqQZ8Q4G7WB6SKOklaEmeHqzjTfL4HvBwgX/RJKSS0Z8Fc3CjqB3mV1wpL4z aVxUBck0eZ4uvvuKkT96k2s4xYngTXJce2hRQHZKbd4wZRjqOCg7BAXscHCSCfHuh72w BO4XcZxBnV+rJZgNOkD1FkHtAYYQdnTP9Mz2Pf1nVgg8V3WZhOeuURkXonheQ1VPbjm4 KlG0c+vadRegMVtIdRB1ixAAavlq62OMABsgBNj7uUgkD0WiDeGL4HBMEn4a7vit+y3w 6V2075D8nUlxCMdvAWU/c8FB0BkzDWT7zjVcPsEAco5L//IOhyvXng/37zGgieDtF+MN 7T1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266128; x=1686858128; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=03Saz/Ae1+wLFRqHrvpgcnIMM8UibULuCXSb3tQDXYE=; b=bbg9SHQmyx/GZcWD1CHD7L9PLHw91ydKt01sHfsEXSjKSNxwnNugl+n6FYvP4Aquj2 ADgMVNPomYTpp3B4FjcjE8tmcWrp2lwOqoXBcckWESJYt1jWv2fYjkUEisgsSXAhz5P0 SkMChtgxbRqeUfGfUW57+n227FaLn64XTGXw9us6o13rZNsr6U9EydGpkGxWfXvKDiKm MscI+P3W6N3/UBS3oXQFERb13M6KAUKj+Nz0ObL2buGcixjN4aczu6Z90L5MOWlje4e4 lOxglBzwItkd4c1PTmMkcTy9V8KUtZIcIQICWpJ7bJu88nELpO/NWuWJXtfGWZK/WROI Q7Pg== X-Gm-Message-State: AC+VfDzmnhrvbDJns8FHeg1F78mgLkN4Wa5j0ZkSZvEFu56b8sTS7Zm6 yMs3n5+rXY/Ydx81W4PoB6ktRgMD4PAJMn5XNww= X-Received: by 2002:a05:6a20:7354:b0:101:1d1d:43a4 with SMTP id v20-20020a056a20735400b001011d1d43a4mr34273036pzc.15.1684266128632; Tue, 16 May 2023 12:42:08 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:08 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 24/80] target/sparc64: Remove tcg_out_movi_s13 case from tcg_out_movi_imm32 Date: Tue, 16 May 2023 12:40:49 -0700 Message-Id: <20230516194145.1749305-25-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::530; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x530.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Shuffle the order in tcg_out_movi_int to check s13 first, and drop this check from tcg_out_movi_imm32. This might make the sequence for in_prologue larger, but not worth worrying about. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/sparc64/tcg-target.c.inc | 25 ++++++++++--------------- 1 file changed, 10 insertions(+), 15 deletions(-) diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index 15d6a9fd73..2689599fd6 100644 --- a/tcg/sparc64/tcg-target.c.inc +++ b/tcg/sparc64/tcg-target.c.inc @@ -407,15 +407,10 @@ static void tcg_out_movi_s13(TCGContext *s, TCGReg ret, int32_t arg) static void tcg_out_movi_imm32(TCGContext *s, TCGReg ret, int32_t arg) { - if (check_fit_i32(arg, 13)) { - /* A 13-bit constant sign-extended to 64-bits. */ - tcg_out_movi_s13(s, ret, arg); - } else { - /* A 32-bit constant zero-extended to 64 bits. */ - tcg_out_sethi(s, ret, arg); - if (arg & 0x3ff) { - tcg_out_arithi(s, ret, ret, arg & 0x3ff, ARITH_OR); - } + /* A 32-bit constant zero-extended to 64 bits. */ + tcg_out_sethi(s, ret, arg); + if (arg & 0x3ff) { + tcg_out_arithi(s, ret, ret, arg & 0x3ff, ARITH_OR); } } @@ -426,18 +421,18 @@ static void tcg_out_movi_int(TCGContext *s, TCGType type, TCGReg ret, tcg_target_long hi, lo = (int32_t)arg; tcg_target_long test, lsb; - /* A 32-bit constant, or 32-bit zero-extended to 64-bits. */ - if (type == TCG_TYPE_I32 || arg == (uint32_t)arg) { - tcg_out_movi_imm32(s, ret, arg); - return; - } - /* A 13-bit constant sign-extended to 64-bits. */ if (check_fit_tl(arg, 13)) { tcg_out_movi_s13(s, ret, arg); return; } + /* A 32-bit constant, or 32-bit zero-extended to 64-bits. */ + if (type == TCG_TYPE_I32 || arg == (uint32_t)arg) { + tcg_out_movi_imm32(s, ret, arg); + return; + } + /* A 13-bit constant relative to the TB. */ if (!in_prologue) { test = tcg_tbrel_diff(s, (void *)arg); From patchwork Tue May 16 19:40:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682358 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp638786ltp; Tue, 16 May 2023 12:42:57 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4E62tWcavyl71YjVnDrNfqD+vW+nChT4SQKNsGwdYakA0m4cJ3WKRRhw+Kbv0IQ1++S4Ch X-Received: by 2002:a05:622a:8e:b0:3e4:d1c0:36a9 with SMTP id o14-20020a05622a008e00b003e4d1c036a9mr61718994qtw.48.1684266177327; Tue, 16 May 2023 12:42:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266177; cv=none; d=google.com; s=arc-20160816; b=Uv2jlT9EdoQxm7wEUlrnObFBIAFFHj9QvtzOVaaiSSRItFEnGlhysDMBPB2TGCWoNK My00oyns3B/GBqklC3zlUmDJhrQ4vwT8oCr0Jrhj8YVgjkQvor2/0vyNkMszjEzQ103R 3AHkXNIc4BAq75LffKjfv58gLm1p/9Qq2zhxdRWHFIRwwQSuJ0BYEjfkNhRfu//kD5+J UnR87PXyEmR1vInG0gOx1g1z9o3K1W6tmxekQChyqs2G83chiNW7b5VDfHM1OV7JxDHL sR7jJuOpllf1WeO94RFKiADExDYOjI7aLVlotbDxSeCVyDZajbGJsZ9O/aHp45BOmGpr IgLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=XQUAkxU919ge4RGYSTop1y6/iu8bQJHk3fDtn7Q1o1I=; b=L8rkceUBE/81IcpXiw/3Tbr0tU3PMyHg6AOvMd+OQQdNyI0NrRTJ10IT8fQPBa+i2z 3tEaaNrmYyQDso96reFdOyAmqYZrbohSwVoBgtJfjMIR+igiupXFmS1JI814xgYBCF4e pvAO/j91Dy7GX4Sq7yn42GgQ96K5SFPDGqNbGfJibJ+oT8XM6uoBAh/Z0AhVSbFZcBBY mun6RSX4un1snh9eU1cspd2mq7P7ys9QHkMzKbbTqu8L5JfQXQTbjzaipmnLKMxtwBQd wsnf3gqxtykiAUFKSyb7QMIRJOPDRxgG5LVRAcCVnIPl1M7OX+d4Cb+W06NPaMm7XOG6 POYQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hroZJ+bF; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id o11-20020a05622a044b00b003f56af9d547si333826qtx.616.2023.05.16.12.42.57 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:42:57 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hroZJ+bF; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yp-0006jq-12; Tue, 16 May 2023 15:42:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Ym-0006it-8R for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:12 -0400 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yk-0002rh-Pv for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:12 -0400 Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-643b7b8f8ceso8505297b3a.1 for ; Tue, 16 May 2023 12:42:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266129; x=1686858129; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XQUAkxU919ge4RGYSTop1y6/iu8bQJHk3fDtn7Q1o1I=; b=hroZJ+bFaghd8BJA3lzEDbSM74IXlbqDpdkbpZCNyhCIcheowu2G00stjncumD6/s0 CgymDeIHU/6m+Pja9F8djsFE5RBNZHDeCABjSElUsPrMgk+i3Mu3+TY0vUBxcV0DN88T DciiGWTFoUJqSrodZxemM+8wXdy4mpo2NVgcic1WMsfG4U9CLvO5lldprdIfbxaKwVAG gtV1FwsZRN4Q5SgEM0ztTdxvVkEVj2eS83W4og1yoOPuoUc59+Ecuh0EnAj1KfPHxF5c dSYBj2IMSH4yWj1ySiRjricq1YSnM2k5VvEETNZE64DoJMqBpAuFgjHD5Ctu9WVIcxDV L98w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266129; x=1686858129; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XQUAkxU919ge4RGYSTop1y6/iu8bQJHk3fDtn7Q1o1I=; b=CUs7yj/MsGUtvids3biXek4M9jkTW9LI2Sq3mrsv83qCZynX+GZYm+tb70NljwuzIw V1EMAx+R/O9Kx03+Lb+10BEbTpggz7Rh8MP1SdkFuF9lUJBxlDucyy8FfwFwxg5TTMOo GBMD+FE5KKQUuXqq3GQIN0DU0EQxfQaKCpRSeIkM2dFEJMov35itRzahCEJzmihV+Ebp VmV7bqr99WpFTonq5mJwBcV5U2d4PEYR4lqXXPzB5nVFnca0aAkIZ+aByx6PbA9VZaDC BD1m7GRdQZjZY26OXOPUeksCfplwvOJ7tenlBsPXTGOJBRmMb5KzqAifqBBtlCeo3TV9 7XiA== X-Gm-Message-State: AC+VfDwb+cVfp8VHKhoAeMzhgxga59qkHpmaE3j408SJysEpoOKlAGvR Jy5H+x6xjU1WMApoYuP7bpWfPYku8mWhLOxvsNs= X-Received: by 2002:a05:6a00:c82:b0:64a:a1ba:50fd with SMTP id a2-20020a056a000c8200b0064aa1ba50fdmr21449002pfv.22.1684266129518; Tue, 16 May 2023 12:42:09 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:09 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 25/80] tcg/sparc64: Rename tcg_out_movi_imm32 to tcg_out_movi_u32 Date: Tue, 16 May 2023 12:40:50 -0700 Message-Id: <20230516194145.1749305-26-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::432; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x432.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Emphasize that the constant is unsigned. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/sparc64/tcg-target.c.inc | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index 2689599fd6..e244209890 100644 --- a/tcg/sparc64/tcg-target.c.inc +++ b/tcg/sparc64/tcg-target.c.inc @@ -405,9 +405,9 @@ static void tcg_out_movi_s13(TCGContext *s, TCGReg ret, int32_t arg) tcg_out_arithi(s, ret, TCG_REG_G0, arg, ARITH_OR); } -static void tcg_out_movi_imm32(TCGContext *s, TCGReg ret, int32_t arg) +/* A 32-bit constant zero-extended to 64 bits. */ +static void tcg_out_movi_u32(TCGContext *s, TCGReg ret, uint32_t arg) { - /* A 32-bit constant zero-extended to 64 bits. */ tcg_out_sethi(s, ret, arg); if (arg & 0x3ff) { tcg_out_arithi(s, ret, ret, arg & 0x3ff, ARITH_OR); @@ -429,7 +429,7 @@ static void tcg_out_movi_int(TCGContext *s, TCGType type, TCGReg ret, /* A 32-bit constant, or 32-bit zero-extended to 64-bits. */ if (type == TCG_TYPE_I32 || arg == (uint32_t)arg) { - tcg_out_movi_imm32(s, ret, arg); + tcg_out_movi_u32(s, ret, arg); return; } @@ -473,13 +473,13 @@ static void tcg_out_movi_int(TCGContext *s, TCGType type, TCGReg ret, /* A 64-bit constant decomposed into 2 32-bit pieces. */ if (check_fit_i32(lo, 13)) { hi = (arg - lo) >> 32; - tcg_out_movi_imm32(s, ret, hi); + tcg_out_movi_u32(s, ret, hi); tcg_out_arithi(s, ret, ret, 32, SHIFT_SLLX); tcg_out_arithi(s, ret, ret, lo, ARITH_ADD); } else { hi = arg >> 32; - tcg_out_movi_imm32(s, ret, hi); - tcg_out_movi_imm32(s, scratch, lo); + tcg_out_movi_u32(s, ret, hi); + tcg_out_movi_u32(s, scratch, lo); tcg_out_arithi(s, ret, ret, 32, SHIFT_SLLX); tcg_out_arith(s, ret, ret, scratch, ARITH_OR); } From patchwork Tue May 16 19:40:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682431 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp21495wrt; Tue, 16 May 2023 13:00:34 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7A6YNibH54Pv53akRGAOWXvsdQvSrISxcB3lxCE2+E5uhnhk8nIJmb3+KTFCKAXeHCKgRK X-Received: by 2002:adf:fe45:0:b0:307:cf55:a7d8 with SMTP id m5-20020adffe45000000b00307cf55a7d8mr14808141wrs.42.1684267234324; Tue, 16 May 2023 13:00:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267234; cv=none; d=google.com; s=arc-20160816; b=fPVVDaNNp1wYdhA7TSP1n8eDBmNuB5Fww+QHDA55X6ydDDSNqcaUuMN5ms0tC0ksax WaNlOAzdhICxokLU3/A4z+EZE5AvpVJubkrhPxpTlBonFVlMm4ogCqs8bRRggaAdOOL4 J8mRMoBILPudN/Mr1ktrHG7oE41t/9i+eNEOVptlNejBTZCWrl7rXHJKjJRmhH1c8CSM J+BGhLk6/jELo7WNxH1HUgePnivjn7QsIqZab7OVfdg6PGzzDnFjNW2pUHTi6gfwFMx7 23pfWQ+8ZnJxF8GjCYhDtQX/qv0tOSC78DqpoJ/r9czlvhM0MC+d3MQhZxys+nusC5nn BuzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=5+dVuSBLTF8ePfEeAWZlWoNLu6H+GdqE/W0YcabdOx4=; b=Tk9GBoR/sf6WxmJqxcYOgViE+mve+uX7SQ7DKXLDCl4STnUl0cB4LKThg6MpjjCVoi kwWgj9wPcMoiP9igkuLVxlJey+oXkhdyfwUhyQwA7o8FmauWRWgJC7+MmN8cAVdCmi+h RnRrle8jx46sFOj4LW2yqEyQTBAcE2RELE4U9cpsv8NCdvC+oaVNuIym/UtJRRgru5Lh fuye1k6vIloPrOgMSvHLhfrq2vgL67URiHikH3yfDqDyWf+qdGBQMrL7Z7Hf1McqRL+g 4Ne+UGi1F1gYRVCT9aHi2l37TxBqQCK3tQoeHsMfiq1yA2HPJecAUL/Ixz12Wih8cv92 i7Ag== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=a1bs5mhY; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x4-20020a5d54c4000000b003062e049da5si160961wrv.825.2023.05.16.13.00.34 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 13:00:34 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=a1bs5mhY; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yn-0006je-KH; Tue, 16 May 2023 15:42:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Ym-0006j5-Lj for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:12 -0400 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yl-0002jW-5O for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:12 -0400 Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-643a6f993a7so9345452b3a.1 for ; Tue, 16 May 2023 12:42:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266130; x=1686858130; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5+dVuSBLTF8ePfEeAWZlWoNLu6H+GdqE/W0YcabdOx4=; b=a1bs5mhY0ROnk5///ulxqVXpro91VUTPz/6pBcL4Geuf4cjiJrBHDv3PyUfuzGcCWU SyvZDJ1EGhiVWYfc5WuvpPLw2Z+NKQyl0Mr/sv/MBr9x8FBexTqcr+Kz/BmNn54esiZN yDjTLsx3SG1AEvcFTqNifja3bskPV/wzbL1dosVK36DJDRmfTBMvgXgucGflG2yLe4Mp hODzD0r51r9VRT8XXTm2YWf1i0KuRR1r/t1eE/saHO5xbfFIMsWiQvNY9Cbr5CA3c1ZO 69m2YtsGew5qHSmHW4wVHOVoP6Nu0u85cltAGCmVEQ/nd5lZE7LyKwy90jW6IcL22ePf ibjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266130; x=1686858130; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5+dVuSBLTF8ePfEeAWZlWoNLu6H+GdqE/W0YcabdOx4=; b=YOrPV9rjvLLsCnZ3/oBmiPd8EQpiIzgK5sRDdlDDojaHskpuWDjDaeXVuZARxfVoR3 zdnAuWO3abbOFsjW/UoCVzgQO7Klpb5HuKD/8vZpnk1iH0vEjQD4E0/69KPD8AXubd4y /1IWWJozGkZQmHZbNVcn7Bq39BGxbMV0WkK1Cv+NFoixoIvzezy0Xpmw1uRhx86mZYoR +/isJwHDKfhAJ6hyHAPETr1Os8t9UMAFx9FEFbKxQrYEZvxQKrIoJwFcvptLV0qkFNnC PuEQLwMLibIUEv4DpKNAGNefOaVDKQNJHY/5G6UehN6L1DUvLJ0nWUH4NK+2FZIKndsv IEuw== X-Gm-Message-State: AC+VfDzRgMo2ms8y839T35G6PKfCI6xuAIuvxytHeWBFhcr16SNpstqQ YOVLrCoP2DeH+Tra/+IPUSzancEb6cGBOC4112M= X-Received: by 2002:a05:6a00:10d2:b0:643:96bc:b292 with SMTP id d18-20020a056a0010d200b0064396bcb292mr41983348pfu.5.1684266130414; Tue, 16 May 2023 12:42:10 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:10 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 26/80] tcg/sparc64: Split out tcg_out_movi_s32 Date: Tue, 16 May 2023 12:40:51 -0700 Message-Id: <20230516194145.1749305-27-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::432; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x432.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/sparc64/tcg-target.c.inc | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index e244209890..4375a06377 100644 --- a/tcg/sparc64/tcg-target.c.inc +++ b/tcg/sparc64/tcg-target.c.inc @@ -405,6 +405,13 @@ static void tcg_out_movi_s13(TCGContext *s, TCGReg ret, int32_t arg) tcg_out_arithi(s, ret, TCG_REG_G0, arg, ARITH_OR); } +/* A 32-bit constant sign-extended to 64 bits. */ +static void tcg_out_movi_s32(TCGContext *s, TCGReg ret, int32_t arg) +{ + tcg_out_sethi(s, ret, ~arg); + tcg_out_arithi(s, ret, ret, (arg & 0x3ff) | -0x400, ARITH_XOR); +} + /* A 32-bit constant zero-extended to 64 bits. */ static void tcg_out_movi_u32(TCGContext *s, TCGReg ret, uint32_t arg) { @@ -444,8 +451,7 @@ static void tcg_out_movi_int(TCGContext *s, TCGType type, TCGReg ret, /* A 32-bit constant sign-extended to 64-bits. */ if (arg == lo) { - tcg_out_sethi(s, ret, ~arg); - tcg_out_arithi(s, ret, ret, (arg & 0x3ff) | -0x400, ARITH_XOR); + tcg_out_movi_s32(s, ret, arg); return; } From patchwork Tue May 16 19:40:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682374 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp640265ltp; Tue, 16 May 2023 12:46:38 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ476HQmQnm19PE3/ytrqlsIb+kERN+uGki/SDdMyyIpUurknRQV+kWO+nld1k5wVb5aAH/F X-Received: by 2002:a05:6214:246a:b0:61b:17bd:c5fc with SMTP id im10-20020a056214246a00b0061b17bdc5fcmr65606821qvb.12.1684266398651; Tue, 16 May 2023 12:46:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266398; cv=none; d=google.com; s=arc-20160816; b=O5kbqE9MnN6vF93BN3tA2NGquzYdhtOQsjo78nYv2SBouUyRh+0FL6Gc51WcxJiyu9 dB82PuLyRVo4JHd61aoK7BYAjH1e5w+w+gwt2Nc6KwGJOQpvvSiLuRlEQXETXkg+rRP7 4tjKJrjLLWjEZ25j262OlOFwUU7Pwl2BTcGkTzSO3/Sc41RWGZVu3lPM9tRe9R7NnEia 1Kq3KQkzLaNpFWUCfl9gnlw8037zXcwgljS3dS75BVQUVwn0P+FV/lebFB1cz++zJl2G wsLP/xFCdm/9g0v/SGa0FQ8vtLdMMfMPJ/qIALuDysOJvMOT1BP0EV+CeWfKAOFH5XBe DQaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=nCBxsKdmU0HAiDvF2cveqxGr+sx47E3CobY3Y5DYN2E=; b=UEabGRQoVSFO+NBw0vallV8WWjtFrSCptOlAZ+4znvaYO23qfhDLn9gX1+OV11KJiR riAx1ReeoOIYDXiYZ0gpvoY7aAllYMsBOCOxbPMOOLSWs1ynnDLoAdRsQh0Qxz9aC/fm 5h6zumGX89ncEqdIMlaABuTkaBGjGjJYjjRu2OZhbiqdSI6lBPS+RuhX8C+MXRz8t4Vg JAGqC1BwLDdQd4IahvKpIUKc+UJCBk49HNUw8TjP3ut/YuCe4cPZDl31MDOar+FHN8iu yqLNzJmy4Yoxeb6j0mruGgakOw+r2pzd50sz8LI5TcC4zLN4nziS8Iz+LCjGzouKt38x VkbA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=yrNk7SbJ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id gg3-20020a056214252300b006211b7352c1si12645339qvb.588.2023.05.16.12.46.38 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:46:38 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=yrNk7SbJ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yq-0006lB-JX; Tue, 16 May 2023 15:42:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yp-0006kS-OW for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:15 -0400 Received: from mail-pg1-x531.google.com ([2607:f8b0:4864:20::531]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Ym-0002sF-Ri for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:15 -0400 Received: by mail-pg1-x531.google.com with SMTP id 41be03b00d2f7-52cbd7d0c37so8622031a12.3 for ; Tue, 16 May 2023 12:42:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266131; x=1686858131; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nCBxsKdmU0HAiDvF2cveqxGr+sx47E3CobY3Y5DYN2E=; b=yrNk7SbJ79YIBflVXwCqnaJQtbfG/TYRl5nvBU22NUzwIQidYYQ4XaG9udqzI8oUja O7NmjfZIPqZAFdrA2wAzmAC9IrN2ZILqCe3uAsA31StwCD/jNH27slAkEPuEiOVyn/47 NN+jsQND1iI0XD6cL55XagVPPC72hdvPAggV1j5BBrWsuYIkaQYkoZ+0q6RRcCsGhyG5 Ru6q9PoKJ1vpY7l4cW1PGNqujtaf/wL8qtSZRwg3EW7Ny/1N6dT5q0ypj1OJD4t9nil+ uzw3ALYZyE2OLEq2ZJ7DLlOnsr2aBJUWuoh4TsCChnvUinrYnEsAo87FKPHWWb+JnGfp 7qyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266131; x=1686858131; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nCBxsKdmU0HAiDvF2cveqxGr+sx47E3CobY3Y5DYN2E=; b=MvxKwbjqwK9JQV8A/EogrlRCOX+TVWHyYGGJivtcGTKJX/QnPXNcqJHUgDwatkIvy7 tsGwrsz7rRyCh3HaGg6XYSviniSlCgt7+sal72knH23SpjuX/0UAnmAU+exvEGABZd/v maGugjgZMweJ1gX5Y09zkwE1GuI8e+/s08EJhLMU3Jm25eBK3IQHIYfhmlSh0kuiSIoP dt8XOlPS9ODIWuaA/s+VCIL8nGzPcQ8HUrjEEXJJ7G+FK5tmdwMqFLs2dVmFOeGg7Y8+ Uka4TWrt3ycOTryPGnRVP/6xxi0oH3NGpvtj4wqwTRf63GliTPLglcVbRdJmLnNtcu5M gUVQ== X-Gm-Message-State: AC+VfDwj6pxbEv6/GVJ6O8aP7LPMNW3OkavQfYEbJQ7JWkO25sQZ7vd/ PFEwOW41ri3gORrEvJ+KKL8bAEh/1zXklVIDzwU= X-Received: by 2002:a05:6a21:339f:b0:101:6f60:120f with SMTP id yy31-20020a056a21339f00b001016f60120fmr33567629pzb.39.1684266131289; Tue, 16 May 2023 12:42:11 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:10 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 27/80] tcg/sparc64: Use standard slow path for softmmu Date: Tue, 16 May 2023 12:40:52 -0700 Message-Id: <20230516194145.1749305-28-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::531; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x531.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Drop the target-specific trampolines for the standard slow path. This lets us use tcg_out_helper_{ld,st}_args, and handles the new atomicity bits within MemOp. At the same time, use the full load/store helpers for user-only mode. Drop inline unaligned access support for user-only mode, as it does not handle atomicity. Use TCG_REG_T[1-3] in the tlb lookup, instead of TCG_REG_O[0-2]. This allows the constraints to be simplified. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/sparc64/tcg-target-con-set.h | 2 - tcg/sparc64/tcg-target-con-str.h | 1 - tcg/sparc64/tcg-target.h | 1 + tcg/sparc64/tcg-target.c.inc | 610 +++++++++---------------------- 4 files changed, 182 insertions(+), 432 deletions(-) diff --git a/tcg/sparc64/tcg-target-con-set.h b/tcg/sparc64/tcg-target-con-set.h index 31e6fea1fc..434bf25072 100644 --- a/tcg/sparc64/tcg-target-con-set.h +++ b/tcg/sparc64/tcg-target-con-set.h @@ -12,8 +12,6 @@ C_O0_I1(r) C_O0_I2(rZ, r) C_O0_I2(rZ, rJ) -C_O0_I2(sZ, s) -C_O1_I1(r, s) C_O1_I1(r, r) C_O1_I2(r, r, r) C_O1_I2(r, rZ, rJ) diff --git a/tcg/sparc64/tcg-target-con-str.h b/tcg/sparc64/tcg-target-con-str.h index 8f5c7aef97..0577ec4942 100644 --- a/tcg/sparc64/tcg-target-con-str.h +++ b/tcg/sparc64/tcg-target-con-str.h @@ -9,7 +9,6 @@ * REGS(letter, register_mask) */ REGS('r', ALL_GENERAL_REGS) -REGS('s', ALL_QLDST_REGS) /* * Define constraint letters for constants: diff --git a/tcg/sparc64/tcg-target.h b/tcg/sparc64/tcg-target.h index ffe22b1d21..7434cc99d4 100644 --- a/tcg/sparc64/tcg-target.h +++ b/tcg/sparc64/tcg-target.h @@ -155,6 +155,7 @@ extern bool use_vis3_instructions; #define TCG_TARGET_DEFAULT_MO (0) #define TCG_TARGET_HAS_MEMORY_BSWAP 1 +#define TCG_TARGET_NEED_LDST_LABELS #define TCG_TARGET_NEED_POOL_LABELS #endif diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index 4375a06377..0237188d65 100644 --- a/tcg/sparc64/tcg-target.c.inc +++ b/tcg/sparc64/tcg-target.c.inc @@ -27,6 +27,7 @@ #error "unsupported code generation mode" #endif +#include "../tcg-ldst.c.inc" #include "../tcg-pool.c.inc" #ifdef CONFIG_DEBUG_TCG @@ -70,18 +71,7 @@ static const char * const tcg_target_reg_names[TCG_TARGET_NB_REGS] = { #define TCG_CT_CONST_S13 0x200 #define TCG_CT_CONST_ZERO 0x400 -/* - * For softmmu, we need to avoid conflicts with the first 3 - * argument registers to perform the tlb lookup, and to call - * the helper function. - */ -#ifdef CONFIG_SOFTMMU -#define SOFTMMU_RESERVE_REGS MAKE_64BIT_MASK(TCG_REG_O0, 3) -#else -#define SOFTMMU_RESERVE_REGS 0 -#endif -#define ALL_GENERAL_REGS MAKE_64BIT_MASK(0, 32) -#define ALL_QLDST_REGS (ALL_GENERAL_REGS & ~SOFTMMU_RESERVE_REGS) +#define ALL_GENERAL_REGS MAKE_64BIT_MASK(0, 32) /* Define some temporary registers. T3 is used for constant generation. */ #define TCG_REG_T1 TCG_REG_G1 @@ -918,82 +908,6 @@ static void tcg_out_mb(TCGContext *s, TCGArg a0) tcg_out32(s, MEMBAR | (a0 & TCG_MO_ALL)); } -#ifdef CONFIG_SOFTMMU -static const tcg_insn_unit *qemu_ld_trampoline[MO_SSIZE + 1]; -static const tcg_insn_unit *qemu_st_trampoline[MO_SIZE + 1]; - -static void build_trampolines(TCGContext *s) -{ - int i; - - for (i = 0; i < ARRAY_SIZE(qemu_ld_helpers); ++i) { - if (qemu_ld_helpers[i] == NULL) { - continue; - } - - /* May as well align the trampoline. */ - while ((uintptr_t)s->code_ptr & 15) { - tcg_out_nop(s); - } - qemu_ld_trampoline[i] = tcg_splitwx_to_rx(s->code_ptr); - - /* Set the retaddr operand. */ - tcg_out_mov(s, TCG_TYPE_PTR, TCG_REG_O3, TCG_REG_O7); - /* Tail call. */ - tcg_out_jmpl_const(s, qemu_ld_helpers[i], true, true); - /* delay slot -- set the env argument */ - tcg_out_mov_delay(s, TCG_REG_O0, TCG_AREG0); - } - - for (i = 0; i < ARRAY_SIZE(qemu_st_helpers); ++i) { - if (qemu_st_helpers[i] == NULL) { - continue; - } - - /* May as well align the trampoline. */ - while ((uintptr_t)s->code_ptr & 15) { - tcg_out_nop(s); - } - qemu_st_trampoline[i] = tcg_splitwx_to_rx(s->code_ptr); - - /* Set the retaddr operand. */ - tcg_out_mov(s, TCG_TYPE_PTR, TCG_REG_O4, TCG_REG_O7); - - /* Tail call. */ - tcg_out_jmpl_const(s, qemu_st_helpers[i], true, true); - /* delay slot -- set the env argument */ - tcg_out_mov_delay(s, TCG_REG_O0, TCG_AREG0); - } -} -#else -static const tcg_insn_unit *qemu_unalign_ld_trampoline; -static const tcg_insn_unit *qemu_unalign_st_trampoline; - -static void build_trampolines(TCGContext *s) -{ - for (int ld = 0; ld < 2; ++ld) { - void *helper; - - while ((uintptr_t)s->code_ptr & 15) { - tcg_out_nop(s); - } - - if (ld) { - helper = helper_unaligned_ld; - qemu_unalign_ld_trampoline = tcg_splitwx_to_rx(s->code_ptr); - } else { - helper = helper_unaligned_st; - qemu_unalign_st_trampoline = tcg_splitwx_to_rx(s->code_ptr); - } - - /* Tail call. */ - tcg_out_jmpl_const(s, helper, true, true); - /* delay slot -- set the env argument */ - tcg_out_mov_delay(s, TCG_REG_O0, TCG_AREG0); - } -} -#endif - /* Generate global QEMU prologue and epilogue code */ static void tcg_target_qemu_prologue(TCGContext *s) { @@ -1039,8 +953,6 @@ static void tcg_target_qemu_prologue(TCGContext *s) tcg_out_arithi(s, TCG_REG_G0, TCG_REG_I7, 8, RETURN); /* delay slot */ tcg_out_movi_s13(s, TCG_REG_O0, 0); - - build_trampolines(s); } static void tcg_out_nop_fill(tcg_insn_unit *p, int count) @@ -1051,381 +963,224 @@ static void tcg_out_nop_fill(tcg_insn_unit *p, int count) } } -#if defined(CONFIG_SOFTMMU) +static const TCGLdstHelperParam ldst_helper_param = { + .ntmp = 1, .tmp = { TCG_REG_T1 } +}; -/* We expect to use a 13-bit negative offset from ENV. */ -QEMU_BUILD_BUG_ON(TLB_MASK_TABLE_OFS(0) > 0); -QEMU_BUILD_BUG_ON(TLB_MASK_TABLE_OFS(0) < -(1 << 12)); - -/* Perform the TLB load and compare. - - Inputs: - ADDRLO and ADDRHI contain the possible two parts of the address. - - MEM_INDEX and S_BITS are the memory context and log2 size of the load. - - WHICH is the offset into the CPUTLBEntry structure of the slot to read. - This should be offsetof addr_read or addr_write. - - The result of the TLB comparison is in %[ix]cc. The sanitized address - is in the returned register, maybe %o0. The TLB addend is in %o1. */ - -static TCGReg tcg_out_tlb_load(TCGContext *s, TCGReg addr, int mem_index, - MemOp opc, int which) +static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) { + MemOp opc = get_memop(lb->oi); + MemOp sgn; + + if (!patch_reloc(lb->label_ptr[0], R_SPARC_WDISP19, + (intptr_t)tcg_splitwx_to_rx(s->code_ptr), 0)) { + return false; + } + + /* Use inline tcg_out_ext32s; otherwise let the helper sign-extend. */ + sgn = (opc & MO_SIZE) < MO_32 ? MO_SIGN : 0; + + tcg_out_ld_helper_args(s, lb, &ldst_helper_param); + tcg_out_call(s, qemu_ld_helpers[opc & (MO_SIZE | sgn)], NULL); + tcg_out_ld_helper_ret(s, lb, sgn, &ldst_helper_param); + + tcg_out_bpcc0(s, COND_A, BPCC_A | BPCC_PT, 0); + return patch_reloc(s->code_ptr - 1, R_SPARC_WDISP19, + (intptr_t)lb->raddr, 0); +} + +static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) +{ + MemOp opc = get_memop(lb->oi); + + if (!patch_reloc(lb->label_ptr[0], R_SPARC_WDISP19, + (intptr_t)tcg_splitwx_to_rx(s->code_ptr), 0)) { + return false; + } + + tcg_out_st_helper_args(s, lb, &ldst_helper_param); + tcg_out_call(s, qemu_st_helpers[opc & MO_SIZE], NULL); + + tcg_out_bpcc0(s, COND_A, BPCC_A | BPCC_PT, 0); + return patch_reloc(s->code_ptr - 1, R_SPARC_WDISP19, + (intptr_t)lb->raddr, 0); +} + +typedef struct { + TCGReg base; + TCGReg index; +} HostAddress; + +/* + * For softmmu, perform the TLB load and compare. + * For useronly, perform any required alignment tests. + * In both cases, return a TCGLabelQemuLdst structure if the slow path + * is required and fill in @h with the host address for the fast path. + */ +static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, + TCGReg addr_reg, MemOpIdx oi, + bool is_ld) +{ + TCGLabelQemuLdst *ldst = NULL; + MemOp opc = get_memop(oi); + unsigned a_bits = get_alignment_bits(opc); + unsigned s_bits = opc & MO_SIZE; + unsigned a_mask; + + /* We don't support unaligned accesses. */ + a_bits = MAX(a_bits, s_bits); + a_mask = (1u << a_bits) - 1; + +#ifdef CONFIG_SOFTMMU + int mem_index = get_mmuidx(oi); int fast_off = TLB_MASK_TABLE_OFS(mem_index); int mask_off = fast_off + offsetof(CPUTLBDescFast, mask); int table_off = fast_off + offsetof(CPUTLBDescFast, table); - const TCGReg r0 = TCG_REG_O0; - const TCGReg r1 = TCG_REG_O1; - const TCGReg r2 = TCG_REG_O2; - unsigned s_bits = opc & MO_SIZE; - unsigned a_bits = get_alignment_bits(opc); - tcg_target_long compare_mask; + int cmp_off = is_ld ? offsetof(CPUTLBEntry, addr_read) + : offsetof(CPUTLBEntry, addr_write); + int add_off = offsetof(CPUTLBEntry, addend); + int compare_mask; + int cc; /* Load tlb_mask[mmu_idx] and tlb_table[mmu_idx]. */ - tcg_out_ld(s, TCG_TYPE_PTR, r0, TCG_AREG0, mask_off); - tcg_out_ld(s, TCG_TYPE_PTR, r1, TCG_AREG0, table_off); + QEMU_BUILD_BUG_ON(TLB_MASK_TABLE_OFS(0) > 0); + QEMU_BUILD_BUG_ON(TLB_MASK_TABLE_OFS(0) < -(1 << 12)); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_T2, TCG_AREG0, mask_off); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_T3, TCG_AREG0, table_off); /* Extract the page index, shifted into place for tlb index. */ - tcg_out_arithi(s, r2, addr, TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS, - SHIFT_SRL); - tcg_out_arith(s, r2, r2, r0, ARITH_AND); + tcg_out_arithi(s, TCG_REG_T1, addr_reg, + TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS, SHIFT_SRL); + tcg_out_arith(s, TCG_REG_T1, TCG_REG_T1, TCG_REG_T2, ARITH_AND); /* Add the tlb_table pointer, creating the CPUTLBEntry address into R2. */ - tcg_out_arith(s, r2, r2, r1, ARITH_ADD); + tcg_out_arith(s, TCG_REG_T1, TCG_REG_T1, TCG_REG_T3, ARITH_ADD); - /* Load the tlb comparator and the addend. */ - tcg_out_ld(s, TCG_TYPE_TL, r0, r2, which); - tcg_out_ld(s, TCG_TYPE_PTR, r1, r2, offsetof(CPUTLBEntry, addend)); + /* Load the tlb comparator and the addend. */ + tcg_out_ld(s, TCG_TYPE_TL, TCG_REG_T2, TCG_REG_T1, cmp_off); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_T1, TCG_REG_T1, add_off); + h->base = TCG_REG_T1; - /* Mask out the page offset, except for the required alignment. - We don't support unaligned accesses. */ - if (a_bits < s_bits) { - a_bits = s_bits; - } - compare_mask = (tcg_target_ulong)TARGET_PAGE_MASK | ((1 << a_bits) - 1); + /* Mask out the page offset, except for the required alignment. */ + compare_mask = TARGET_PAGE_MASK | a_mask; if (check_fit_tl(compare_mask, 13)) { - tcg_out_arithi(s, r2, addr, compare_mask, ARITH_AND); + tcg_out_arithi(s, TCG_REG_T3, addr_reg, compare_mask, ARITH_AND); } else { - tcg_out_movi(s, TCG_TYPE_TL, r2, compare_mask); - tcg_out_arith(s, r2, addr, r2, ARITH_AND); + tcg_out_movi_s32(s, TCG_REG_T3, compare_mask); + tcg_out_arith(s, TCG_REG_T3, addr_reg, TCG_REG_T3, ARITH_AND); } - tcg_out_cmp(s, r0, r2, 0); + tcg_out_cmp(s, TCG_REG_T2, TCG_REG_T3, 0); - /* If the guest address must be zero-extended, do so now. */ + ldst = new_ldst_label(s); + ldst->is_ld = is_ld; + ldst->oi = oi; + ldst->addrlo_reg = addr_reg; + ldst->label_ptr[0] = s->code_ptr; + + /* bne,pn %[xi]cc, label0 */ + cc = TARGET_LONG_BITS == 64 ? BPCC_XCC : BPCC_ICC; + tcg_out_bpcc0(s, COND_NE, BPCC_PN | cc, 0); +#else + if (a_bits != s_bits) { + /* + * Test for at least natural alignment, and defer + * everything else to the helper functions. + */ + tcg_debug_assert(check_fit_tl(a_mask, 13)); + tcg_out_arithi(s, TCG_REG_G0, addr_reg, a_mask, ARITH_ANDCC); + + ldst = new_ldst_label(s); + ldst->is_ld = is_ld; + ldst->oi = oi; + ldst->addrlo_reg = addr_reg; + ldst->label_ptr[0] = s->code_ptr; + + /* bne,pn %icc, label0 */ + tcg_out_bpcc0(s, COND_NE, BPCC_PN | BPCC_ICC, 0); + } + h->base = guest_base ? TCG_GUEST_BASE_REG : TCG_REG_G0; +#endif + + /* If the guest address must be zero-extended, do in the delay slot. */ if (TARGET_LONG_BITS == 32) { - tcg_out_ext32u(s, r0, addr); - return r0; + tcg_out_ext32u(s, TCG_REG_T2, addr_reg); + h->index = TCG_REG_T2; + } else { + if (ldst) { + tcg_out_nop(s); + } + h->index = addr_reg; } - return addr; + return ldst; } -#endif /* CONFIG_SOFTMMU */ - -static const int qemu_ld_opc[(MO_SSIZE | MO_BSWAP) + 1] = { - [MO_UB] = LDUB, - [MO_SB] = LDSB, - [MO_UB | MO_LE] = LDUB, - [MO_SB | MO_LE] = LDSB, - - [MO_BEUW] = LDUH, - [MO_BESW] = LDSH, - [MO_BEUL] = LDUW, - [MO_BESL] = LDSW, - [MO_BEUQ] = LDX, - [MO_BESQ] = LDX, - - [MO_LEUW] = LDUH_LE, - [MO_LESW] = LDSH_LE, - [MO_LEUL] = LDUW_LE, - [MO_LESL] = LDSW_LE, - [MO_LEUQ] = LDX_LE, - [MO_LESQ] = LDX_LE, -}; - -static const int qemu_st_opc[(MO_SIZE | MO_BSWAP) + 1] = { - [MO_UB] = STB, - - [MO_BEUW] = STH, - [MO_BEUL] = STW, - [MO_BEUQ] = STX, - - [MO_LEUW] = STH_LE, - [MO_LEUL] = STW_LE, - [MO_LEUQ] = STX_LE, -}; static void tcg_out_qemu_ld(TCGContext *s, TCGReg data, TCGReg addr, MemOpIdx oi, TCGType data_type) { - MemOp memop = get_memop(oi); - tcg_insn_unit *label_ptr; + static const int ld_opc[(MO_SSIZE | MO_BSWAP) + 1] = { + [MO_UB] = LDUB, + [MO_SB] = LDSB, + [MO_UB | MO_LE] = LDUB, + [MO_SB | MO_LE] = LDSB, -#ifdef CONFIG_SOFTMMU - unsigned memi = get_mmuidx(oi); - TCGReg addrz; - const tcg_insn_unit *func; + [MO_BEUW] = LDUH, + [MO_BESW] = LDSH, + [MO_BEUL] = LDUW, + [MO_BESL] = LDSW, + [MO_BEUQ] = LDX, + [MO_BESQ] = LDX, - addrz = tcg_out_tlb_load(s, addr, memi, memop, - offsetof(CPUTLBEntry, addr_read)); + [MO_LEUW] = LDUH_LE, + [MO_LESW] = LDSH_LE, + [MO_LEUL] = LDUW_LE, + [MO_LESL] = LDSW_LE, + [MO_LEUQ] = LDX_LE, + [MO_LESQ] = LDX_LE, + }; - /* The fast path is exactly one insn. Thus we can perform the - entire TLB Hit in the (annulled) delay slot of the branch - over the TLB Miss case. */ + TCGLabelQemuLdst *ldst; + HostAddress h; - /* beq,a,pt %[xi]cc, label0 */ - label_ptr = s->code_ptr; - tcg_out_bpcc0(s, COND_E, BPCC_A | BPCC_PT - | (TARGET_LONG_BITS == 64 ? BPCC_XCC : BPCC_ICC), 0); - /* delay slot */ - tcg_out_ldst_rr(s, data, addrz, TCG_REG_O1, - qemu_ld_opc[memop & (MO_BSWAP | MO_SSIZE)]); + ldst = prepare_host_addr(s, &h, addr, oi, true); - /* TLB Miss. */ + tcg_out_ldst_rr(s, data, h.base, h.index, + ld_opc[get_memop(oi) & (MO_BSWAP | MO_SSIZE)]); - tcg_out_mov(s, TCG_TYPE_REG, TCG_REG_O1, addrz); - - /* We use the helpers to extend SB and SW data, leaving the case - of SL needing explicit extending below. */ - if ((memop & MO_SSIZE) == MO_SL) { - func = qemu_ld_trampoline[MO_UL]; - } else { - func = qemu_ld_trampoline[memop & MO_SSIZE]; + if (ldst) { + ldst->type = data_type; + ldst->datalo_reg = data; + ldst->raddr = tcg_splitwx_to_rx(s->code_ptr); } - tcg_debug_assert(func != NULL); - tcg_out_call_nodelay(s, func, false); - /* delay slot */ - tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_O2, oi); - - /* We let the helper sign-extend SB and SW, but leave SL for here. */ - if ((memop & MO_SSIZE) == MO_SL) { - tcg_out_ext32s(s, data, TCG_REG_O0); - } else { - tcg_out_mov(s, TCG_TYPE_REG, data, TCG_REG_O0); - } - - *label_ptr |= INSN_OFF19(tcg_ptr_byte_diff(s->code_ptr, label_ptr)); -#else - TCGReg index = (guest_base ? TCG_GUEST_BASE_REG : TCG_REG_G0); - unsigned a_bits = get_alignment_bits(memop); - unsigned s_bits = memop & MO_SIZE; - unsigned t_bits; - - if (TARGET_LONG_BITS == 32) { - tcg_out_ext32u(s, TCG_REG_T1, addr); - addr = TCG_REG_T1; - } - - /* - * Normal case: alignment equal to access size. - */ - if (a_bits == s_bits) { - tcg_out_ldst_rr(s, data, addr, index, - qemu_ld_opc[memop & (MO_BSWAP | MO_SSIZE)]); - return; - } - - /* - * Test for at least natural alignment, and assume most accesses - * will be aligned -- perform a straight load in the delay slot. - * This is required to preserve atomicity for aligned accesses. - */ - t_bits = MAX(a_bits, s_bits); - tcg_debug_assert(t_bits < 13); - tcg_out_arithi(s, TCG_REG_G0, addr, (1u << t_bits) - 1, ARITH_ANDCC); - - /* beq,a,pt %icc, label */ - label_ptr = s->code_ptr; - tcg_out_bpcc0(s, COND_E, BPCC_A | BPCC_PT | BPCC_ICC, 0); - /* delay slot */ - tcg_out_ldst_rr(s, data, addr, index, - qemu_ld_opc[memop & (MO_BSWAP | MO_SSIZE)]); - - if (a_bits >= s_bits) { - /* - * Overalignment: A successful alignment test will perform the memory - * operation in the delay slot, and failure need only invoke the - * handler for SIGBUS. - */ - tcg_out_call_nodelay(s, qemu_unalign_ld_trampoline, false); - /* delay slot -- move to low part of argument reg */ - tcg_out_mov_delay(s, TCG_REG_O1, addr); - } else { - /* Underalignment: load by pieces of minimum alignment. */ - int ld_opc, a_size, s_size, i; - - /* - * Force full address into T1 early; avoids problems with - * overlap between @addr and @data. - */ - tcg_out_arith(s, TCG_REG_T1, addr, index, ARITH_ADD); - - a_size = 1 << a_bits; - s_size = 1 << s_bits; - if ((memop & MO_BSWAP) == MO_BE) { - ld_opc = qemu_ld_opc[a_bits | MO_BE | (memop & MO_SIGN)]; - tcg_out_ldst(s, data, TCG_REG_T1, 0, ld_opc); - ld_opc = qemu_ld_opc[a_bits | MO_BE]; - for (i = a_size; i < s_size; i += a_size) { - tcg_out_ldst(s, TCG_REG_T2, TCG_REG_T1, i, ld_opc); - tcg_out_arithi(s, data, data, a_size, SHIFT_SLLX); - tcg_out_arith(s, data, data, TCG_REG_T2, ARITH_OR); - } - } else if (a_bits == 0) { - ld_opc = LDUB; - tcg_out_ldst(s, data, TCG_REG_T1, 0, ld_opc); - for (i = a_size; i < s_size; i += a_size) { - if ((memop & MO_SIGN) && i == s_size - a_size) { - ld_opc = LDSB; - } - tcg_out_ldst(s, TCG_REG_T2, TCG_REG_T1, i, ld_opc); - tcg_out_arithi(s, TCG_REG_T2, TCG_REG_T2, i * 8, SHIFT_SLLX); - tcg_out_arith(s, data, data, TCG_REG_T2, ARITH_OR); - } - } else { - ld_opc = qemu_ld_opc[a_bits | MO_LE]; - tcg_out_ldst_rr(s, data, TCG_REG_T1, TCG_REG_G0, ld_opc); - for (i = a_size; i < s_size; i += a_size) { - tcg_out_arithi(s, TCG_REG_T1, TCG_REG_T1, a_size, ARITH_ADD); - if ((memop & MO_SIGN) && i == s_size - a_size) { - ld_opc = qemu_ld_opc[a_bits | MO_LE | MO_SIGN]; - } - tcg_out_ldst_rr(s, TCG_REG_T2, TCG_REG_T1, TCG_REG_G0, ld_opc); - tcg_out_arithi(s, TCG_REG_T2, TCG_REG_T2, i * 8, SHIFT_SLLX); - tcg_out_arith(s, data, data, TCG_REG_T2, ARITH_OR); - } - } - } - - *label_ptr |= INSN_OFF19(tcg_ptr_byte_diff(s->code_ptr, label_ptr)); -#endif /* CONFIG_SOFTMMU */ } static void tcg_out_qemu_st(TCGContext *s, TCGReg data, TCGReg addr, MemOpIdx oi, TCGType data_type) { - MemOp memop = get_memop(oi); - tcg_insn_unit *label_ptr; + static const int st_opc[(MO_SIZE | MO_BSWAP) + 1] = { + [MO_UB] = STB, -#ifdef CONFIG_SOFTMMU - unsigned memi = get_mmuidx(oi); - TCGReg addrz; - const tcg_insn_unit *func; + [MO_BEUW] = STH, + [MO_BEUL] = STW, + [MO_BEUQ] = STX, - addrz = tcg_out_tlb_load(s, addr, memi, memop, - offsetof(CPUTLBEntry, addr_write)); + [MO_LEUW] = STH_LE, + [MO_LEUL] = STW_LE, + [MO_LEUQ] = STX_LE, + }; - /* The fast path is exactly one insn. Thus we can perform the entire - TLB Hit in the (annulled) delay slot of the branch over TLB Miss. */ - /* beq,a,pt %[xi]cc, label0 */ - label_ptr = s->code_ptr; - tcg_out_bpcc0(s, COND_E, BPCC_A | BPCC_PT - | (TARGET_LONG_BITS == 64 ? BPCC_XCC : BPCC_ICC), 0); - /* delay slot */ - tcg_out_ldst_rr(s, data, addrz, TCG_REG_O1, - qemu_st_opc[memop & (MO_BSWAP | MO_SIZE)]); + TCGLabelQemuLdst *ldst; + HostAddress h; - /* TLB Miss. */ + ldst = prepare_host_addr(s, &h, addr, oi, false); - tcg_out_mov(s, TCG_TYPE_REG, TCG_REG_O1, addrz); - tcg_out_movext(s, (memop & MO_SIZE) == MO_64 ? TCG_TYPE_I64 : TCG_TYPE_I32, - TCG_REG_O2, data_type, memop & MO_SIZE, data); + tcg_out_ldst_rr(s, data, h.base, h.index, + st_opc[get_memop(oi) & (MO_BSWAP | MO_SIZE)]); - func = qemu_st_trampoline[memop & MO_SIZE]; - tcg_debug_assert(func != NULL); - tcg_out_call_nodelay(s, func, false); - /* delay slot */ - tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_O3, oi); - - *label_ptr |= INSN_OFF19(tcg_ptr_byte_diff(s->code_ptr, label_ptr)); -#else - TCGReg index = (guest_base ? TCG_GUEST_BASE_REG : TCG_REG_G0); - unsigned a_bits = get_alignment_bits(memop); - unsigned s_bits = memop & MO_SIZE; - unsigned t_bits; - - if (TARGET_LONG_BITS == 32) { - tcg_out_ext32u(s, TCG_REG_T1, addr); - addr = TCG_REG_T1; + if (ldst) { + ldst->type = data_type; + ldst->datalo_reg = data; + ldst->raddr = tcg_splitwx_to_rx(s->code_ptr); } - - /* - * Normal case: alignment equal to access size. - */ - if (a_bits == s_bits) { - tcg_out_ldst_rr(s, data, addr, index, - qemu_st_opc[memop & (MO_BSWAP | MO_SIZE)]); - return; - } - - /* - * Test for at least natural alignment, and assume most accesses - * will be aligned -- perform a straight store in the delay slot. - * This is required to preserve atomicity for aligned accesses. - */ - t_bits = MAX(a_bits, s_bits); - tcg_debug_assert(t_bits < 13); - tcg_out_arithi(s, TCG_REG_G0, addr, (1u << t_bits) - 1, ARITH_ANDCC); - - /* beq,a,pt %icc, label */ - label_ptr = s->code_ptr; - tcg_out_bpcc0(s, COND_E, BPCC_A | BPCC_PT | BPCC_ICC, 0); - /* delay slot */ - tcg_out_ldst_rr(s, data, addr, index, - qemu_st_opc[memop & (MO_BSWAP | MO_SIZE)]); - - if (a_bits >= s_bits) { - /* - * Overalignment: A successful alignment test will perform the memory - * operation in the delay slot, and failure need only invoke the - * handler for SIGBUS. - */ - tcg_out_call_nodelay(s, qemu_unalign_st_trampoline, false); - /* delay slot -- move to low part of argument reg */ - tcg_out_mov_delay(s, TCG_REG_O1, addr); - } else { - /* Underalignment: store by pieces of minimum alignment. */ - int st_opc, a_size, s_size, i; - - /* - * Force full address into T1 early; avoids problems with - * overlap between @addr and @data. - */ - tcg_out_arith(s, TCG_REG_T1, addr, index, ARITH_ADD); - - a_size = 1 << a_bits; - s_size = 1 << s_bits; - if ((memop & MO_BSWAP) == MO_BE) { - st_opc = qemu_st_opc[a_bits | MO_BE]; - for (i = 0; i < s_size; i += a_size) { - TCGReg d = data; - int shift = (s_size - a_size - i) * 8; - if (shift) { - d = TCG_REG_T2; - tcg_out_arithi(s, d, data, shift, SHIFT_SRLX); - } - tcg_out_ldst(s, d, TCG_REG_T1, i, st_opc); - } - } else if (a_bits == 0) { - tcg_out_ldst(s, data, TCG_REG_T1, 0, STB); - for (i = 1; i < s_size; i++) { - tcg_out_arithi(s, TCG_REG_T2, data, i * 8, SHIFT_SRLX); - tcg_out_ldst(s, TCG_REG_T2, TCG_REG_T1, i, STB); - } - } else { - /* Note that ST*A with immediate asi must use indexed address. */ - st_opc = qemu_st_opc[a_bits + MO_LE]; - tcg_out_ldst_rr(s, data, TCG_REG_T1, TCG_REG_G0, st_opc); - for (i = a_size; i < s_size; i += a_size) { - tcg_out_arithi(s, TCG_REG_T2, data, i * 8, SHIFT_SRLX); - tcg_out_arithi(s, TCG_REG_T1, TCG_REG_T1, a_size, ARITH_ADD); - tcg_out_ldst_rr(s, TCG_REG_T2, TCG_REG_T1, TCG_REG_G0, st_opc); - } - } - } - - *label_ptr |= INSN_OFF19(tcg_ptr_byte_diff(s->code_ptr, label_ptr)); -#endif /* CONFIG_SOFTMMU */ } static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) @@ -1744,6 +1499,8 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_extu_i32_i64: case INDEX_op_extrl_i64_i32: case INDEX_op_extrh_i64_i32: + case INDEX_op_qemu_ld_i32: + case INDEX_op_qemu_ld_i64: return C_O1_I1(r, r); case INDEX_op_st8_i32: @@ -1753,6 +1510,8 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_st_i32: case INDEX_op_st32_i64: case INDEX_op_st_i64: + case INDEX_op_qemu_st_i32: + case INDEX_op_qemu_st_i64: return C_O0_I2(rZ, r); case INDEX_op_add_i32: @@ -1802,13 +1561,6 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_muluh_i64: return C_O1_I2(r, r, r); - case INDEX_op_qemu_ld_i32: - case INDEX_op_qemu_ld_i64: - return C_O1_I1(r, s); - case INDEX_op_qemu_st_i32: - case INDEX_op_qemu_st_i64: - return C_O0_I2(sZ, s); - default: g_assert_not_reached(); } From patchwork Tue May 16 19:40:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682371 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp639682ltp; Tue, 16 May 2023 12:45:11 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4j4WsZv4eAB3mmSgFNN+WdEGTM+fX33ftrC4Oe0c+1ucUwMIK6mjSApiFMeWUxQXjxUOth X-Received: by 2002:a05:622a:14c:b0:3f3:8ac9:9966 with SMTP id v12-20020a05622a014c00b003f38ac99966mr58578716qtw.21.1684266311261; Tue, 16 May 2023 12:45:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266311; cv=none; d=google.com; s=arc-20160816; b=d68PF1p45OoCTvIlMLCxTKsuZ1drZTnCtJc8TUen4pDQrgmh3L/knkMfZGLT+Arnvr GVsI8UZMxjW2xWFrcN9HWWs5YmOVL3V2aEUY9msyZAwv7KR/uK46PhOuK8Sr80SM3vNm tswePbCHrp3L55w0JNVrGwrxvauCK0Uafb7uGEfCueUh6AwANcdBt+3EIBUyvcS96siu 9iNMaVnVSFLtj3AsVkUF/9kWbIdLOkuFf2p2t5TbJJ2vpbvkYhPj19XXnnKgvEtjAYNA 7lrNxZwjeOw7/SHasDbmB8bRtNGxXq8zzNj5ME6IHVYrYqh5eNBhYKmm/hiiTWEPhQp3 Idww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=3QMQeFWwqcUFumXr1WKfxiG/Sk++Gp7SwpIzlewGlEA=; b=ltVBzDk6IKtTEGyuA8s04v70q56D20v/43dU+8C24+tZHuY7t23DRpi0Hlz/IspKq9 y8SR6kuRg/FXb8I8quKyM99Tc+7qv+NYODCfUxkxeoKaAixVCMVyN1r+M4/YGpLia5rC MNfYsdgSI0+XVxcOtYZEKok3EhDonHIUPyXJqClbGPs4tCELkdEqOBx+BnALrN85N/Ne pwp3/pYxJG6MCQOlfJ7ais6BHjtdSXv3FudhljKaPkAyDQOwKoZanf/Yb1dYQJfwjFZq G8AvZtBKU5vhNwWU7at/MR/M/SoibEEsFW7n0vV1f0B+9cUTwoVqHqXrbMXZOgKiwTXY zMig== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OXKcHHf+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 8-20020ac84e88000000b003f200130580si809376qtp.554.2023.05.16.12.45.11 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:45:11 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OXKcHHf+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yp-0006kR-OA; Tue, 16 May 2023 15:42:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yo-0006jn-T2 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:14 -0400 Received: from mail-pf1-x435.google.com ([2607:f8b0:4864:20::435]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yn-0002sa-G1 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:14 -0400 Received: by mail-pf1-x435.google.com with SMTP id d2e1a72fcca58-64a9335a8e7so8218438b3a.0 for ; Tue, 16 May 2023 12:42:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266132; x=1686858132; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3QMQeFWwqcUFumXr1WKfxiG/Sk++Gp7SwpIzlewGlEA=; b=OXKcHHf+ipX6PzfByHE3D/+Kx9npd4tYhTXklF066RPQp4CWW1hTGQWTU9uv6GlItL pr/UAYa5fO60wHJ0QZlYTWPyd23E9E6s4DKk/fn9d8H371DBOtN064zWvBwL12lB/5a4 1v5lKxHSin7cejEyHlb4cWHuTkau/Y9ppRayDSLaWjKMMQCixlRlXI2sUQEZJwH98g9U iBjEHC5VgxKXfwGF8s+el8JAnEgkmyDcWMFe/euiSBS0pFH+l8iBcfWwcIogc2T8JNTM MZMONxMRtCcS84C3qTe+hLna98Tz6Ry2ieq78kFW/AoAyw9sj1KF2P6vLk9F+YkqZ+K5 mpiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266132; x=1686858132; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3QMQeFWwqcUFumXr1WKfxiG/Sk++Gp7SwpIzlewGlEA=; b=S76SdYtJgHBB4PPQ5fY6MBBneFnVUl4kNepf1Cc+VcEWYzG/1vqD0f2Cvz6chCpHmd sSnzm0nFca/0B9PfjupDcnUBbTqqAnZvTu6OPukFzAE+f68Ro1Dt5nLEBgkajYEOlXW1 mzwe22R1lC62O3cz7Sjy8exXLPx78BGcxKlfAECpThQIJUcRrGJuV+4QLF7APJzYtwpu FtYwUNPmWKVyzC+Cs0Eyaqtu1apfBHaIXlr/baHCEI+DCpGACOspYCT6W38j8x8ir+VW jIhScitrAoLvD/+ptIBXdoh7k034sQjtw+t5d2Y8/nTHWPJLL3tjqV0ntYHKHAy0Iezs ZAtQ== X-Gm-Message-State: AC+VfDw4Km+tqsj3g2N3KxcwHgnyP6Prt0mU7Uj5qvcKlz6SqV95iiw1 5DnJdvFWmIwMhs/4pDO0xpyHHQHw+fHwpXh/vWs= X-Received: by 2002:a05:6a00:15c6:b0:635:d9e1:8e1e with SMTP id o6-20020a056a0015c600b00635d9e18e1emr54936991pfu.11.1684266132290; Tue, 16 May 2023 12:42:12 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:11 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 28/80] accel/tcg: Remove helper_unaligned_{ld,st} Date: Tue, 16 May 2023 12:40:53 -0700 Message-Id: <20230516194145.1749305-29-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::435; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x435.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org These functions are now unused. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- include/tcg/tcg-ldst.h | 6 ------ accel/tcg/user-exec.c | 10 ---------- 2 files changed, 16 deletions(-) diff --git a/include/tcg/tcg-ldst.h b/include/tcg/tcg-ldst.h index 64f48e6990..7dd57013e9 100644 --- a/include/tcg/tcg-ldst.h +++ b/include/tcg/tcg-ldst.h @@ -60,10 +60,4 @@ void helper_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, void helper_st16_mmu(CPUArchState *env, target_ulong addr, Int128 val, MemOpIdx oi, uintptr_t retaddr); -#ifdef CONFIG_USER_ONLY - -G_NORETURN void helper_unaligned_ld(CPUArchState *env, target_ulong addr); -G_NORETURN void helper_unaligned_st(CPUArchState *env, target_ulong addr); - -#endif /* CONFIG_USER_ONLY */ #endif /* TCG_LDST_H */ diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index 8f86254eb4..7b824dcde8 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -889,16 +889,6 @@ void page_reset_target_data(target_ulong start, target_ulong last) { } /* The softmmu versions of these helpers are in cputlb.c. */ -void helper_unaligned_ld(CPUArchState *env, target_ulong addr) -{ - cpu_loop_exit_sigbus(env_cpu(env), addr, MMU_DATA_LOAD, GETPC()); -} - -void helper_unaligned_st(CPUArchState *env, target_ulong addr) -{ - cpu_loop_exit_sigbus(env_cpu(env), addr, MMU_DATA_STORE, GETPC()); -} - static void *cpu_mmu_lookup(CPUArchState *env, abi_ptr addr, MemOp mop, uintptr_t ra, MMUAccessType type) { From patchwork Tue May 16 19:40:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682379 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp17983wrt; Tue, 16 May 2023 12:48:49 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4Jw5wNzl9UTwzk4Z19kLhxri5KQsJRfRx+mlscU/hcQZqRtnmgXrPQjuALcYxQy+5MzWzw X-Received: by 2002:a05:622a:210:b0:3e2:1005:c6b with SMTP id b16-20020a05622a021000b003e210050c6bmr63427947qtx.22.1684266529528; Tue, 16 May 2023 12:48:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266529; cv=none; d=google.com; s=arc-20160816; b=sM/81OkgNoz8v8AXCRma8kP2q7jD7jat0I5TI6Q0dslkMoOZUyfZkFYX2NcYCJscR0 9t9nqVV1mZTPhdr/bXm4rGWaJPT/WeHMyl9a2iAcmDz1wDIobvRwFxmn8CkHSqkoIYpD 2cGP01APRH9pSgcR21qHDtYmP+HEBIsrRzbw7y3LBOYVy3lgwqTEXMWNqL4BGE+Ifdpl Sat0sbVTdiddoMW54G+y1ftkxjFQnV8hnCLvSNFyWyC0AoZjFtsGcBw2R96BCDpw7drE izcpx16MRnx/oanFGJ17xswOjlGCs3dALL2jCg1H865eKStU5wIPREyi08cSeTKVV41I P6kw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=wb0Auln9ncK1FIC00yKgRrVnu9FaIzhjuMCmgHUtjOQ=; b=Fk2Wk7/GW5sxc1lcRIybMOeY1ouAFQKdcBF9QC2mV95qpehaCK+nFuzeYHe2eUcD4l /78VpApiIaQKAr7ZGBVDNV60wPyCLYzHfEt8KOzoVHNmDJ2VtP1JKk4JaM52juwiW+d4 pp9nTAXFf6r7D3AHerr/weCzNoCo+4MD15ecRAQ8fC/T7v2w2i7SJ2+T3kRxjA9YcmPZ m3Nbm8HObi+0Heus61znlTiPN8baVk560BU7dc4e2MtTRbsegT+hWo0KmLnDC0qr0tm2 QluSS7sJBnp4Bxxo+AnrRb+WZmrb5O3j26ZrDXTfGG/GGFDHn4bvX5JqU2joDyOBsLD8 5cJw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ReEUNC0s; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a9-20020a05622a064900b003e3894ef0a3si11715443qtb.575.2023.05.16.12.48.49 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:48:49 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ReEUNC0s; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yt-0006lr-7t; Tue, 16 May 2023 15:42:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yq-0006ks-1B for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:16 -0400 Received: from mail-pf1-x42b.google.com ([2607:f8b0:4864:20::42b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yn-0002jw-V2 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:15 -0400 Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-64ab2a37812so8110268b3a.1 for ; Tue, 16 May 2023 12:42:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266133; x=1686858133; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wb0Auln9ncK1FIC00yKgRrVnu9FaIzhjuMCmgHUtjOQ=; b=ReEUNC0sIAIqj9J9LzHnMiJpJA2tuMeubm3vwilclHc5a7p6wk4c2axs7/sSCq/NUg lDEz+pE2kyjj3VKPdUwZNAnBbQfhfWDJ89+8mW9ey8b0LVtmr2C9tggG8WlETB0fI+t6 z0kZz1CvnHi7lutjmZyZlp9cBPT9VGg/W4f0di/xoovTuRpm9XiTlsenrfEK5byF7E7Q 3WCRzzo7sR8EWF3TJytPP06bsRWc9zIQJ9eA5iGP8GrJAF56xXVgaGqn032vAt13aD0r x3brpq5nMopRRP+LZK7AjEwKSvJmqlQ8lIIotICJX2LWu6OhSFmACpCSn2wsrP7SI4yJ +mlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266133; x=1686858133; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wb0Auln9ncK1FIC00yKgRrVnu9FaIzhjuMCmgHUtjOQ=; b=H3PMXU4tsvlTXtY36a55fEi5vKYNXyjMhs5N91IDMkX3Q9YskJeLwx2CehqxuQMGFu +eOLteMJKutEqGzXOgdp9zwZwMBtdtgko0uZ5llwkFcMJ2KTmm87jGcx5J0+4pHPdGc1 6I3MEJvFRuLqhHuGVuflPpaZchPKTgWwMTvxi2XyRK1/ZR52VwL6yXWF26v6ts/FvLoq lwjiVWLj1AR36SHPa/yzngTQCb+UBfg4FI89UyRMWWcfzqUTCvR5kOLljWDnL7I/tcD4 1jK2iFFU7IIqmejgFZTY8JwC6VMoQBoYnUI70r5iiEWi9nv9Mn7whHhnRLyjt/ziG8/9 DnbQ== X-Gm-Message-State: AC+VfDw4Cm4S+p0Dey7q/R4G+PjZnMDPkhg1bVNDpSTMtSqAtmvu3/6Z M3O2g16mQk3MfoyXf//cWCeEohv/TBfATd8+TPg= X-Received: by 2002:a05:6a20:a115:b0:100:c3fe:a653 with SMTP id q21-20020a056a20a11500b00100c3fea653mr35368928pzk.29.1684266133258; Tue, 16 May 2023 12:42:13 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:12 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 29/80] tcg/loongarch64: Check the host supports unaligned accesses Date: Tue, 16 May 2023 12:40:54 -0700 Message-Id: <20230516194145.1749305-30-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42b; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org This should be true of all loongarch64 running Linux. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/loongarch64/tcg-target.c.inc | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/tcg/loongarch64/tcg-target.c.inc b/tcg/loongarch64/tcg-target.c.inc index e651ec5c71..33d8e67513 100644 --- a/tcg/loongarch64/tcg-target.c.inc +++ b/tcg/loongarch64/tcg-target.c.inc @@ -30,6 +30,7 @@ */ #include "../tcg-ldst.c.inc" +#include #ifdef CONFIG_DEBUG_TCG static const char * const tcg_target_reg_names[TCG_TARGET_NB_REGS] = { @@ -1674,6 +1675,14 @@ static void tcg_target_qemu_prologue(TCGContext *s) static void tcg_target_init(TCGContext *s) { + unsigned long hwcap = qemu_getauxval(AT_HWCAP); + + /* Server and desktop class cpus have UAL; embedded cpus do not. */ + if (!(hwcap & HWCAP_LOONGARCH_UAL)) { + error_report("TCG: unaligned access support required; exiting"); + exit(EXIT_FAILURE); + } + tcg_target_available_regs[TCG_TYPE_I32] = ALL_GENERAL_REGS; tcg_target_available_regs[TCG_TYPE_I64] = ALL_GENERAL_REGS; From patchwork Tue May 16 19:40:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682372 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp639686ltp; Tue, 16 May 2023 12:45:11 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5lsJuZk2x1kyoMRxWyfe1aTkybtAcf6EVchv5Gej80ribp3SH3wjuOt9aEmhL4ykoXaIcU X-Received: by 2002:a05:6214:519a:b0:621:65de:f600 with SMTP id kl26-20020a056214519a00b0062165def600mr31588310qvb.1.1684266311731; Tue, 16 May 2023 12:45:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266311; cv=none; d=google.com; s=arc-20160816; b=Ed5A+ivURKpD94gm1ilfOsZWOOwkk4hoEbbinDUQCX5CTIZ3PM9Dsrprxwk9A3FuAH he5xH/Fa7Fsgn7OX1yms3FNEj5tS3fXgQvfmrjP/TagsNW7W7kX/hMkHPGbyPm7uuKMH nF8i1QDAieXhrkzMTLFswBJ4Flqsi1FwM+KlbvLhusGrHJttGuBAi/GyaNDiBOoxQYoa SNwqb0b3A50RkgZHElWDGtyHAFoLOWQNZfdRtnA9W95S0z7YMK5svAqGqvcV50mJqJqF fb/zJzslyNBdw5SPKaIw0Wl+BTJ5er/PNZkE92pG2idtXCTbpqqVk/rcJ1kxUKv12Hxi Ar2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=4qhfCxW8nZ+AIzOPAndxpM5sZuukyWLZqrgdNp4ChwI=; b=bp9gtVTevvwvgXoqO4AxYYtLCkEmHPhDxtmW3ToSz4Nskg/924ZiiX0tU8OI29E8Vr 5/vmoN6vyB+SEJUrSSv3VNH6aLmLRNYZxec7qHKz7rzDVLiFwX4uHkKsgCwBo4xpc3ix +VipIAXsTaIJEBG9V5LE/D7fi3lVTW04m/naE6kA1zDNBEznF74/zhGo7Nwif1NYkqjn QRxhttv1pfR+OvzpyUYPOg7sQyVb/83lQS1ezwrTOnRJVid4G3jxas47PTsR9QakS8TF zqiAfFKbqGVdTRTJMCZ/gt4IkVGcTaa79E9q2HPQ98ydb3VynkBQPIbOa5bBX+rMAi02 C50Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=zMKX732x; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id q7-20020ad45ca7000000b005aa3fcc4175si12598271qvh.388.2023.05.16.12.45.11 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:45:11 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=zMKX732x; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Ys-0006m6-Ct; Tue, 16 May 2023 15:42:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yq-0006lF-TR for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:16 -0400 Received: from mail-pf1-x42b.google.com ([2607:f8b0:4864:20::42b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yp-0002su-8a for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:16 -0400 Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-64a9335a8e7so8218593b3a.0 for ; Tue, 16 May 2023 12:42:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266134; x=1686858134; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4qhfCxW8nZ+AIzOPAndxpM5sZuukyWLZqrgdNp4ChwI=; b=zMKX732xEa5SPswvvFk1EloHDtswzovT+7ivlvjr0R7evVBzi6cVTqOUmT6+BpEKfS ubmh674OUeyGVPX2rh1KXHOi0bXjm4yMoWlFOqHNd8ZIqOu8f6sJp6/YLNaad2wuzzPa O5i1mm0fUnjLYmAvQdpEkmQLuc768o8R+v/qaYMHZxef/8XPRlFa1jEO1OjVofmuSQAZ 2y/tCMQpIhMQk5NupOp5IlN9LYjHpVJuWGv260skCRjFgPaJNXWElboPyVL9MtQo/Yir 8X3q0qjAvuzI1l5pGMpFTi3l7jtn9J/yJdjT2iQcawcuB7nAPqAc667FC+PNOHU/N7EF Jx/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266134; x=1686858134; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4qhfCxW8nZ+AIzOPAndxpM5sZuukyWLZqrgdNp4ChwI=; b=Dru8Wk6EKt0GXAER2rpC8mvxKQ7xmCQWFvfYni+sQLPeeoxyNM8Y7hrIt2MGy/ycLS 5g3L5/TTrkmNEKyH7ybi+XcU/psaPFDh5RdQUn0pxKTbWIrGwf9NhqKwqwwJizTQfop0 RE5qj/c+DTw1QMJwHQlHwHupgJL0LYHU53PXho661qQPXJ4346li9R9h3aN7++aSwrw1 BjoBjB3EV6UnW5FoaQ6LY8FDRDajrQztNewopMhbN4tHC2pBne9JtPeo7TSKFMIHHGjD 4/F56JpDraZhC26SKzrgHjDoal0FleoD5F5byp/f4bUFBMRiLpqKnVWcFzYYIObFJhnF Szgg== X-Gm-Message-State: AC+VfDzVGNF5vaW7Uct/lDvUlRdyOG9crdVcJBdC/lRKdZArqZ7ZoLWV UyaLurIXAiMmKC3vnmaxJAua1QzHlWQv+l+cpNo= X-Received: by 2002:a05:6a00:b45:b0:63d:3c39:ecc2 with SMTP id p5-20020a056a000b4500b0063d3c39ecc2mr46827241pfo.12.1684266134012; Tue, 16 May 2023 12:42:14 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:13 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 30/80] tcg/loongarch64: Support softmmu unaligned accesses Date: Tue, 16 May 2023 12:40:55 -0700 Message-Id: <20230516194145.1749305-31-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42b; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Test the final byte of an unaligned access. Use BSTRINS.D to clear the range of bits, rather than AND. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/loongarch64/tcg-target.c.inc | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/tcg/loongarch64/tcg-target.c.inc b/tcg/loongarch64/tcg-target.c.inc index 33d8e67513..7d0165349d 100644 --- a/tcg/loongarch64/tcg-target.c.inc +++ b/tcg/loongarch64/tcg-target.c.inc @@ -848,7 +848,6 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, int fast_ofs = TLB_MASK_TABLE_OFS(mem_index); int mask_ofs = fast_ofs + offsetof(CPUTLBDescFast, mask); int table_ofs = fast_ofs + offsetof(CPUTLBDescFast, table); - tcg_target_long compare_mask; ldst = new_ldst_label(s); ldst->is_ld = is_ld; @@ -872,14 +871,20 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP2, TCG_REG_TMP2, offsetof(CPUTLBEntry, addend)); - /* We don't support unaligned accesses. */ + /* + * For aligned accesses, we check the first byte and include the alignment + * bits within the address. For unaligned access, we check that we don't + * cross pages using the address of the last byte of the access. + */ if (a_bits < s_bits) { - a_bits = s_bits; + unsigned a_mask = (1u << a_bits) - 1; + unsigned s_mask = (1u << s_bits) - 1; + tcg_out_addi(s, TCG_TYPE_TL, TCG_REG_TMP1, addr_reg, s_mask - a_mask); + } else { + tcg_out_mov(s, TCG_TYPE_TL, TCG_REG_TMP1, addr_reg); } - /* Clear the non-page, non-alignment bits from the address. */ - compare_mask = (tcg_target_long)TARGET_PAGE_MASK | ((1 << a_bits) - 1); - tcg_out_movi(s, TCG_TYPE_TL, TCG_REG_TMP1, compare_mask); - tcg_out_opc_and(s, TCG_REG_TMP1, TCG_REG_TMP1, addr_reg); + tcg_out_opc_bstrins_d(s, TCG_REG_TMP1, TCG_REG_ZERO, + a_bits, TARGET_PAGE_BITS - 1); /* Compare masked address with the TLB entry. */ ldst->label_ptr[0] = s->code_ptr; From patchwork Tue May 16 19:40:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682408 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp20532wrt; Tue, 16 May 2023 12:57:30 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5FIWsIm2qV66C0nxza7Tk6FoiSNW8nAkvAeb33QdI0/hdnjsIxDNjhUecNJxykxOf3UJBm X-Received: by 2002:a5d:448b:0:b0:307:9194:9a94 with SMTP id j11-20020a5d448b000000b0030791949a94mr23968126wrq.17.1684267050385; Tue, 16 May 2023 12:57:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267050; cv=none; d=google.com; s=arc-20160816; b=V78To7bkVEKOWfgy6s/QbMCxwh83jzFaR0OlvlID8/vcExKD+lviDD2OXcmBcpRGcX /GMA/8vgjkdn2rmH9H/m5O3O/7ls2M1sG/Q9GIhn+YSZuq/SJl6xhH/IgBp0HUaT8CZe 3M6tYA0b8xvAgwJpMBS1sH+Y7vSMUCupTDj0HpEApzcDiuVlvi3Rq6fsEX0LyhbZXkWU D6MbyyypXUOzxN8cIHI516C/xBTrs2fiddU/tyI3Fhts6EpyxcPUVIELF6Rvk+1qwFE+ q1ybxZR+pg4MZh7nNxyQZOms6AaYYWLLGrzJwiQE/O9BlvSwcWtrAh65Gng2ruRsJNPJ 6hlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=RcueoQ7mogTzwxRR1biig20yNfysNgxqkYSawavzYAo=; b=RNLPOilfQ0e6ZninSdA7sB8CX7CnguSP8mbUR9B7dTxMbyn+lE8wOqB9Aej4iuUu0a YyzWf2782UUwJtImYRTDju1mq1dY13g26xcJkl7Bzmz7v90Tox8BqvbHm8C1fsuiM/cb K9YWuEal249qcK0IrmXPgRn0LnJTp00mYls+48XESZyhJqK/UYDHX4V4WK1s+D635c37 BnIDgXU0V/5cs2iz4Eyzy9R8Hx/j6JeHEROgj+SL1f7i6MyX60VX9R0MMyVOTwQ1zZiq y/4/cEl3k1cfoyoz0K5RCKPFh59K9R56b4hYtBkY3lqJvbn5SVGYRTW8uQ1Njk7C//Zt WuRg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=b8dcxUmm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id c15-20020adfef4f000000b0030639d3dedcsi166394wrp.547.2023.05.16.12.57.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:57:30 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=b8dcxUmm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Ys-0006mH-R9; Tue, 16 May 2023 15:42:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yr-0006lc-RR for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:17 -0400 Received: from mail-pg1-x533.google.com ([2607:f8b0:4864:20::533]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yq-0002tC-5g for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:17 -0400 Received: by mail-pg1-x533.google.com with SMTP id 41be03b00d2f7-52c30fa5271so7877690a12.0 for ; Tue, 16 May 2023 12:42:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266135; x=1686858135; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RcueoQ7mogTzwxRR1biig20yNfysNgxqkYSawavzYAo=; b=b8dcxUmm07jotRHjdt55bHkHvuKu/bc3QYVHPyPcmNoHwsBv1iCr86YxNUYcWN1vrJ aBC1E61xrU0Cm9UuYP4nq8UqmkF14MpaIZdIcVJqTdkEDMjzATDOd4m8BZTt463X+t3w pBMKYONPeKPHQ6aSChd5nXVhLBoqcR26FlY9k20+c1Yo49hqn1OdsT6hO/K8JJGZMJQ6 deLYzUTST22956R8j/NYs1Urg9wCOL3q4RLxCyWpmefjs62GP/DBYL0I2VdOyNprV7fj UVQcc7V78IbEAOkECb9Jb2A+CH+peTcV0nWejn74TD6lQ/aIRhYHsypoNU9ooApRQF2X uAWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266135; x=1686858135; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RcueoQ7mogTzwxRR1biig20yNfysNgxqkYSawavzYAo=; b=DIH4FrcfK7RzNlPO+A5p1v2Uwbtjten5fKKrBGv7v9XQobIf3Xsp+KfbeWvrefj3JC CN53FH/e4B/OBpL0PqjRe+DvLPVjn6Trdvfr94j7WFtxgou3jqVIdxDUxp/oFgnKoJvA g/qezODA2NaYZD/E0jZ9HSh/t/wf+TuN3vr4OYAelMcSOFqQGCgpVTwFmZ0l4Fu2Ey2W CpQeCX8gacq8OmhwIMWGw2ge45DfTAjZMaMUDk1r1wyKyQsT+yG1zeqJ1TQBebwzWBsu bHsH6vnkAtSHCNfXh9AgsFapzNoAqrnIW1Wf0NstcwnW2/4cRUB8QLwyfNnQQEMSN2hW 1x2g== X-Gm-Message-State: AC+VfDzd9DjYsL0TAgepOOe82uztOfNNDX/pFkAW/iqQ0uld7KgO3Wal xyYbiOTNnQ2eCu3UlFLJ+S/kSI7vXaMLGGcoJB4= X-Received: by 2002:a05:6a20:734a:b0:107:10b6:4c8a with SMTP id v10-20020a056a20734a00b0010710b64c8amr3200208pzc.24.1684266134824; Tue, 16 May 2023 12:42:14 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:14 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: LIU Zhiwei Subject: [PULL 31/80] tcg/riscv: Support softmmu unaligned accesses Date: Tue, 16 May 2023 12:40:56 -0700 Message-Id: <20230516194145.1749305-32-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::533; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x533.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org The system is required to emulate unaligned accesses, even if the hardware does not support it. The resulting trap may or may not be more efficient than the qemu slow path. There are linux kernel patches in flight to allow userspace to query hardware support; we can re-evaluate whether to enable this by default after that. In the meantime, softmmu now matches useronly, where we already assumed that unaligned accesses are supported. Reviewed-by: LIU Zhiwei Signed-off-by: Richard Henderson --- tcg/riscv/tcg-target.c.inc | 48 ++++++++++++++++++++++---------------- 1 file changed, 28 insertions(+), 20 deletions(-) diff --git a/tcg/riscv/tcg-target.c.inc b/tcg/riscv/tcg-target.c.inc index 19cd4507fb..415e6c6e15 100644 --- a/tcg/riscv/tcg-target.c.inc +++ b/tcg/riscv/tcg-target.c.inc @@ -910,12 +910,13 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, TCGReg *pbase, #ifdef CONFIG_SOFTMMU unsigned s_bits = opc & MO_SIZE; + unsigned s_mask = (1u << s_bits) - 1; int mem_index = get_mmuidx(oi); int fast_ofs = TLB_MASK_TABLE_OFS(mem_index); int mask_ofs = fast_ofs + offsetof(CPUTLBDescFast, mask); int table_ofs = fast_ofs + offsetof(CPUTLBDescFast, table); - TCGReg mask_base = TCG_AREG0, table_base = TCG_AREG0; - tcg_target_long compare_mask; + int compare_mask; + TCGReg addr_adj; ldst = new_ldst_label(s); ldst->is_ld = is_ld; @@ -924,14 +925,33 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, TCGReg *pbase, QEMU_BUILD_BUG_ON(TLB_MASK_TABLE_OFS(0) > 0); QEMU_BUILD_BUG_ON(TLB_MASK_TABLE_OFS(0) < -(1 << 11)); - tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP0, mask_base, mask_ofs); - tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP1, table_base, table_ofs); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP0, TCG_AREG0, mask_ofs); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP1, TCG_AREG0, table_ofs); tcg_out_opc_imm(s, OPC_SRLI, TCG_REG_TMP2, addr_reg, TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); tcg_out_opc_reg(s, OPC_AND, TCG_REG_TMP2, TCG_REG_TMP2, TCG_REG_TMP0); tcg_out_opc_reg(s, OPC_ADD, TCG_REG_TMP2, TCG_REG_TMP2, TCG_REG_TMP1); + /* + * For aligned accesses, we check the first byte and include the alignment + * bits within the address. For unaligned access, we check that we don't + * cross pages using the address of the last byte of the access. + */ + addr_adj = addr_reg; + if (a_bits < s_bits) { + addr_adj = TCG_REG_TMP0; + tcg_out_opc_imm(s, TARGET_LONG_BITS == 32 ? OPC_ADDIW : OPC_ADDI, + addr_adj, addr_reg, s_mask - a_mask); + } + compare_mask = TARGET_PAGE_MASK | a_mask; + if (compare_mask == sextreg(compare_mask, 0, 12)) { + tcg_out_opc_imm(s, OPC_ANDI, TCG_REG_TMP1, addr_adj, compare_mask); + } else { + tcg_out_movi(s, TCG_TYPE_TL, TCG_REG_TMP1, compare_mask); + tcg_out_opc_reg(s, OPC_AND, TCG_REG_TMP1, TCG_REG_TMP1, addr_adj); + } + /* Load the tlb comparator and the addend. */ tcg_out_ld(s, TCG_TYPE_TL, TCG_REG_TMP0, TCG_REG_TMP2, is_ld ? offsetof(CPUTLBEntry, addr_read) @@ -939,29 +959,17 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, TCGReg *pbase, tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP2, TCG_REG_TMP2, offsetof(CPUTLBEntry, addend)); - /* We don't support unaligned accesses. */ - if (a_bits < s_bits) { - a_bits = s_bits; - } - /* Clear the non-page, non-alignment bits from the address. */ - compare_mask = (tcg_target_long)TARGET_PAGE_MASK | a_mask; - if (compare_mask == sextreg(compare_mask, 0, 12)) { - tcg_out_opc_imm(s, OPC_ANDI, TCG_REG_TMP1, addr_reg, compare_mask); - } else { - tcg_out_movi(s, TCG_TYPE_TL, TCG_REG_TMP1, compare_mask); - tcg_out_opc_reg(s, OPC_AND, TCG_REG_TMP1, TCG_REG_TMP1, addr_reg); - } - /* Compare masked address with the TLB entry. */ ldst->label_ptr[0] = s->code_ptr; tcg_out_opc_branch(s, OPC_BNE, TCG_REG_TMP0, TCG_REG_TMP1, 0); /* TLB Hit - translate address using addend. */ + addr_adj = addr_reg; if (TARGET_LONG_BITS == 32) { - tcg_out_ext32u(s, TCG_REG_TMP0, addr_reg); - addr_reg = TCG_REG_TMP0; + addr_adj = TCG_REG_TMP0; + tcg_out_ext32u(s, addr_adj, addr_reg); } - tcg_out_opc_reg(s, OPC_ADD, TCG_REG_TMP0, TCG_REG_TMP2, addr_reg); + tcg_out_opc_reg(s, OPC_ADD, TCG_REG_TMP0, TCG_REG_TMP2, addr_adj); *pbase = TCG_REG_TMP0; #else if (a_mask) { From patchwork Tue May 16 19:40:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682360 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp638804ltp; Tue, 16 May 2023 12:42:59 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7qo4pp3Sz+zy/iEabiWfLGL9ZmMpuWPNIMWmO5jA3uAoD2hekSrluf5Wius1UvrlJWgk3s X-Received: by 2002:a05:6214:76a:b0:61c:c8d0:ebb0 with SMTP id f10-20020a056214076a00b0061cc8d0ebb0mr60023217qvz.14.1684266179616; Tue, 16 May 2023 12:42:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266179; cv=none; d=google.com; s=arc-20160816; b=FOCHQR4IpbGL7V/UHuGfxUanCQw3wWE8Ys7TicVA2oudEqzmp2JdNBsU56AyQ9l86C 9bhjilF8qXZJwHaKCRGZTCbU9CWO+QzvjZHjNIbi0wSRZXWZkH8QoLiXucGeM41DWCvr 3fNN2EKom9KNspP6Kh/+Yo46YUnMDmv2/zzBF2h0ksfchF9DfS2eJg5lVj0N7opQfzd3 6j1zWRihKUTMjjfYP5GQ42p8J8om0nCwa2ZsMSgIshF0w1fX7AdQzVcTSZXB6UgJWda8 crlgRMGl65MQvin8IGSRSs86+HLInP237rTzg5vCGIcHbfEkOYOawWKzhPYrmeHazRQ2 Ub0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=HF3N9t+WR/vu9xazr4+Gr/alUcZraRNVjVLeq4yAhBM=; b=k3Fmy+/3KmCLZZ8sFnYNkUq4odgGkonOWmXkLMOAS6NmM+qaG2U2rgWjV5ZhnC3AwR Do7qsDBLLE/AN0/cZL7ExPGnC1D2SEf58rvZGc5LZKBU6yRx3MLNQRz7+NJLd2driG6x kGPpwMMifsMGQB0psvMPivRgNwSUX/Wlhc8Ct4RANNVxa5ICEBzxwVTeT9a4PU/9n89v 1ibM4T1SqWtAaZz16p5NdBO9XNctdeIUvseXwfXOyKwRDBSmP7XZcxlURX+WGlohqoGU RTVybfm+uqZcRjy4TrRhpULAtCS37/nNfuPQw8hcjYjxAUIwBYhHe309oDBtItuVTIQV z0Ag== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CZ1WcllT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id eq2-20020ad45962000000b0057115385fd0si11888042qvb.373.2023.05.16.12.42.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:42:59 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CZ1WcllT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yu-0006ml-4p; Tue, 16 May 2023 15:42:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yt-0006mI-5o for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:19 -0400 Received: from mail-pg1-x52e.google.com ([2607:f8b0:4864:20::52e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yr-0002tT-0Y for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:18 -0400 Received: by mail-pg1-x52e.google.com with SMTP id 41be03b00d2f7-52c759b7d45so13439154a12.3 for ; Tue, 16 May 2023 12:42:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266135; x=1686858135; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HF3N9t+WR/vu9xazr4+Gr/alUcZraRNVjVLeq4yAhBM=; b=CZ1WcllT0I90o5dtNQ800ZBRZTeYQgIrM3yftTXD9Ivo6yQIKskvROD0GBS0r1Xu3a 6aEasWlKb1+7dtTf6TnInRDlBiVtZ0iQheXOb94rFWOP/SS41mNu0Twz8bYh3pJ9Q5hr 8ywaFSGi81Xfh34RzwYJEC3mnG4RuWxcDN3hKU39DlKXXLehTmSzthkEDepbEXnOF8Fs 1quAXI7JRSjrycq8YiACnF94ZelHkrl0F1CXsDl06NOp/00yDIzqF8088Mp9UKow0L1y GtqXq/1GP7KvnBrxKG4aq21W0AhyDGCl3c2gcRCqefjTGpSZjw1jRt718oiZTCt2zXCn M6EQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266135; x=1686858135; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HF3N9t+WR/vu9xazr4+Gr/alUcZraRNVjVLeq4yAhBM=; b=UgKB30X07HjYcGidAybYNZQZ+D0ws4E+GtRAzJMaC/TDdufmvrwj7pJ17TFME44DZ2 8+jeE/KB09p61/maShSjD+U3ToCKOc6hVGzM0TWKZIZ+gI8L6yMrVCnj7xanS1ff07g9 i+KUZCHqxP1AjQaBxAzDJ1SW5nQ86df3zEbzWW7iY6vmyWoydIKkHuBM/1VHr2dOd23k H7+OlXLmWFKEJdCLftot3UazmhFWprMB34+Oj2J10Z4LBm0aI2tsdOW3c06pgmWDecIx VZ1nb2zR8+b64d5KD65QSSHFhugBXrWl8ocxio6ZD9LaQ5SSzfZXZ402GLNG/JcnWyhb iSjg== X-Gm-Message-State: AC+VfDxgao1JOa1S9dJ9J/GKqr24rIeXu3mp72oHAnOcAsqN7HADeXas hs9bFktzEhDSI39QgSD6cRN9P2Kv0fQRQHJGaZI= X-Received: by 2002:a05:6a20:9144:b0:104:41a0:c51a with SMTP id x4-20020a056a20914400b0010441a0c51amr21079441pzc.38.1684266135561; Tue, 16 May 2023 12:42:15 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:15 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 32/80] tcg: Introduce tcg_target_has_memory_bswap Date: Tue, 16 May 2023 12:40:57 -0700 Message-Id: <20230516194145.1749305-33-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52e; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Replace the unparameterized TCG_TARGET_HAS_MEMORY_BSWAP macro with a function with a memop argument. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.h | 1 - tcg/arm/tcg-target.h | 1 - tcg/i386/tcg-target.h | 3 --- tcg/loongarch64/tcg-target.h | 2 -- tcg/mips/tcg-target.h | 2 -- tcg/ppc/tcg-target.h | 1 - tcg/riscv/tcg-target.h | 2 -- tcg/s390x/tcg-target.h | 2 -- tcg/sparc64/tcg-target.h | 1 - tcg/tcg-internal.h | 2 ++ tcg/tci/tcg-target.h | 2 -- tcg/tcg-op.c | 20 +++++++++++--------- tcg/aarch64/tcg-target.c.inc | 5 +++++ tcg/arm/tcg-target.c.inc | 5 +++++ tcg/i386/tcg-target.c.inc | 5 +++++ tcg/loongarch64/tcg-target.c.inc | 5 +++++ tcg/mips/tcg-target.c.inc | 5 +++++ tcg/ppc/tcg-target.c.inc | 5 +++++ tcg/riscv/tcg-target.c.inc | 5 +++++ tcg/s390x/tcg-target.c.inc | 5 +++++ tcg/sparc64/tcg-target.c.inc | 5 +++++ tcg/tci/tcg-target.c.inc | 5 +++++ 22 files changed, 63 insertions(+), 26 deletions(-) diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index 3c0b0d312d..378e01d9d8 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -154,7 +154,6 @@ extern bool have_lse2; #define TCG_TARGET_HAS_cmpsel_vec 0 #define TCG_TARGET_DEFAULT_MO (0) -#define TCG_TARGET_HAS_MEMORY_BSWAP 0 #define TCG_TARGET_NEED_LDST_LABELS #define TCG_TARGET_NEED_POOL_LABELS diff --git a/tcg/arm/tcg-target.h b/tcg/arm/tcg-target.h index def2a189e6..4c2d3332d5 100644 --- a/tcg/arm/tcg-target.h +++ b/tcg/arm/tcg-target.h @@ -150,7 +150,6 @@ extern bool use_neon_instructions; #define TCG_TARGET_HAS_cmpsel_vec 0 #define TCG_TARGET_DEFAULT_MO (0) -#define TCG_TARGET_HAS_MEMORY_BSWAP 0 #define TCG_TARGET_NEED_LDST_LABELS #define TCG_TARGET_NEED_POOL_LABELS diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index 0421776cb8..8fe6958abd 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -240,9 +240,6 @@ extern bool have_atomic16; #include "tcg/tcg-mo.h" #define TCG_TARGET_DEFAULT_MO (TCG_MO_ALL & ~TCG_MO_ST_LD) - -#define TCG_TARGET_HAS_MEMORY_BSWAP have_movbe - #define TCG_TARGET_NEED_LDST_LABELS #define TCG_TARGET_NEED_POOL_LABELS diff --git a/tcg/loongarch64/tcg-target.h b/tcg/loongarch64/tcg-target.h index 17b8193aa5..75c3d80ed2 100644 --- a/tcg/loongarch64/tcg-target.h +++ b/tcg/loongarch64/tcg-target.h @@ -173,6 +173,4 @@ typedef enum { #define TCG_TARGET_NEED_LDST_LABELS -#define TCG_TARGET_HAS_MEMORY_BSWAP 0 - #endif /* LOONGARCH_TCG_TARGET_H */ diff --git a/tcg/mips/tcg-target.h b/tcg/mips/tcg-target.h index 42bd7fff01..47088af9cb 100644 --- a/tcg/mips/tcg-target.h +++ b/tcg/mips/tcg-target.h @@ -205,8 +205,6 @@ extern bool use_mips32r2_instructions; #endif #define TCG_TARGET_DEFAULT_MO 0 -#define TCG_TARGET_HAS_MEMORY_BSWAP 0 - #define TCG_TARGET_NEED_LDST_LABELS #endif diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h index af81c5a57f..d55f0266bb 100644 --- a/tcg/ppc/tcg-target.h +++ b/tcg/ppc/tcg-target.h @@ -179,7 +179,6 @@ extern bool have_vsx; #define TCG_TARGET_HAS_cmpsel_vec 0 #define TCG_TARGET_DEFAULT_MO (0) -#define TCG_TARGET_HAS_MEMORY_BSWAP 1 #define TCG_TARGET_NEED_LDST_LABELS #define TCG_TARGET_NEED_POOL_LABELS diff --git a/tcg/riscv/tcg-target.h b/tcg/riscv/tcg-target.h index dddf2486c1..dece3b3c27 100644 --- a/tcg/riscv/tcg-target.h +++ b/tcg/riscv/tcg-target.h @@ -168,6 +168,4 @@ typedef enum { #define TCG_TARGET_NEED_LDST_LABELS #define TCG_TARGET_NEED_POOL_LABELS -#define TCG_TARGET_HAS_MEMORY_BSWAP 0 - #endif diff --git a/tcg/s390x/tcg-target.h b/tcg/s390x/tcg-target.h index a05b473117..fe05680124 100644 --- a/tcg/s390x/tcg-target.h +++ b/tcg/s390x/tcg-target.h @@ -172,8 +172,6 @@ extern uint64_t s390_facilities[3]; #define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_BY_REF #define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_BY_REF -#define TCG_TARGET_HAS_MEMORY_BSWAP 1 - #define TCG_TARGET_DEFAULT_MO (TCG_MO_ALL & ~TCG_MO_ST_LD) #define TCG_TARGET_NEED_LDST_LABELS #define TCG_TARGET_NEED_POOL_LABELS diff --git a/tcg/sparc64/tcg-target.h b/tcg/sparc64/tcg-target.h index 7434cc99d4..f6cd86975a 100644 --- a/tcg/sparc64/tcg-target.h +++ b/tcg/sparc64/tcg-target.h @@ -154,7 +154,6 @@ extern bool use_vis3_instructions; #define TCG_AREG0 TCG_REG_I0 #define TCG_TARGET_DEFAULT_MO (0) -#define TCG_TARGET_HAS_MEMORY_BSWAP 1 #define TCG_TARGET_NEED_LDST_LABELS #define TCG_TARGET_NEED_POOL_LABELS diff --git a/tcg/tcg-internal.h b/tcg/tcg-internal.h index 0f1ba01a9a..67b698bd5c 100644 --- a/tcg/tcg-internal.h +++ b/tcg/tcg-internal.h @@ -126,4 +126,6 @@ static inline TCGv_i64 TCGV128_HIGH(TCGv_i128 t) return temp_tcgv_i64(tcgv_i128_temp(t) + o); } +bool tcg_target_has_memory_bswap(MemOp memop); + #endif /* TCG_INTERNAL_H */ diff --git a/tcg/tci/tcg-target.h b/tcg/tci/tcg-target.h index 7140a76a73..364012e4d2 100644 --- a/tcg/tci/tcg-target.h +++ b/tcg/tci/tcg-target.h @@ -176,6 +176,4 @@ typedef enum { We prefer consistency across hosts on this. */ #define TCG_TARGET_DEFAULT_MO (0) -#define TCG_TARGET_HAS_MEMORY_BSWAP 1 - #endif /* TCG_TARGET_H */ diff --git a/tcg/tcg-op.c b/tcg/tcg-op.c index 22481a344c..b13ded10df 100644 --- a/tcg/tcg-op.c +++ b/tcg/tcg-op.c @@ -2959,7 +2959,7 @@ void tcg_gen_qemu_ld_i32(TCGv_i32 val, TCGv addr, TCGArg idx, MemOp memop) oi = make_memop_idx(memop, idx); orig_memop = memop; - if (!TCG_TARGET_HAS_MEMORY_BSWAP && (memop & MO_BSWAP)) { + if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { memop &= ~MO_BSWAP; /* The bswap primitive benefits from zero-extended input. */ if ((memop & MO_SSIZE) == MO_SW) { @@ -2996,7 +2996,7 @@ void tcg_gen_qemu_st_i32(TCGv_i32 val, TCGv addr, TCGArg idx, MemOp memop) memop = tcg_canonicalize_memop(memop, 0, 1); oi = make_memop_idx(memop, idx); - if (!TCG_TARGET_HAS_MEMORY_BSWAP && (memop & MO_BSWAP)) { + if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { swap = tcg_temp_ebb_new_i32(); switch (memop & MO_SIZE) { case MO_16: @@ -3045,7 +3045,7 @@ void tcg_gen_qemu_ld_i64(TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop) oi = make_memop_idx(memop, idx); orig_memop = memop; - if (!TCG_TARGET_HAS_MEMORY_BSWAP && (memop & MO_BSWAP)) { + if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { memop &= ~MO_BSWAP; /* The bswap primitive benefits from zero-extended input. */ if ((memop & MO_SIGN) && (memop & MO_SIZE) < MO_64) { @@ -3091,7 +3091,7 @@ void tcg_gen_qemu_st_i64(TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop) memop = tcg_canonicalize_memop(memop, 1, 1); oi = make_memop_idx(memop, idx); - if (!TCG_TARGET_HAS_MEMORY_BSWAP && (memop & MO_BSWAP)) { + if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { swap = tcg_temp_ebb_new_i64(); switch (memop & MO_SIZE) { case MO_16: @@ -3157,11 +3157,6 @@ static void canonicalize_memop_i128_as_i64(MemOp ret[2], MemOp orig) tcg_debug_assert((orig & MO_SIZE) == MO_128); tcg_debug_assert((orig & MO_SIGN) == 0); - /* Use a memory ordering implemented by the host. */ - if (!TCG_TARGET_HAS_MEMORY_BSWAP && (orig & MO_BSWAP)) { - mop_1 &= ~MO_BSWAP; - } - /* Reduce the size to 64-bit. */ mop_1 = (mop_1 & ~MO_SIZE) | MO_64; @@ -3191,6 +3186,13 @@ static void canonicalize_memop_i128_as_i64(MemOp ret[2], MemOp orig) default: g_assert_not_reached(); } + + /* Use a memory ordering implemented by the host. */ + if ((orig & MO_BSWAP) && !tcg_target_has_memory_bswap(mop_1)) { + mop_1 &= ~MO_BSWAP; + mop_2 &= ~MO_BSWAP; + } + ret[0] = mop_1; ret[1] = mop_2; } diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index 36d8798bca..0cc719d799 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -1595,6 +1595,11 @@ typedef struct { TCGType index_ext; } HostAddress; +bool tcg_target_has_memory_bswap(MemOp memop) +{ + return false; +} + static const TCGLdstHelperParam ldst_helper_param = { .ntmp = 1, .tmp = { TCG_REG_TMP } }; diff --git a/tcg/arm/tcg-target.c.inc b/tcg/arm/tcg-target.c.inc index eb0542f32e..e5aed03247 100644 --- a/tcg/arm/tcg-target.c.inc +++ b/tcg/arm/tcg-target.c.inc @@ -1325,6 +1325,11 @@ typedef struct { bool index_scratch; } HostAddress; +bool tcg_target_has_memory_bswap(MemOp memop) +{ + return false; +} + static TCGReg ldst_ra_gen(TCGContext *s, const TCGLabelQemuLdst *l, int arg) { /* We arrive at the slow path via "BLNE", so R14 contains l->raddr. */ diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index 21553f3c39..6d55ba5a1c 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -1776,6 +1776,11 @@ typedef struct { int seg; } HostAddress; +bool tcg_target_has_memory_bswap(MemOp memop) +{ + return have_movbe; +} + /* * Because i686 has no register parameters and because x86_64 has xchg * to handle addr/data register overlap, we have placed all input arguments diff --git a/tcg/loongarch64/tcg-target.c.inc b/tcg/loongarch64/tcg-target.c.inc index 7d0165349d..d26174dde5 100644 --- a/tcg/loongarch64/tcg-target.c.inc +++ b/tcg/loongarch64/tcg-target.c.inc @@ -828,6 +828,11 @@ typedef struct { TCGReg index; } HostAddress; +bool tcg_target_has_memory_bswap(MemOp memop) +{ + return false; +} + /* * For softmmu, perform the TLB load and compare. * For useronly, perform any required alignment tests. diff --git a/tcg/mips/tcg-target.c.inc b/tcg/mips/tcg-target.c.inc index fa0f334e8d..cd0254a0d7 100644 --- a/tcg/mips/tcg-target.c.inc +++ b/tcg/mips/tcg-target.c.inc @@ -1141,6 +1141,11 @@ typedef struct { MemOp align; } HostAddress; +bool tcg_target_has_memory_bswap(MemOp memop) +{ + return false; +} + /* * For softmmu, perform the TLB load and compare. * For useronly, perform any required alignment tests. diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc index 218602c10c..b62a163014 100644 --- a/tcg/ppc/tcg-target.c.inc +++ b/tcg/ppc/tcg-target.c.inc @@ -2017,6 +2017,11 @@ typedef struct { TCGReg index; } HostAddress; +bool tcg_target_has_memory_bswap(MemOp memop) +{ + return true; +} + /* * For softmmu, perform the TLB load and compare. * For useronly, perform any required alignment tests. diff --git a/tcg/riscv/tcg-target.c.inc b/tcg/riscv/tcg-target.c.inc index 415e6c6e15..37870c89fc 100644 --- a/tcg/riscv/tcg-target.c.inc +++ b/tcg/riscv/tcg-target.c.inc @@ -853,6 +853,11 @@ static void tcg_out_goto(TCGContext *s, const tcg_insn_unit *target) tcg_debug_assert(ok); } +bool tcg_target_has_memory_bswap(MemOp memop) +{ + return false; +} + /* We have three temps, we might as well expose them. */ static const TCGLdstHelperParam ldst_helper_param = { .ntmp = 3, .tmp = { TCG_REG_TMP0, TCG_REG_TMP1, TCG_REG_TMP2 } diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc index de8aed5f77..22f0206b5a 100644 --- a/tcg/s390x/tcg-target.c.inc +++ b/tcg/s390x/tcg-target.c.inc @@ -1574,6 +1574,11 @@ typedef struct { int disp; } HostAddress; +bool tcg_target_has_memory_bswap(MemOp memop) +{ + return true; +} + static void tcg_out_qemu_ld_direct(TCGContext *s, MemOp opc, TCGReg data, HostAddress h) { diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index 0237188d65..bb23038529 100644 --- a/tcg/sparc64/tcg-target.c.inc +++ b/tcg/sparc64/tcg-target.c.inc @@ -1011,6 +1011,11 @@ typedef struct { TCGReg index; } HostAddress; +bool tcg_target_has_memory_bswap(MemOp memop) +{ + return true; +} + /* * For softmmu, perform the TLB load and compare. * For useronly, perform any required alignment tests. diff --git a/tcg/tci/tcg-target.c.inc b/tcg/tci/tcg-target.c.inc index 4cf03a579c..41fbf042da 100644 --- a/tcg/tci/tcg-target.c.inc +++ b/tcg/tci/tcg-target.c.inc @@ -963,3 +963,8 @@ static void tcg_target_init(TCGContext *s) static inline void tcg_target_qemu_prologue(TCGContext *s) { } + +bool tcg_target_has_memory_bswap(MemOp memop) +{ + return true; +} From patchwork Tue May 16 19:40:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682381 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp17996wrt; Tue, 16 May 2023 12:48:50 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ61y5ni3STyFj5W2bicDd6Shwo4HO4JNSkeqEZ+XfcGrxSqwYeH7iAm56lK/ZMSMqqT5P7A X-Received: by 2002:a05:6214:501b:b0:5ef:45a7:a3c0 with SMTP id jo27-20020a056214501b00b005ef45a7a3c0mr60563463qvb.27.1684266530712; Tue, 16 May 2023 12:48:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266530; cv=none; d=google.com; s=arc-20160816; b=biMMxgjhaHsfSwrPZ/AoCmDpdMQwYKQrhIhbXgc4NPWFIEbystb8c7GnqtmEV9M5ie H6ibrCDjP2P+2yRO3s8pB8F7obatz5LIwZ/8CejMJ8igoaCPjW80m0Whdx01sGtjFHj1 IWQ2wn9esjlT8ZQC3FmVk2FVDGP2VXpfCD6c3vorUArgxfDkUAcoMCx575uYLcd5dmGy rEgQ5cAZ/6MNLtEQoF1+NFFKbUEhnM3a1cJ8Q0evAzKeoUhFe9bFQPyNbWm3Aij6c+GF oMmvOCMillqgJotvRDY6QNb/fHry+PxMG8jR8xUAWAMKJYuJ9jdXZJK+pF8iHe8b6vZF 8aqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=XdPzDSOCHvvRCvemyVsghgHnaE6R9G/vTXbZ+3RusvY=; b=bhfMpA0hMnKOadD96pX2VW2Ib33DCETTw29tuLvRbMo7ZuT7TaBmoylNJPAKgxxcfd 1gFvXLFDL6oSwF6zsc+M5sHI/Wj3gqLmwoq1z1EnMG52HkRiVf8p5KkuUrZhsGvMs9WW G2CTJZwuPttTUGjXtw0BhHm048lb+p66kuMXKSURPar9QnKs7Az6tO2vjAAu7dF1Wpm0 HhzPiKlFVzZ1+KuOliEG4MT5aRhk1TuFPjVDZ+Pi9PLbpuQoXNwkTBQ2n93y1u9DOKcz Dbzya+kXp79NBBm7kk6UNQsD6RhfFJdoQdBLl3L5Rwm3nHqWbcXn1h6ESxFdPuoWWMqu Q9MA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hazqHjCg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id ey2-20020a0562140b6200b0056ea3d10d19si12167881qvb.396.2023.05.16.12.48.50 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:48:50 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hazqHjCg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Z0-0006og-Co; Tue, 16 May 2023 15:42:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yx-0006nw-Ee for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:23 -0400 Received: from mail-pg1-x532.google.com ([2607:f8b0:4864:20::532]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yr-0002to-Rs for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:23 -0400 Received: by mail-pg1-x532.google.com with SMTP id 41be03b00d2f7-53033a0b473so7581412a12.0 for ; Tue, 16 May 2023 12:42:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266136; x=1686858136; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XdPzDSOCHvvRCvemyVsghgHnaE6R9G/vTXbZ+3RusvY=; b=hazqHjCg2AgDddPmeGEp4/OXNdjzrdgE/o7lOg6zhkkZqXGPw29sJXUGhoh2/+R2Qr l+W+/8iuSN5LunG/L6XI4qxRKL7SKpqrSiuzeSp5rpXtReG/HoV32vHL5cg8AQN4lV9t lKXfv5tD154LAOceGMFyeCl/J2asiH1gOnwBhwJiYGWZmLcYqIziHRAgzG3/7TykzfUl ip8k+m4ae1IYx48zXi1rJVqgVRuGkvB/HK5UKrpYKZsrhOc70JbX/RImbRLwhNhVvGiB neGa7ww+qDiwnSRzYzCAqF69T75D0RyMbol91Fwe8gs7AC4Hs/iuoqiGSRSi3/r1otU3 twuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266136; x=1686858136; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XdPzDSOCHvvRCvemyVsghgHnaE6R9G/vTXbZ+3RusvY=; b=Zaa9bpFT2GSDEXxm+9FpDS4OLvLQMV0LZg2Dzu0DOBSS82r8kF8c0p96aR9oj81Oie y1DS1ueE/R2YFlFwkWKUtrJV5LIVagnj/9m3QxGeUK4c9i2s+qz8k3ZDr6yv9hTFtdHp IQn6J6byZ9l4d2nHAv0Lx35fx8iWT4yyfoG8LnQOK3Dg2be6mNfrsGaM4JE9m2V+tfVa VQ78sGRhE66z7S1CieI/fvATwADyZyUfErjDBgNfCxjD4T9vbZy17kqmQnTfnvmluuhN vs94g6guhdgGrmcIK6D1xctcCsHm3sDB1HMWuUi4rYMO4Vs7Am8Ou/ncQn09QEpTRFRn xpgQ== X-Gm-Message-State: AC+VfDw52EddT7uKRAn5uf9uXmKgyR396xV+PsGy5e/29eSuEQE22M82 HHyftcDz2FbKAbu4idy5WCj/XOgIbsM0ekiAHdg= X-Received: by 2002:a05:6a21:6d97:b0:101:d031:9a41 with SMTP id wl23-20020a056a216d9700b00101d0319a41mr31136216pzb.24.1684266136497; Tue, 16 May 2023 12:42:16 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:16 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell , =?utf-8?q?Philippe_Mathieu-Dau?= =?utf-8?q?d=C3=A9?= Subject: [PULL 33/80] tcg: Add INDEX_op_qemu_{ld,st}_i128 Date: Tue, 16 May 2023 12:40:58 -0700 Message-Id: <20230516194145.1749305-34-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::532; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x532.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, T_SPF_HELO_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Add opcodes for backend support for 128-bit memory operations. Reviewed-by: Peter Maydell Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- docs/devel/tcg-ops.rst | 11 +++--- include/tcg/tcg-opc.h | 8 +++++ tcg/aarch64/tcg-target.h | 2 ++ tcg/arm/tcg-target.h | 2 ++ tcg/i386/tcg-target.h | 2 ++ tcg/loongarch64/tcg-target.h | 1 + tcg/mips/tcg-target.h | 2 ++ tcg/ppc/tcg-target.h | 2 ++ tcg/riscv/tcg-target.h | 2 ++ tcg/s390x/tcg-target.h | 2 ++ tcg/sparc64/tcg-target.h | 2 ++ tcg/tci/tcg-target.h | 2 ++ tcg/tcg-op.c | 69 ++++++++++++++++++++++++++++++++---- tcg/tcg.c | 6 ++++ 14 files changed, 103 insertions(+), 10 deletions(-) diff --git a/docs/devel/tcg-ops.rst b/docs/devel/tcg-ops.rst index f3f451b77f..6a166c5665 100644 --- a/docs/devel/tcg-ops.rst +++ b/docs/devel/tcg-ops.rst @@ -672,19 +672,20 @@ QEMU specific operations | This operation is optional. If the TCG backend does not implement the goto_ptr opcode, emitting this op is equivalent to emitting exit_tb(0). - * - qemu_ld_i32/i64 *t0*, *t1*, *flags*, *memidx* + * - qemu_ld_i32/i64/i128 *t0*, *t1*, *flags*, *memidx* - qemu_st_i32/i64 *t0*, *t1*, *flags*, *memidx* + qemu_st_i32/i64/i128 *t0*, *t1*, *flags*, *memidx* qemu_st8_i32 *t0*, *t1*, *flags*, *memidx* - | Load data at the guest address *t1* into *t0*, or store data in *t0* at guest - address *t1*. The _i32/_i64 size applies to the size of the input/output + address *t1*. The _i32/_i64/_i128 size applies to the size of the input/output register *t0* only. The address *t1* is always sized according to the guest, and the width of the memory operation is controlled by *flags*. | | Both *t0* and *t1* may be split into little-endian ordered pairs of registers - if dealing with 64-bit quantities on a 32-bit host. + if dealing with 64-bit quantities on a 32-bit host, or 128-bit quantities on + a 64-bit host. | | The *memidx* selects the qemu tlb index to use (e.g. user or kernel access). The flags are the MemOp bits, selecting the sign, width, and endianness @@ -693,6 +694,8 @@ QEMU specific operations | For a 32-bit host, qemu_ld/st_i64 is guaranteed to only be used with a 64-bit memory access specified in *flags*. | + | For qemu_ld/st_i128, these are only supported for a 64-bit host. + | | For i386, qemu_st8_i32 is exactly like qemu_st_i32, except the size of the memory operation is known to be 8-bit. This allows the backend to provide a different set of register constraints. diff --git a/include/tcg/tcg-opc.h b/include/tcg/tcg-opc.h index dd444734d9..94cf7c5d6a 100644 --- a/include/tcg/tcg-opc.h +++ b/include/tcg/tcg-opc.h @@ -213,6 +213,14 @@ DEF(qemu_st8_i32, 0, TLADDR_ARGS + 1, 1, TCG_OPF_CALL_CLOBBER | TCG_OPF_SIDE_EFFECTS | IMPL(TCG_TARGET_HAS_qemu_st8_i32)) +/* Only for 64-bit hosts at the moment. */ +DEF(qemu_ld_i128, 2, 1, 1, + TCG_OPF_CALL_CLOBBER | TCG_OPF_SIDE_EFFECTS | TCG_OPF_64BIT | + IMPL(TCG_TARGET_HAS_qemu_ldst_i128)) +DEF(qemu_st_i128, 0, 3, 1, + TCG_OPF_CALL_CLOBBER | TCG_OPF_SIDE_EFFECTS | TCG_OPF_64BIT | + IMPL(TCG_TARGET_HAS_qemu_ldst_i128)) + /* Host vector support. */ #define IMPLVEC TCG_OPF_VECTOR | IMPL(TCG_TARGET_MAYBE_vec) diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index 378e01d9d8..74ee2ed255 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -129,6 +129,8 @@ extern bool have_lse2; #define TCG_TARGET_HAS_muluh_i64 1 #define TCG_TARGET_HAS_mulsh_i64 1 +#define TCG_TARGET_HAS_qemu_ldst_i128 0 + #define TCG_TARGET_HAS_v64 1 #define TCG_TARGET_HAS_v128 1 #define TCG_TARGET_HAS_v256 0 diff --git a/tcg/arm/tcg-target.h b/tcg/arm/tcg-target.h index 4c2d3332d5..65efc538f4 100644 --- a/tcg/arm/tcg-target.h +++ b/tcg/arm/tcg-target.h @@ -125,6 +125,8 @@ extern bool use_neon_instructions; #define TCG_TARGET_HAS_rem_i32 0 #define TCG_TARGET_HAS_qemu_st8_i32 0 +#define TCG_TARGET_HAS_qemu_ldst_i128 0 + #define TCG_TARGET_HAS_v64 use_neon_instructions #define TCG_TARGET_HAS_v128 use_neon_instructions #define TCG_TARGET_HAS_v256 0 diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index 8fe6958abd..943af6775e 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -194,6 +194,8 @@ extern bool have_atomic16; #define TCG_TARGET_HAS_qemu_st8_i32 1 #endif +#define TCG_TARGET_HAS_qemu_ldst_i128 0 + /* We do not support older SSE systems, only beginning with AVX1. */ #define TCG_TARGET_HAS_v64 have_avx1 #define TCG_TARGET_HAS_v128 have_avx1 diff --git a/tcg/loongarch64/tcg-target.h b/tcg/loongarch64/tcg-target.h index 75c3d80ed2..482901ac15 100644 --- a/tcg/loongarch64/tcg-target.h +++ b/tcg/loongarch64/tcg-target.h @@ -168,6 +168,7 @@ typedef enum { #define TCG_TARGET_HAS_muls2_i64 0 #define TCG_TARGET_HAS_muluh_i64 1 #define TCG_TARGET_HAS_mulsh_i64 1 +#define TCG_TARGET_HAS_qemu_ldst_i128 0 #define TCG_TARGET_DEFAULT_MO (0) diff --git a/tcg/mips/tcg-target.h b/tcg/mips/tcg-target.h index 47088af9cb..7277a117ef 100644 --- a/tcg/mips/tcg-target.h +++ b/tcg/mips/tcg-target.h @@ -204,6 +204,8 @@ extern bool use_mips32r2_instructions; #define TCG_TARGET_HAS_ext16u_i64 0 /* andi rt, rs, 0xffff */ #endif +#define TCG_TARGET_HAS_qemu_ldst_i128 0 + #define TCG_TARGET_DEFAULT_MO 0 #define TCG_TARGET_NEED_LDST_LABELS diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h index d55f0266bb..0914380bd7 100644 --- a/tcg/ppc/tcg-target.h +++ b/tcg/ppc/tcg-target.h @@ -149,6 +149,8 @@ extern bool have_vsx; #define TCG_TARGET_HAS_mulsh_i64 1 #endif +#define TCG_TARGET_HAS_qemu_ldst_i128 0 + /* * While technically Altivec could support V64, it has no 64-bit store * instruction and substituting two 32-bit stores makes the generated diff --git a/tcg/riscv/tcg-target.h b/tcg/riscv/tcg-target.h index dece3b3c27..494c986b49 100644 --- a/tcg/riscv/tcg-target.h +++ b/tcg/riscv/tcg-target.h @@ -163,6 +163,8 @@ typedef enum { #define TCG_TARGET_HAS_muluh_i64 1 #define TCG_TARGET_HAS_mulsh_i64 1 +#define TCG_TARGET_HAS_qemu_ldst_i128 0 + #define TCG_TARGET_DEFAULT_MO (0) #define TCG_TARGET_NEED_LDST_LABELS diff --git a/tcg/s390x/tcg-target.h b/tcg/s390x/tcg-target.h index fe05680124..170007bea5 100644 --- a/tcg/s390x/tcg-target.h +++ b/tcg/s390x/tcg-target.h @@ -140,6 +140,8 @@ extern uint64_t s390_facilities[3]; #define TCG_TARGET_HAS_muluh_i64 0 #define TCG_TARGET_HAS_mulsh_i64 0 +#define TCG_TARGET_HAS_qemu_ldst_i128 0 + #define TCG_TARGET_HAS_v64 HAVE_FACILITY(VECTOR) #define TCG_TARGET_HAS_v128 HAVE_FACILITY(VECTOR) #define TCG_TARGET_HAS_v256 0 diff --git a/tcg/sparc64/tcg-target.h b/tcg/sparc64/tcg-target.h index f6cd86975a..31c5537379 100644 --- a/tcg/sparc64/tcg-target.h +++ b/tcg/sparc64/tcg-target.h @@ -151,6 +151,8 @@ extern bool use_vis3_instructions; #define TCG_TARGET_HAS_muluh_i64 use_vis3_instructions #define TCG_TARGET_HAS_mulsh_i64 0 +#define TCG_TARGET_HAS_qemu_ldst_i128 0 + #define TCG_AREG0 TCG_REG_I0 #define TCG_TARGET_DEFAULT_MO (0) diff --git a/tcg/tci/tcg-target.h b/tcg/tci/tcg-target.h index 364012e4d2..28dc6d5cfc 100644 --- a/tcg/tci/tcg-target.h +++ b/tcg/tci/tcg-target.h @@ -127,6 +127,8 @@ #define TCG_TARGET_HAS_mulu2_i32 1 #endif /* TCG_TARGET_REG_BITS == 64 */ +#define TCG_TARGET_HAS_qemu_ldst_i128 0 + /* Number of registers available. */ #define TCG_TARGET_NB_REGS 16 diff --git a/tcg/tcg-op.c b/tcg/tcg-op.c index b13ded10df..c419228cc4 100644 --- a/tcg/tcg-op.c +++ b/tcg/tcg-op.c @@ -3205,7 +3205,7 @@ static void canonicalize_memop_i128_as_i64(MemOp ret[2], MemOp orig) void tcg_gen_qemu_ld_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) { - MemOpIdx oi = make_memop_idx(memop, idx); + const MemOpIdx oi = make_memop_idx(memop, idx); tcg_debug_assert((memop & MO_SIZE) == MO_128); tcg_debug_assert((memop & MO_SIGN) == 0); @@ -3213,9 +3213,36 @@ void tcg_gen_qemu_ld_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) tcg_gen_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); addr = plugin_prep_mem_callbacks(addr); - /* TODO: allow the tcg backend to see the whole operation. */ + /* TODO: For now, force 32-bit hosts to use the helper. */ + if (TCG_TARGET_HAS_qemu_ldst_i128 && TCG_TARGET_REG_BITS == 64) { + TCGv_i64 lo, hi; + TCGArg addr_arg; + MemOpIdx adj_oi; + bool need_bswap = false; - if (use_two_i64_for_i128(memop)) { + if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { + lo = TCGV128_HIGH(val); + hi = TCGV128_LOW(val); + adj_oi = make_memop_idx(memop & ~MO_BSWAP, idx); + need_bswap = true; + } else { + lo = TCGV128_LOW(val); + hi = TCGV128_HIGH(val); + adj_oi = oi; + } + +#if TARGET_LONG_BITS == 32 + addr_arg = tcgv_i32_arg(addr); +#else + addr_arg = tcgv_i64_arg(addr); +#endif + tcg_gen_op4ii_i64(INDEX_op_qemu_ld_i128, lo, hi, addr_arg, adj_oi); + + if (need_bswap) { + tcg_gen_bswap64_i64(lo, lo); + tcg_gen_bswap64_i64(hi, hi); + } + } else if (use_two_i64_for_i128(memop)) { MemOp mop[2]; TCGv addr_p8; TCGv_i64 x, y; @@ -3258,7 +3285,7 @@ void tcg_gen_qemu_ld_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) void tcg_gen_qemu_st_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) { - MemOpIdx oi = make_memop_idx(memop, idx); + const MemOpIdx oi = make_memop_idx(memop, idx); tcg_debug_assert((memop & MO_SIZE) == MO_128); tcg_debug_assert((memop & MO_SIGN) == 0); @@ -3266,9 +3293,39 @@ void tcg_gen_qemu_st_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) tcg_gen_req_mo(TCG_MO_ST_LD | TCG_MO_ST_ST); addr = plugin_prep_mem_callbacks(addr); - /* TODO: allow the tcg backend to see the whole operation. */ + /* TODO: For now, force 32-bit hosts to use the helper. */ - if (use_two_i64_for_i128(memop)) { + if (TCG_TARGET_HAS_qemu_ldst_i128 && TCG_TARGET_REG_BITS == 64) { + TCGv_i64 lo, hi; + TCGArg addr_arg; + MemOpIdx adj_oi; + bool need_bswap = false; + + if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { + lo = tcg_temp_new_i64(); + hi = tcg_temp_new_i64(); + tcg_gen_bswap64_i64(lo, TCGV128_HIGH(val)); + tcg_gen_bswap64_i64(hi, TCGV128_LOW(val)); + adj_oi = make_memop_idx(memop & ~MO_BSWAP, idx); + need_bswap = true; + } else { + lo = TCGV128_LOW(val); + hi = TCGV128_HIGH(val); + adj_oi = oi; + } + +#if TARGET_LONG_BITS == 32 + addr_arg = tcgv_i32_arg(addr); +#else + addr_arg = tcgv_i64_arg(addr); +#endif + tcg_gen_op4ii_i64(INDEX_op_qemu_st_i128, lo, hi, addr_arg, adj_oi); + + if (need_bswap) { + tcg_temp_free_i64(lo); + tcg_temp_free_i64(hi); + } + } else if (use_two_i64_for_i128(memop)) { MemOp mop[2]; TCGv addr_p8; TCGv_i64 x, y; diff --git a/tcg/tcg.c b/tcg/tcg.c index a864ff1f4b..44b8c700a7 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -1735,6 +1735,10 @@ bool tcg_op_supported(TCGOpcode op) case INDEX_op_qemu_st8_i32: return TCG_TARGET_HAS_qemu_st8_i32; + case INDEX_op_qemu_ld_i128: + case INDEX_op_qemu_st_i128: + return TCG_TARGET_HAS_qemu_ldst_i128; + case INDEX_op_mov_i32: case INDEX_op_setcond_i32: case INDEX_op_brcond_i32: @@ -2357,6 +2361,8 @@ static void tcg_dump_ops(TCGContext *s, FILE *f, bool have_prefs) case INDEX_op_qemu_st8_i32: case INDEX_op_qemu_ld_i64: case INDEX_op_qemu_st_i64: + case INDEX_op_qemu_ld_i128: + case INDEX_op_qemu_st_i128: { const char *s_al, *s_op, *s_at; MemOpIdx oi = op->args[k++]; From patchwork Tue May 16 19:40:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682368 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp639277ltp; Tue, 16 May 2023 12:44:11 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5lDJSElOVBKmKxU97HyUYVyatHym8LCFtLvWBiPp9jr5Vfv1dVin+luthgjNEnM2ulrtn5 X-Received: by 2002:a05:6214:2a8b:b0:621:1808:e93a with SMTP id jr11-20020a0562142a8b00b006211808e93amr51092213qvb.42.1684266251715; Tue, 16 May 2023 12:44:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266251; cv=none; d=google.com; s=arc-20160816; b=ReBISGDVAQYeR2Hlvo94OiwezTb/rRTe7jDTNZd0vCDXuZ+YCWG0O7hB+DGH2LVsj3 Q+3mA4TRU/6+LWCI5yu3/GKo2ef0djIgJ+VDjcuDBpp2x2k200z6abv4tatLInrLKi5k cICmqis+SV6ghgJD2rCSohWVpREGSvB2Rot+8BQprSFPI2QBpriowSyArbWSXTrdrzMq V+9MTwIVOjiNHBMI0pysocLEjsQA2+ws+wZ/9t9zhoHPAJJrFYU1LwB7Wu+3MUc2Y4kv U+HajYYJtCqFQPtD2Aq6cZ48qQRzzAsUwQ8JIUOSMqreiMmqaJW2lMbMXKMnDC4mBVNG m8Dg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=kD4l8epPfORT7xhdho6r//UErZNcfQbpb7324wXt3a4=; b=rdDy4s2wjfv66sDhc1gnVyJkcAl0dBZoDX6m94sl3Q0aLwmATHIbMQald1f1Y/PHea W9e2Ho8lP3f3Mkvzre/Myy4IKLsc6q/zJdaZDQyBcPBvnzyvSmR41bWVm9Kyzd1wDmvC o2TwmB9weDI+ZvENST6ux9WNuMigE1YcQlEEG2F6els43z6sYlTgvvK9hZnoG6ZDCmF0 4oYgRO1gWmoeQkrU+dofR07enySL/7gQyUW7BAvJwXgrr/Rw4Tzh7RoxrL0OHj1nswnk zIXN77ETltsPaDuD72XZAR53zTWzzkBdBqpztsamFj6P8TayPseQsgNrSO2lw+fXzpFg qa7Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="c/C67CUh"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id ke11-20020a056214300b00b005f16a6cce13si11857770qvb.573.2023.05.16.12.44.11 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:44:11 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="c/C67CUh"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yv-0006nC-PD; Tue, 16 May 2023 15:42:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yu-0006mm-Bb for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:20 -0400 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Ys-0002o7-E8 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:20 -0400 Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-64384c6797eso11570558b3a.2 for ; Tue, 16 May 2023 12:42:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266137; x=1686858137; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kD4l8epPfORT7xhdho6r//UErZNcfQbpb7324wXt3a4=; b=c/C67CUhEa5yOSBg+BNaNKAn+XzlPiugXtW9Ek5KjIis4Y6yRsYcKCKoCfyZ4PzhQx 20ll2WA4BHP9ryt/sBfWWG8zJl+wbsoHJVrmue6zrTEjgGCA+xaz1Pc0IgMyuY6SJgfA E8lMIV7ESqvR7A5/OQg2e7qh5smpLSO+NBNzLwaloAHztg0p/X+Nsb0kSCWv7FmWnMNQ h7AXA8WIEJM0RAgT693KHE6pqmrA+zTTq5kSj8ppPlksLsSRgsp5ZkwHpBmlyNVqX4W/ EhLgLbu5YRdnFjqR/Ifg4vRMdoNUjgpTOHeGU0O7Fj/tDV0e2bRt//VfJUAjMLyTtWra sr2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266137; x=1686858137; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kD4l8epPfORT7xhdho6r//UErZNcfQbpb7324wXt3a4=; b=mG0aH6emJSAx/gnUdX8jyqcv3s6pLpt+NTwuAslBskqflX6NKmVCRWJLINUeY6bDm2 /q87f7ScFjGkQx1jrpIMru06PJiEUHjvKjCiOhNA6gLCi+odme4DwfyXLiVH75A4xaPf O0vzqBrASKXHtpZ6yzppYfBVwCt+Wl5fFpfkwfQXayhQ9OXUtHz4l/5mouthbjupnwLn 2iQ+hwjtImso/7Tyk6UtiurjfPcSAnKiV1TRRJHhlDsv2aVGJ0YdgYjXfGyPvV6Gkg6T L2RlrKVL9cj5DXzh+zxZ1/ZO9KBewmupLRmoa56nTpQOOoCQ0ye3d4uRHY6T8p1ZCNCX 5hWQ== X-Gm-Message-State: AC+VfDyw3k/nhOcqivNINILGNdbVH8xgOyH9g0JtAzpjMZytSZ0fHTVi JRO1TTvVIlwD/VY9vYFIdMRa9jtvuftwo7MoXxY= X-Received: by 2002:a05:6a00:1582:b0:643:b37d:d350 with SMTP id u2-20020a056a00158200b00643b37dd350mr51882112pfk.31.1684266137618; Tue, 16 May 2023 12:42:17 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:17 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 34/80] tcg: Introduce tcg_out_movext3 Date: Tue, 16 May 2023 12:40:59 -0700 Message-Id: <20230516194145.1749305-35-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42c; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org With x86_64 as host, we do not have any temporaries with which to resolve cycles, but we do have xchg. As a side bonus, the set of graphs that can be made with 3 nodes and all nodes conflicting is small: two. We can solve the cycle with a single temp. This is required for x86_64 to handle stores of i128: 1 address register and 2 data registers. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/tcg.c | 138 ++++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 108 insertions(+), 30 deletions(-) diff --git a/tcg/tcg.c b/tcg/tcg.c index 44b8c700a7..5de9380c6b 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -532,6 +532,82 @@ static void tcg_out_movext2(TCGContext *s, const TCGMovExtend *i1, tcg_out_movext1_new_src(s, i1, src1); } +/** + * tcg_out_movext3 -- move and extend three pair + * @s: tcg context + * @i1: first move description + * @i2: second move description + * @i3: third move description + * @scratch: temporary register, or -1 for none + * + * As tcg_out_movext, for all of @i1, @i2 and @i3, caring for overlap + * between the sources and destinations. + */ + +static void tcg_out_movext3(TCGContext *s, const TCGMovExtend *i1, + const TCGMovExtend *i2, const TCGMovExtend *i3, + int scratch) +{ + TCGReg src1 = i1->src; + TCGReg src2 = i2->src; + TCGReg src3 = i3->src; + + if (i1->dst != src2 && i1->dst != src3) { + tcg_out_movext1(s, i1); + tcg_out_movext2(s, i2, i3, scratch); + return; + } + if (i2->dst != src1 && i2->dst != src3) { + tcg_out_movext1(s, i2); + tcg_out_movext2(s, i1, i3, scratch); + return; + } + if (i3->dst != src1 && i3->dst != src2) { + tcg_out_movext1(s, i3); + tcg_out_movext2(s, i1, i2, scratch); + return; + } + + /* + * There is a cycle. Since there are only 3 nodes, the cycle is + * either "clockwise" or "anti-clockwise", and can be solved with + * a single scratch or two xchg. + */ + if (i1->dst == src2 && i2->dst == src3 && i3->dst == src1) { + /* "Clockwise" */ + if (tcg_out_xchg(s, MAX(i1->src_type, i2->src_type), src1, src2)) { + tcg_out_xchg(s, MAX(i2->src_type, i3->src_type), src2, src3); + /* The data is now in the correct registers, now extend. */ + tcg_out_movext1_new_src(s, i1, i1->dst); + tcg_out_movext1_new_src(s, i2, i2->dst); + tcg_out_movext1_new_src(s, i3, i3->dst); + } else { + tcg_debug_assert(scratch >= 0); + tcg_out_mov(s, i1->src_type, scratch, src1); + tcg_out_movext1(s, i3); + tcg_out_movext1(s, i2); + tcg_out_movext1_new_src(s, i1, scratch); + } + } else if (i1->dst == src3 && i2->dst == src1 && i3->dst == src2) { + /* "Anti-clockwise" */ + if (tcg_out_xchg(s, MAX(i2->src_type, i3->src_type), src2, src3)) { + tcg_out_xchg(s, MAX(i1->src_type, i2->src_type), src1, src2); + /* The data is now in the correct registers, now extend. */ + tcg_out_movext1_new_src(s, i1, i1->dst); + tcg_out_movext1_new_src(s, i2, i2->dst); + tcg_out_movext1_new_src(s, i3, i3->dst); + } else { + tcg_debug_assert(scratch >= 0); + tcg_out_mov(s, i1->src_type, scratch, src1); + tcg_out_movext1(s, i2); + tcg_out_movext1(s, i3); + tcg_out_movext1_new_src(s, i1, scratch); + } + } else { + g_assert_not_reached(); + } +} + #define C_PFX1(P, A) P##A #define C_PFX2(P, A, B) P##A##_##B #define C_PFX3(P, A, B, C) P##A##_##B##_##C @@ -5151,46 +5227,48 @@ static int tcg_out_helper_stk_ofs(TCGType type, unsigned slot) static void tcg_out_helper_load_regs(TCGContext *s, unsigned nmov, TCGMovExtend *mov, - unsigned ntmp, const int *tmp) + const TCGLdstHelperParam *parm) { + TCGReg dst3; + switch (nmov) { - default: + case 4: /* The backend must have provided enough temps for the worst case. */ - tcg_debug_assert(ntmp + 1 >= nmov); + tcg_debug_assert(parm->ntmp >= 2); - for (unsigned i = nmov - 1; i >= 2; --i) { - TCGReg dst = mov[i].dst; + dst3 = mov[3].dst; + for (unsigned j = 0; j < 3; ++j) { + if (dst3 == mov[j].src) { + /* + * Conflict. Copy the source to a temporary, perform the + * remaining moves, then the extension from our scratch + * on the way out. + */ + TCGReg scratch = parm->tmp[1]; - for (unsigned j = 0; j < i; ++j) { - if (dst == mov[j].src) { - /* - * Conflict. - * Copy the source to a temporary, recurse for the - * remaining moves, perform the extension from our - * scratch on the way out. - */ - TCGReg scratch = tmp[--ntmp]; - tcg_out_mov(s, mov[i].src_type, scratch, mov[i].src); - mov[i].src = scratch; - - tcg_out_helper_load_regs(s, i, mov, ntmp, tmp); - tcg_out_movext1(s, &mov[i]); - return; - } + tcg_out_mov(s, mov[3].src_type, scratch, mov[3].src); + tcg_out_movext3(s, mov, mov + 1, mov + 2, parm->tmp[0]); + tcg_out_movext1_new_src(s, &mov[3], scratch); + break; } - - /* No conflicts: perform this move and continue. */ - tcg_out_movext1(s, &mov[i]); } - /* fall through for the final two moves */ + /* No conflicts: perform this move and continue. */ + tcg_out_movext1(s, &mov[3]); + /* fall through */ + + case 3: + tcg_out_movext3(s, mov, mov + 1, mov + 2, + parm->ntmp ? parm->tmp[0] : -1); + break; case 2: - tcg_out_movext2(s, mov, mov + 1, ntmp ? tmp[0] : -1); - return; + tcg_out_movext2(s, mov, mov + 1, + parm->ntmp ? parm->tmp[0] : -1); + break; case 1: tcg_out_movext1(s, mov); - return; - case 0: + break; + default: g_assert_not_reached(); } } @@ -5237,7 +5315,7 @@ static void tcg_out_helper_load_slots(TCGContext *s, for (i = 0; i < nmov; ++i) { mov[i].dst = tcg_target_call_iarg_regs[mov[i].dst]; } - tcg_out_helper_load_regs(s, nmov, mov, parm->ntmp, parm->tmp); + tcg_out_helper_load_regs(s, nmov, mov, parm); } static void tcg_out_helper_load_imm(TCGContext *s, unsigned slot, From patchwork Tue May 16 19:41:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682363 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp639006ltp; Tue, 16 May 2023 12:43:35 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4kQyzlXwLU8HfbWJTCaACRcHeFNJ7Q41NWR5/DkZfjfneukX6ND19RERcQkX0IPGJSkExJ X-Received: by 2002:ac8:5c15:0:b0:3f5:230a:44 with SMTP id i21-20020ac85c15000000b003f5230a0044mr15642049qti.61.1684266215461; Tue, 16 May 2023 12:43:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266215; cv=none; d=google.com; s=arc-20160816; b=WY/bYKz16kcXqke0DqDbzjZ9Vqcma0zZj36J31RGtKiaCIlV1J0ypeVM9kig+PFgVl X7jqEKfL2MI14KYhfpa9nSWq/frhqKjFz3YRyTI5Yq1r8k8ildF3tTFgyudhpN5r31W2 cqIeNwfq6VSh0Gt6GpU6CxvJh05vgWZ0JKmogyR99iBwU8BQRPbSejfgp+y7mPghMLrv B8nbmxPDUaCXSalv9c/8gM5sfG0iMNCsnzxbfAPjoRLWoYeYwCS6co0gpjENAviVIX4h 4Gwb5eZQgFUAeumaiiu4CgEfTcbgJixYkfbC0zRlpOLTfwM0QV+r4y7EPPOMtdtuqWoz Z27Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=1hA2GUGH30ti8Xg9mqUcD6D2A4Cc3BlkQs5kBCnxI8w=; b=nhfaPGK9dh2+sQUWJxbvieFGshwhembaJ4jVEIopjHQRsLlJN8F/I9VhrTUyEKw985 bT0/ZfEHt5/dsUDYDCQJnKq7mPGT2tn1zl9utwFHqAw7kIhIWjFTDt6fQvZIaq7xzUHb 2FA9hTxjdaueioga2MZVQ3e+1PRkl+IrFhl8FGAQvxlRlVjeC1xd6h/KKzBsMrmeNlgi BgcZMp5wxakh4JKgN7Yk1bKexZXSnmeHWC6dw03Rq2mnyC83tdpPgNtrD/mvkgf6ZV4s 7MZdPS3KUZKJTbFEPhaAS+7Pun5irhtezM6r5N5JDe/xkVw5ov0x5Pkce2PtYxH9p3sX d8sw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=zNrNJ8Pv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a18-20020ac85b92000000b003f0842d3c40si12040181qta.703.2023.05.16.12.43.35 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:43:35 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=zNrNJ8Pv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yw-0006nZ-V0; Tue, 16 May 2023 15:42:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yv-0006nA-Ly for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:21 -0400 Received: from mail-pf1-x433.google.com ([2607:f8b0:4864:20::433]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yt-0002uI-Nd for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:21 -0400 Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-64384c6797eso11570566b3a.2 for ; Tue, 16 May 2023 12:42:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266138; x=1686858138; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1hA2GUGH30ti8Xg9mqUcD6D2A4Cc3BlkQs5kBCnxI8w=; b=zNrNJ8PvJG3zv2inwkyhs3XvNIXnxroA6s+TdPBeZeOZIEoEH5b7EParXxY6fn6MJs Bd4VIt7EDBJmRTK4UNLyFfVe3aJui1kQYnEwyom3hyRdNYeUbf79CHzWzCVZVdHJs5O4 VwStKP6G2hw56CtzKdf6enF+2Z9gmlSxPWpKmHVEtV6HB6eWHQIqO2RMsijiLsz5zI27 JQM/Gisemi9eNuM+fFLYOMKGhbzbeBF6WUx6/nm06djh77J0Sli1NGTZarWHux/K8jGr khsLEZn9hElwt8QUOVq8yl8r/NcPjPM/Q0ZEMjfQMNAoWM3L3jdHd1fudGT6EV+q8auF YGBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266138; x=1686858138; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1hA2GUGH30ti8Xg9mqUcD6D2A4Cc3BlkQs5kBCnxI8w=; b=LK0HDRHy9mJdhlkgbm/uiPQGXs7NOaifF/sVyumKkaJ4DbOZYGtBjG7GY4MTpvMZIJ BjWnkvRoD9528aPWnU1141fIEKSCRfBKvPs/Cs+kSP4yvTg/Evm/xxGW+sGg0wqqUDbZ XKScvLBI/7fvqzdqC5602ab22OqXRbQJNWjRcnWD7YdiTvm6sLfHt52vCEyuGxlTO2Wk DlMqEyQOFIiRIC23LnQBmI3am7fEqCNUZzN7mpHCgXzq4Q5jSfKhPcFf2zVdv9E+0Vay 2CG61j/YbVs04vhigvtrEHGzhnBpEBGcw0BwpKiR4sSCu/5ZeV1yosKyK7S4YlHmFgjQ JbLw== X-Gm-Message-State: AC+VfDx7Vjw/yx7CHssqpNEAThIpWvLka4VC8KL1E53bd97i82Zp02Ws /DnJZhz+7POxpRGUjJXyLTgjRTmhV0OlgnQyATE= X-Received: by 2002:a05:6a00:2194:b0:643:b653:3aa with SMTP id h20-20020a056a00219400b00643b65303aamr46908884pfi.32.1684266138546; Tue, 16 May 2023 12:42:18 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:18 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 35/80] tcg: Merge tcg_out_helper_load_regs into caller Date: Tue, 16 May 2023 12:41:00 -0700 Message-Id: <20230516194145.1749305-36-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::433; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x433.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Now that tcg_out_helper_load_regs is not recursive, we can merge it into its only caller, tcg_out_helper_load_slots. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/tcg.c | 89 +++++++++++++++++++++++++------------------------------ 1 file changed, 41 insertions(+), 48 deletions(-) diff --git a/tcg/tcg.c b/tcg/tcg.c index 5de9380c6b..2b9e032b70 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -5225,12 +5225,50 @@ static int tcg_out_helper_stk_ofs(TCGType type, unsigned slot) return ofs; } -static void tcg_out_helper_load_regs(TCGContext *s, - unsigned nmov, TCGMovExtend *mov, - const TCGLdstHelperParam *parm) +static void tcg_out_helper_load_slots(TCGContext *s, + unsigned nmov, TCGMovExtend *mov, + const TCGLdstHelperParam *parm) { + unsigned i; TCGReg dst3; + /* + * Start from the end, storing to the stack first. + * This frees those registers, so we need not consider overlap. + */ + for (i = nmov; i-- > 0; ) { + unsigned slot = mov[i].dst; + + if (arg_slot_reg_p(slot)) { + goto found_reg; + } + + TCGReg src = mov[i].src; + TCGType dst_type = mov[i].dst_type; + MemOp dst_mo = dst_type == TCG_TYPE_I32 ? MO_32 : MO_64; + + /* The argument is going onto the stack; extend into scratch. */ + if ((mov[i].src_ext & MO_SIZE) != dst_mo) { + tcg_debug_assert(parm->ntmp != 0); + mov[i].dst = src = parm->tmp[0]; + tcg_out_movext1(s, &mov[i]); + } + + tcg_out_st(s, dst_type, src, TCG_REG_CALL_STACK, + tcg_out_helper_stk_ofs(dst_type, slot)); + } + return; + + found_reg: + /* + * The remaining arguments are in registers. + * Convert slot numbers to argument registers. + */ + nmov = i + 1; + for (i = 0; i < nmov; ++i) { + mov[i].dst = tcg_target_call_iarg_regs[mov[i].dst]; + } + switch (nmov) { case 4: /* The backend must have provided enough temps for the worst case. */ @@ -5273,51 +5311,6 @@ static void tcg_out_helper_load_regs(TCGContext *s, } } -static void tcg_out_helper_load_slots(TCGContext *s, - unsigned nmov, TCGMovExtend *mov, - const TCGLdstHelperParam *parm) -{ - unsigned i; - - /* - * Start from the end, storing to the stack first. - * This frees those registers, so we need not consider overlap. - */ - for (i = nmov; i-- > 0; ) { - unsigned slot = mov[i].dst; - - if (arg_slot_reg_p(slot)) { - goto found_reg; - } - - TCGReg src = mov[i].src; - TCGType dst_type = mov[i].dst_type; - MemOp dst_mo = dst_type == TCG_TYPE_I32 ? MO_32 : MO_64; - - /* The argument is going onto the stack; extend into scratch. */ - if ((mov[i].src_ext & MO_SIZE) != dst_mo) { - tcg_debug_assert(parm->ntmp != 0); - mov[i].dst = src = parm->tmp[0]; - tcg_out_movext1(s, &mov[i]); - } - - tcg_out_st(s, dst_type, src, TCG_REG_CALL_STACK, - tcg_out_helper_stk_ofs(dst_type, slot)); - } - return; - - found_reg: - /* - * The remaining arguments are in registers. - * Convert slot numbers to argument registers. - */ - nmov = i + 1; - for (i = 0; i < nmov; ++i) { - mov[i].dst = tcg_target_call_iarg_regs[mov[i].dst]; - } - tcg_out_helper_load_regs(s, nmov, mov, parm); -} - static void tcg_out_helper_load_imm(TCGContext *s, unsigned slot, TCGType type, tcg_target_long imm, const TCGLdstHelperParam *parm) From patchwork Tue May 16 19:41:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682369 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp639632ltp; Tue, 16 May 2023 12:45:04 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6AUTQhfirsYbhT3QlYs2NMGe55hWjiebPtOL9GiWOpVQCBmajxAiuIfFUDPwwAgxuqc377 X-Received: by 2002:ac8:7fd6:0:b0:3f5:3f9e:ffce with SMTP id b22-20020ac87fd6000000b003f53f9effcemr5632034qtk.43.1684266304245; Tue, 16 May 2023 12:45:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266304; cv=none; d=google.com; s=arc-20160816; b=TwG4VNb7IahCtvajVJ47P0UhQgqu4tROQiYxPF5opQ6rHZlxbkuPUy2FH20n7K5KrG pNU/UIF5TUFm54a68xDGcmO0S/PkLPHLWTFM/nlgA+/ZgOpcbrlcyXDIDmohys9Amg5g ea+UlJBZVpekv1x7uVLMH07kRvDKL5YSJXgMdifJqy0tct8rEJdJsPMEewrHyQefp7cp I/cWhkSI+BaOYHVGiVTOH+D+fszWOFoMuSKazbIAIA1CywPXTYmAMZ/HyIPVx7AXhq/1 on/vThtqBMp1YfY9N21DOrBXm70M+2lmOCj3nRsZlyHYSrR+QzJra0BK891uHvxPLDug 4vBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=hcfRwBSVBOMMwU/KCtGetv2R2rpWXZs+QgPxQTLw33g=; b=SOTQdpqH0grke9dcccISh+wR/EVesMg9YXlOODB0dP72LFMN9Hllv03dJ6PpZ946ca wS+laWJII82VgeoSFfJqUQVigZ6IqRZXxL1F4Z22wJ6yRKTkMGbEd5SoRQ2G8qEykOlh kw8m5sZFdWO3nxn997200sLt34+Muuxcn9cqU/iduzy2JMNiRGPd1S7TStu1mrJZOvU5 OLwbkIKxPz8W2oWjKKnLIyF1AxeOhZXQ7XPcHF4/A0qkenE9NpTs110ya7Yc9VP8P6U8 js+DUvdboxYigJmYqS78Mg2O3mfbj4BCNVooIpUC+YXKawGopMkIMy51dS3dsI6sROgu zK3w== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=QZiI9wtr; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id s4-20020a05620a16a400b007591796c2d0si243901qkj.140.2023.05.16.12.45.04 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:45:04 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=QZiI9wtr; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Z0-0006pJ-Em; Tue, 16 May 2023 15:42:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yy-0006oC-DK for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:24 -0400 Received: from mail-pf1-x42e.google.com ([2607:f8b0:4864:20::42e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yu-0002uZ-M7 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:24 -0400 Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-6434e65d808so15320002b3a.3 for ; Tue, 16 May 2023 12:42:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266139; x=1686858139; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hcfRwBSVBOMMwU/KCtGetv2R2rpWXZs+QgPxQTLw33g=; b=QZiI9wtrW/dSZLKyvxWZzhKddqnVchXJqWcpTFgQ3tG0SZFzIryumJwoJ2BChVHnqE aNSpZwvSIi41TEJP2gEJ0gGNLFXF6TvU9X+GignpOohw+gJ+c82TYb9ZmoEX4aZHplVt 8rUB4wsZ9zYEOcRtVcSyRshNQWl/idSEN8mRDp+DUs14S9As5tGkFtR5BwycteuVFuKm /bOWctxZaOre2ms+QrsOJmlMZwPIp0cI+WhesabFl7LwCKR7GkXPkuc+WokcQ/cCQgDF wjGNnNdAPqMy15cewBgw1bWZ+F1M1dxI8fPFXwbiiCwtJiLjoXOIIwmCTNNnvYYdkfn9 44Gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266139; x=1686858139; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hcfRwBSVBOMMwU/KCtGetv2R2rpWXZs+QgPxQTLw33g=; b=SfD3kMp7zitYmbCTuwdPcXOEXOpKShpMh1qvND7my2f+IIqUeC0Ytual/U+fxsSKcM QAX7wmGAlxDya2L+ziJZtV4XaQshVxWJVHx8LlXcTPTcEmDz9uw2o+9/NaWprFB6oOmt t26augIf9FfBOOivU2r47FCDpgaFXy07YcdjGDCIFKCQTj3jFeEgTaTGR4w8JWlxSGsm kV+kuYt+bRiFp/hI0WpNYtEHCNw4VnEZRCQAvQG2ZbgwW4MXcyhdj3x5lP+6Kj1sAhEP hAscnSpY1DVlhG2BbVELiJMM/uYjEvUjifIMOSH/g0P24/3mbhub4EIPp1wqqKh4iQlO k3vg== X-Gm-Message-State: AC+VfDzNDcjExwVWIYKr/yVyk5Bh5LLGTEmq9lD1dqtisM17j+Gn5+To n7on2BX4YDS73TCaY2vdkycQaCZ2wjhQqDVm/2M= X-Received: by 2002:a05:6a00:88f:b0:648:24c1:fe5a with SMTP id q15-20020a056a00088f00b0064824c1fe5amr31708497pfj.16.1684266139468; Tue, 16 May 2023 12:42:19 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:18 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 36/80] tcg: Support TCG_TYPE_I128 in tcg_out_{ld, st}_helper_{args, ret} Date: Tue, 16 May 2023 12:41:01 -0700 Message-Id: <20230516194145.1749305-37-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42e; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/tcg.c | 196 +++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 163 insertions(+), 33 deletions(-) diff --git a/tcg/tcg.c b/tcg/tcg.c index 2b9e032b70..582fd1b36d 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -206,6 +206,7 @@ static void * const qemu_ld_helpers[MO_SSIZE + 1] __attribute__((unused)) = { [MO_UQ] = helper_ldq_mmu, #if TCG_TARGET_REG_BITS == 64 [MO_SL] = helper_ldsl_mmu, + [MO_128] = helper_ld16_mmu, #endif }; @@ -214,6 +215,9 @@ static void * const qemu_st_helpers[MO_SIZE + 1] __attribute__((unused)) = { [MO_16] = helper_stw_mmu, [MO_32] = helper_stl_mmu, [MO_64] = helper_stq_mmu, +#if TCG_TARGET_REG_BITS == 64 + [MO_128] = helper_st16_mmu, +#endif }; TCGContext tcg_init_ctx; @@ -866,6 +870,15 @@ static TCGHelperInfo info_helper_ld64_mmu = { | dh_typemask(ptr, 4) /* uintptr_t ra */ }; +static TCGHelperInfo info_helper_ld128_mmu = { + .flags = TCG_CALL_NO_WG, + .typemask = dh_typemask(i128, 0) /* return Int128 */ + | dh_typemask(env, 1) + | dh_typemask(tl, 2) /* target_ulong addr */ + | dh_typemask(i32, 3) /* unsigned oi */ + | dh_typemask(ptr, 4) /* uintptr_t ra */ +}; + static TCGHelperInfo info_helper_st32_mmu = { .flags = TCG_CALL_NO_WG, .typemask = dh_typemask(void, 0) @@ -886,6 +899,16 @@ static TCGHelperInfo info_helper_st64_mmu = { | dh_typemask(ptr, 5) /* uintptr_t ra */ }; +static TCGHelperInfo info_helper_st128_mmu = { + .flags = TCG_CALL_NO_WG, + .typemask = dh_typemask(void, 0) + | dh_typemask(env, 1) + | dh_typemask(tl, 2) /* target_ulong addr */ + | dh_typemask(i128, 3) /* Int128 data */ + | dh_typemask(i32, 4) /* unsigned oi */ + | dh_typemask(ptr, 5) /* uintptr_t ra */ +}; + #ifdef CONFIG_TCG_INTERPRETER static ffi_type *typecode_to_ffi(int argmask) { @@ -1299,8 +1322,10 @@ static void tcg_context_init(unsigned max_cpus) init_call_layout(&info_helper_ld32_mmu); init_call_layout(&info_helper_ld64_mmu); + init_call_layout(&info_helper_ld128_mmu); init_call_layout(&info_helper_st32_mmu); init_call_layout(&info_helper_st64_mmu); + init_call_layout(&info_helper_st128_mmu); #ifdef CONFIG_TCG_INTERPRETER init_ffi_layouts(); @@ -5399,6 +5424,8 @@ static unsigned tcg_out_helper_add_mov(TCGMovExtend *mov, TCGType dst_type, TCGType src_type, TCGReg lo, TCGReg hi) { + MemOp reg_mo; + if (dst_type <= TCG_TYPE_REG) { MemOp src_ext; @@ -5426,19 +5453,25 @@ static unsigned tcg_out_helper_add_mov(TCGMovExtend *mov, return 1; } - assert(TCG_TARGET_REG_BITS == 32); + if (TCG_TARGET_REG_BITS == 32) { + assert(dst_type == TCG_TYPE_I64); + reg_mo = MO_32; + } else { + assert(dst_type == TCG_TYPE_I128); + reg_mo = MO_64; + } mov[0].dst = loc[HOST_BIG_ENDIAN].arg_slot; mov[0].src = lo; - mov[0].dst_type = TCG_TYPE_I32; - mov[0].src_type = TCG_TYPE_I32; - mov[0].src_ext = MO_32; + mov[0].dst_type = TCG_TYPE_REG; + mov[0].src_type = TCG_TYPE_REG; + mov[0].src_ext = reg_mo; mov[1].dst = loc[!HOST_BIG_ENDIAN].arg_slot; mov[1].src = hi; - mov[1].dst_type = TCG_TYPE_I32; - mov[1].src_type = TCG_TYPE_I32; - mov[1].src_ext = MO_32; + mov[1].dst_type = TCG_TYPE_REG; + mov[1].src_type = TCG_TYPE_REG; + mov[1].src_ext = reg_mo; return 2; } @@ -5461,6 +5494,9 @@ static void tcg_out_ld_helper_args(TCGContext *s, const TCGLabelQemuLdst *ldst, case MO_64: info = &info_helper_ld64_mmu; break; + case MO_128: + info = &info_helper_ld128_mmu; + break; default: g_assert_not_reached(); } @@ -5475,8 +5511,33 @@ static void tcg_out_ld_helper_args(TCGContext *s, const TCGLabelQemuLdst *ldst, tcg_out_helper_load_slots(s, nmov, mov, parm); - /* No special attention for 32 and 64-bit return values. */ - tcg_debug_assert(info->out_kind == TCG_CALL_RET_NORMAL); + switch (info->out_kind) { + case TCG_CALL_RET_NORMAL: + case TCG_CALL_RET_BY_VEC: + break; + case TCG_CALL_RET_BY_REF: + /* + * The return reference is in the first argument slot. + * We need memory in which to return: re-use the top of stack. + */ + { + int ofs_slot0 = TCG_TARGET_CALL_STACK_OFFSET; + + if (arg_slot_reg_p(0)) { + tcg_out_addi_ptr(s, tcg_target_call_iarg_regs[0], + TCG_REG_CALL_STACK, ofs_slot0); + } else { + tcg_debug_assert(parm->ntmp != 0); + tcg_out_addi_ptr(s, parm->tmp[0], + TCG_REG_CALL_STACK, ofs_slot0); + tcg_out_st(s, TCG_TYPE_PTR, parm->tmp[0], + TCG_REG_CALL_STACK, ofs_slot0); + } + } + break; + default: + g_assert_not_reached(); + } tcg_out_helper_load_common_args(s, ldst, parm, info, next_arg); } @@ -5485,11 +5546,18 @@ static void tcg_out_ld_helper_ret(TCGContext *s, const TCGLabelQemuLdst *ldst, bool load_sign, const TCGLdstHelperParam *parm) { + MemOp mop = get_memop(ldst->oi); TCGMovExtend mov[2]; + int ofs_slot0; - if (ldst->type <= TCG_TYPE_REG) { - MemOp mop = get_memop(ldst->oi); + switch (ldst->type) { + case TCG_TYPE_I64: + if (TCG_TARGET_REG_BITS == 32) { + break; + } + /* fall through */ + case TCG_TYPE_I32: mov[0].dst = ldst->datalo_reg; mov[0].src = tcg_target_call_oarg_reg(TCG_CALL_RET_NORMAL, 0); mov[0].dst_type = ldst->type; @@ -5515,25 +5583,49 @@ static void tcg_out_ld_helper_ret(TCGContext *s, const TCGLabelQemuLdst *ldst, mov[0].src_ext = mop & MO_SSIZE; } tcg_out_movext1(s, mov); - } else { - assert(TCG_TARGET_REG_BITS == 32); + return; - mov[0].dst = ldst->datalo_reg; - mov[0].src = - tcg_target_call_oarg_reg(TCG_CALL_RET_NORMAL, HOST_BIG_ENDIAN); - mov[0].dst_type = TCG_TYPE_I32; - mov[0].src_type = TCG_TYPE_I32; - mov[0].src_ext = MO_32; + case TCG_TYPE_I128: + tcg_debug_assert(TCG_TARGET_REG_BITS == 64); + ofs_slot0 = TCG_TARGET_CALL_STACK_OFFSET; + switch (TCG_TARGET_CALL_RET_I128) { + case TCG_CALL_RET_NORMAL: + break; + case TCG_CALL_RET_BY_VEC: + tcg_out_st(s, TCG_TYPE_V128, + tcg_target_call_oarg_reg(TCG_CALL_RET_BY_VEC, 0), + TCG_REG_CALL_STACK, ofs_slot0); + /* fall through */ + case TCG_CALL_RET_BY_REF: + tcg_out_ld(s, TCG_TYPE_I64, ldst->datalo_reg, + TCG_REG_CALL_STACK, ofs_slot0 + 8 * HOST_BIG_ENDIAN); + tcg_out_ld(s, TCG_TYPE_I64, ldst->datahi_reg, + TCG_REG_CALL_STACK, ofs_slot0 + 8 * !HOST_BIG_ENDIAN); + return; + default: + g_assert_not_reached(); + } + break; - mov[1].dst = ldst->datahi_reg; - mov[1].src = - tcg_target_call_oarg_reg(TCG_CALL_RET_NORMAL, !HOST_BIG_ENDIAN); - mov[1].dst_type = TCG_TYPE_REG; - mov[1].src_type = TCG_TYPE_REG; - mov[1].src_ext = MO_32; - - tcg_out_movext2(s, mov, mov + 1, parm->ntmp ? parm->tmp[0] : -1); + default: + g_assert_not_reached(); } + + mov[0].dst = ldst->datalo_reg; + mov[0].src = + tcg_target_call_oarg_reg(TCG_CALL_RET_NORMAL, HOST_BIG_ENDIAN); + mov[0].dst_type = TCG_TYPE_I32; + mov[0].src_type = TCG_TYPE_I32; + mov[0].src_ext = TCG_TARGET_REG_BITS == 32 ? MO_32 : MO_64; + + mov[1].dst = ldst->datahi_reg; + mov[1].src = + tcg_target_call_oarg_reg(TCG_CALL_RET_NORMAL, !HOST_BIG_ENDIAN); + mov[1].dst_type = TCG_TYPE_REG; + mov[1].src_type = TCG_TYPE_REG; + mov[1].src_ext = TCG_TARGET_REG_BITS == 32 ? MO_32 : MO_64; + + tcg_out_movext2(s, mov, mov + 1, parm->ntmp ? parm->tmp[0] : -1); } static void tcg_out_st_helper_args(TCGContext *s, const TCGLabelQemuLdst *ldst, @@ -5557,6 +5649,10 @@ static void tcg_out_st_helper_args(TCGContext *s, const TCGLabelQemuLdst *ldst, info = &info_helper_st64_mmu; data_type = TCG_TYPE_I64; break; + case MO_128: + info = &info_helper_st128_mmu; + data_type = TCG_TYPE_I128; + break; default: g_assert_not_reached(); } @@ -5574,13 +5670,47 @@ static void tcg_out_st_helper_args(TCGContext *s, const TCGLabelQemuLdst *ldst, /* Handle data argument. */ loc = &info->in[next_arg]; - n = tcg_out_helper_add_mov(mov + nmov, loc, data_type, ldst->type, - ldst->datalo_reg, ldst->datahi_reg); - next_arg += n; - nmov += n; - tcg_debug_assert(nmov <= ARRAY_SIZE(mov)); + switch (loc->kind) { + case TCG_CALL_ARG_NORMAL: + case TCG_CALL_ARG_EXTEND_U: + case TCG_CALL_ARG_EXTEND_S: + n = tcg_out_helper_add_mov(mov + nmov, loc, data_type, ldst->type, + ldst->datalo_reg, ldst->datahi_reg); + next_arg += n; + nmov += n; + tcg_out_helper_load_slots(s, nmov, mov, parm); + break; + + case TCG_CALL_ARG_BY_REF: + tcg_debug_assert(TCG_TARGET_REG_BITS == 64); + tcg_debug_assert(data_type == TCG_TYPE_I128); + tcg_out_st(s, TCG_TYPE_I64, + HOST_BIG_ENDIAN ? ldst->datahi_reg : ldst->datalo_reg, + TCG_REG_CALL_STACK, arg_slot_stk_ofs(loc[0].ref_slot)); + tcg_out_st(s, TCG_TYPE_I64, + HOST_BIG_ENDIAN ? ldst->datalo_reg : ldst->datahi_reg, + TCG_REG_CALL_STACK, arg_slot_stk_ofs(loc[1].ref_slot)); + + tcg_out_helper_load_slots(s, nmov, mov, parm); + + if (arg_slot_reg_p(loc->arg_slot)) { + tcg_out_addi_ptr(s, tcg_target_call_iarg_regs[loc->arg_slot], + TCG_REG_CALL_STACK, + arg_slot_stk_ofs(loc->ref_slot)); + } else { + tcg_debug_assert(parm->ntmp != 0); + tcg_out_addi_ptr(s, parm->tmp[0], TCG_REG_CALL_STACK, + arg_slot_stk_ofs(loc->ref_slot)); + tcg_out_st(s, TCG_TYPE_PTR, parm->tmp[0], + TCG_REG_CALL_STACK, arg_slot_stk_ofs(loc->arg_slot)); + } + next_arg += 2; + break; + + default: + g_assert_not_reached(); + } - tcg_out_helper_load_slots(s, nmov, mov, parm); tcg_out_helper_load_common_args(s, ldst, parm, info, next_arg); } From patchwork Tue May 16 19:41:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682401 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp19805wrt; Tue, 16 May 2023 12:54:51 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6ubzJUmBkFZkynp9QgU85qlCoxXjOdkriLvMZVbvIYxs1PsCqIez6p3y+3zHk7MhXLSjJx X-Received: by 2002:a05:622a:8e:b0:3e4:d1c0:36a9 with SMTP id o14-20020a05622a008e00b003e4d1c036a9mr61775048qtw.48.1684266891090; Tue, 16 May 2023 12:54:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266891; cv=none; d=google.com; s=arc-20160816; b=ztZuP3uCOGM9rUVO2VfCgXyOITAYw+udKpkr2sKu5FiK4CHoYHK1XgIyxLkprPtHmg Al7ktmuxWpX4o0idp4fT3hwYRSHfIStXbnATGzywMv++6OFMIJK6PVGw9C9tKAS25Xug soO0IF5sJuJ4SmAgNAelHEt9ZHGXXx0VMO90TPBdbFzSwG37HGhjkqbuglYnMsa/N3Ao 7xWYmHUKThehza28ZEMcboaJ7G8/jmWVuCIVUkmyt2eq8Wm6tPqBZYOpYQl4nbh0a92S Oak6gC0SkSOA3Hwuos3sZFc2waMIL2XaHNVBUHaJpukXI2IyJnoxINQKk1TwMU+RpcRE XN0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=ribAH55dCyxFTDrwMfk5QxSVbH9673DY11IGNIerYiI=; b=RCfRw0+WmmjMKTNi5+smHRkQaqCGi+Wh7YVOhsg4jfuXqei2JvNUYXCFyi8HN5t6F0 Jwk4L2OhAfsF20VqUdOAnFwzdya1JftZSDwzFUgtKW+DmV0bJ376Kkkt5i04PF1Nzxdm xbZLh29a9KNNx0RBh33KYiVKR60hnZ/vasUim+zLs1jgUEeMjQTtxlE/LtBhA7ZFpu2A aKaXk/v9B9g7ZqbjWu0+o/xtC/U2vsIHYRwDyrVSVhzZOUq99n5mZJ1BRj3UatVB7pVy f3n4mlXLdNMsREt34PDbpEQ42d96Q1wQ/FWbukI+mLCXf2lRoGnSoMaSdeQ95e1fILP+ XNEw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=abX5yocF; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t10-20020a05622a01ca00b003ef3204c658si12116447qtw.490.2023.05.16.12.54.50 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:54:51 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=abX5yocF; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Yy-0006oB-Ds; Tue, 16 May 2023 15:42:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yx-0006nn-3o for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:23 -0400 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yu-0002pV-VW for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:22 -0400 Received: by mail-pg1-x530.google.com with SMTP id 41be03b00d2f7-52867360efcso10426459a12.2 for ; Tue, 16 May 2023 12:42:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266140; x=1686858140; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ribAH55dCyxFTDrwMfk5QxSVbH9673DY11IGNIerYiI=; b=abX5yocFee4UDT2vYAewOC+0jxhncuaSSM5uf7sz4gkaJAbCDTvfb2IQTT94PukY4S mUsN1dHdw6rAntXCpal8Ya9pCfzgFx73OTb2IUiuIxkb4U4LuNBvptBeosNdZuDIAsNF 5eQZMElB0gzhQmVKvzTPPxcB4u3k43daj7WxaU1hk0h81J6tGSA7Dcc9Ax6jClpn4EpI SV8sGTbJvbp3n87tYfQbyUZPx2IvdjnmzK0KqhtYjX0Rg2DXc+EU989Y4fJPiZu497rs v+4iEQOhqaXTKNMt31pXrkeQNPhVNdROsLulMuEkqAX9kLaO1qZ+C5vu5sWDdUps8FKh JAow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266140; x=1686858140; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ribAH55dCyxFTDrwMfk5QxSVbH9673DY11IGNIerYiI=; b=LyuDY3RS11VO4aAKrkLqBuhfv/LIxmaWXSv7+iUTTthBeXb4VBPjRqDATTmqNsEAUI iE1Kdhv3ZCZkv4OSXHpC5NH9XOFktBf31wmDep3DW6z2Fep73v8/ekSmc9PS+1mDcna5 X8R+e1VVkn+Z8mBHUv8eAXzLpgK9lYKDHRvgWyqM4ZLPkb/y/3edgjdpzP7slkUr7LFR 1UJL84MX7HPFcpTkdGZ+P49jVJfM4Xw6v58p+w4+kAPiE+gijaz6x6CLSVHdl+xaT6sQ ncsdjwabZT5sGkOM5h5v3U8szj3lMO/Ez2XpHSBSnVLiH3PWSwpNlVQ5c876+cKnZGUs lmog== X-Gm-Message-State: AC+VfDxJwX/vClJkVzWlyjTRrn73t9VD8xDCzKh0oPuVIKJFGEW68YMN kAEeERNrUk0EUe2/nR5OTe8xQ2DbGQzQwQ8o4dw= X-Received: by 2002:a05:6a20:1448:b0:105:55a7:d5ff with SMTP id a8-20020a056a20144800b0010555a7d5ffmr14185514pzi.28.1684266140276; Tue, 16 May 2023 12:42:20 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:19 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 37/80] tcg: Introduce atom_and_align_for_opc Date: Tue, 16 May 2023 12:41:02 -0700 Message-Id: <20230516194145.1749305-38-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::530; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x530.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Examine MemOp for atomicity and alignment, adjusting alignment as required to implement atomicity on the host. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/tcg.c | 95 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) diff --git a/tcg/tcg.c b/tcg/tcg.c index 582fd1b36d..878e780cfa 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -220,6 +220,15 @@ static void * const qemu_st_helpers[MO_SIZE + 1] __attribute__((unused)) = { #endif }; +typedef struct { + MemOp atom; /* lg2 bits of atomicity required */ + MemOp align; /* lg2 bits of alignment to use */ +} TCGAtomAlign; + +static TCGAtomAlign atom_and_align_for_opc(TCGContext *s, MemOp opc, + MemOp host_atom, bool allow_two_ops) + __attribute__((unused)); + TCGContext tcg_init_ctx; __thread TCGContext *tcg_ctx; @@ -5228,6 +5237,92 @@ static void tcg_reg_alloc_call(TCGContext *s, TCGOp *op) } } +/** + * atom_and_align_for_opc: + * @s: tcg context + * @opc: memory operation code + * @host_atom: MO_ATOM_{IFALIGN,WITHIN16,SUBALIGN} for host operations + * @allow_two_ops: true if we are prepared to issue two operations + * + * Return the alignment and atomicity to use for the inline fast path + * for the given memory operation. The alignment may be larger than + * that specified in @opc, and the correct alignment will be diagnosed + * by the slow path helper. + * + * If @allow_two_ops, the host is prepared to test for 2x alignment, + * and issue two loads or stores for subalignment. + */ +static TCGAtomAlign atom_and_align_for_opc(TCGContext *s, MemOp opc, + MemOp host_atom, bool allow_two_ops) +{ + MemOp align = get_alignment_bits(opc); + MemOp size = opc & MO_SIZE; + MemOp half = size ? size - 1 : 0; + MemOp atmax; + MemOp atom; + + /* When serialized, no further atomicity required. */ + if (s->gen_tb->cflags & CF_PARALLEL) { + atom = opc & MO_ATOM_MASK; + } else { + atom = MO_ATOM_NONE; + } + + switch (atom) { + case MO_ATOM_NONE: + /* The operation requires no specific atomicity. */ + atmax = MO_8; + break; + + case MO_ATOM_IFALIGN: + atmax = size; + break; + + case MO_ATOM_IFALIGN_PAIR: + atmax = half; + break; + + case MO_ATOM_WITHIN16: + atmax = size; + if (size == MO_128) { + /* Misalignment implies !within16, and therefore no atomicity. */ + } else if (host_atom != MO_ATOM_WITHIN16) { + /* The host does not implement within16, so require alignment. */ + align = MAX(align, size); + } + break; + + case MO_ATOM_WITHIN16_PAIR: + atmax = size; + /* + * Misalignment implies !within16, and therefore half atomicity. + * Any host prepared for two operations can implement this with + * half alignment. + */ + if (host_atom != MO_ATOM_WITHIN16 && allow_two_ops) { + align = MAX(align, half); + } + break; + + case MO_ATOM_SUBALIGN: + atmax = size; + if (host_atom != MO_ATOM_SUBALIGN) { + /* If unaligned but not odd, there are subobjects up to half. */ + if (allow_two_ops) { + align = MAX(align, half); + } else { + align = MAX(align, size); + } + } + break; + + default: + g_assert_not_reached(); + } + + return (TCGAtomAlign){ .atom = atmax, .align = align }; +} + /* * Similarly for qemu_ld/st slow path helpers. * We must re-implement tcg_gen_callN and tcg_reg_alloc_call simultaneously, From patchwork Tue May 16 19:41:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682364 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp639096ltp; Tue, 16 May 2023 12:43:45 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7ZqE1IEWf/xKEHJCmhRfOJc09QsIN5YVGDdMoRQMcim8pUVD1aNHC4VMZv+RQng2kPn9+A X-Received: by 2002:a05:6214:c23:b0:61b:73b2:72c with SMTP id a3-20020a0562140c2300b0061b73b2072cmr57159067qvd.16.1684266224924; Tue, 16 May 2023 12:43:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266224; cv=none; d=google.com; s=arc-20160816; b=pVe+Pr+bqs8tV4mfOG8h9uliIkHj4unauz85nHlpHfD0K4Jplg73RC8GII6ogZMlyj Y+jB9liyTUny3dqabEmkVR7KNvjAqpv813HYr/HMIDTZ/OsvCRM+9Ij3htaUPzOBqlGn w/8TreDWBZlw4lR0nAmoU8nHI1vJW/lDlSbR832suP1wZSWM9EOPfkPypnBdLHHj4NQk zUyflfxBAF0xIVP5PO3x6SGjXg0iBbnguWNpshjjucUBufO95m4bl38nFx4Ce0oSQIZ8 em3yavylJUKNzuNrNYmzBI1HclfAlIgN5/+5ZMR3h0QGfhbwOp6zIaZTaPCPocU5J1ct OV/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=Z3YFUaaopV6WveVfV6rOwrHSft/KYLRd6VT0ztExKVY=; b=X8eApu7nAlGlIOfSplccwUqk06oM5Cw5ckV4ao3sTmm8ZW+qSk3LVX7NStOsHNTTjI gmmKJzyolSnR7Qj9TgpKpUC3LYZbWAIntE2FHB7NUFr5eCNv0iFxJsCG7+2ylvP170+E ZiFryvWzmSsWfB68l0XSP1I5wqU/7A9safbVBAL/6kdxm2toy+XJe1sxe2kdhcKGWo6s IbVxd7DDyU0Q8GLpMwT9ZxQiF1F8qUx9zhnBcH5hEUNYEvtXZslgqalfEey4VOehiW+i 3z1r+nmqiZZq0PVR7ixUgTGtMyn2uDYBZuR8jQW34mqFI3IbAylt1zRGVW/BG3c1jf7t +21A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="sOw/RUc9"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r8-20020a056214212800b00623821414cesi975021qvc.284.2023.05.16.12.43.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:43:44 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="sOw/RUc9"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Z2-0006pt-Ic; Tue, 16 May 2023 15:42:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Yy-0006oW-LD for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:24 -0400 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yv-0002qM-VK for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:24 -0400 Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-64359d9c531so10855445b3a.3 for ; Tue, 16 May 2023 12:42:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266141; x=1686858141; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Z3YFUaaopV6WveVfV6rOwrHSft/KYLRd6VT0ztExKVY=; b=sOw/RUc9Mt/aInBfeqFL6pOdR6kkkVAYqZr9IW4ScqNCTYdljtgJxBwu2ScU+bt3mj n171MAIQZgsUrD246NEvfPvHdPvc8CzDfa5zsBf6DtTf6n+QlQWKRnNoWr3EUebVm7/T W8JmMRshGw+ZRAgY6VJWOU8a4q/z8tvGC7jt3r+Yc0pTXroSvgPvy1viuPy49a5y4PG5 zM4/YmRjVoEHBIgAMUMIvjp2GXip4D/3wTZfU60h++JE5u6er50xX9Z1kibd7fI1XJiv MOVMpxrur6ieoijtYybDlcjZxVF+s2NlkDYM90GknuqXb6Z5jpEXqx3RD/KMjYP/uiRD tmcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266141; x=1686858141; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Z3YFUaaopV6WveVfV6rOwrHSft/KYLRd6VT0ztExKVY=; b=etGLCOqd54TYNdncVEpn+jvAIQXzPwU4Au7osiZdLy/+i8jBOH/Z6eLtA7Pkk0xGc0 WwgcteqpqGke5J+fDCFaNXWKGlXsK4DL6SPAZ0+0eu9FOMn2jc4C6h5MFgceNfcj5L6P QsChwWwTExCfmWYLq4MvxQILR+U7ZF3EAG1GD7iqLVkRef04Wzn5E/Z7KEDOo1Wyklsc XvcUiZ2Cs4T1KyWt281uFt3HUufM1tlbNfEQ2TY4t4WyfN49jvraE3FYdJATfsL1WWIF IP8Kkn+Fd9Qy0LJha8QW5fgrWNOVyaxO01WEGGTmMdTyZ5QvwrINEI+GafKE3FIqrQfb 2EoA== X-Gm-Message-State: AC+VfDyYbXgRovvHVS/1R5bJw7A09SuDu8PRJ+nLzBFqYE3OP+ONTBKj HlQmSXnqk0BDu7iIQhcJe0mbFQZybGRqzzb8fxM= X-Received: by 2002:a05:6a00:b54:b0:64c:c841:4e8a with SMTP id p20-20020a056a000b5400b0064cc8414e8amr3006039pfo.22.1684266141160; Tue, 16 May 2023 12:42:21 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:20 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 38/80] tcg/i386: Use atom_and_align_for_opc Date: Tue, 16 May 2023 12:41:03 -0700 Message-Id: <20230516194145.1749305-39-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::429; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x429.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org No change to the ultimate load/store routines yet, so some atomicity conditions not yet honored, but plumbs the change to alignment through the relevant functions. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.c.inc | 27 +++++++++++++++------------ 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index 6d55ba5a1c..3b8528e332 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -1774,6 +1774,7 @@ typedef struct { int index; int ofs; int seg; + TCGAtomAlign aa; } HostAddress; bool tcg_target_has_memory_bswap(MemOp memop) @@ -1895,8 +1896,18 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, { TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); - unsigned a_bits = get_alignment_bits(opc); - unsigned a_mask = (1 << a_bits) - 1; + unsigned a_mask; + +#ifdef CONFIG_SOFTMMU + h->index = TCG_REG_L0; + h->ofs = 0; + h->seg = 0; +#else + *h = x86_guest_base; +#endif + h->base = addrlo; + h->aa = atom_and_align_for_opc(s, opc, MO_ATOM_IFALIGN, false); + a_mask = (1 << h->aa.align) - 1; #ifdef CONFIG_SOFTMMU int cmp_ofs = is_ld ? offsetof(CPUTLBEntry, addr_read) @@ -1946,7 +1957,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, * copy the address and mask. For lesser alignments, check that we don't * cross pages for the complete access. */ - if (a_bits >= s_bits) { + if (a_mask >= s_mask) { tcg_out_mov(s, ttype, TCG_REG_L1, addrlo); } else { tcg_out_modrm_offset(s, OPC_LEA + trexw, TCG_REG_L1, @@ -1977,13 +1988,8 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, /* TLB Hit. */ tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_L0, TCG_REG_L0, offsetof(CPUTLBEntry, addend)); - - *h = (HostAddress) { - .base = addrlo, - .index = TCG_REG_L0, - }; #else - if (a_bits) { + if (a_mask) { ldst = new_ldst_label(s); ldst->is_ld = is_ld; @@ -1997,9 +2003,6 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, ldst->label_ptr[0] = s->code_ptr; s->code_ptr += 4; } - - *h = x86_guest_base; - h->base = addrlo; #endif return ldst; From patchwork Tue May 16 19:41:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682389 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp18793wrt; Tue, 16 May 2023 12:51:28 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4+CJNl6UuwMGumIpWz6xDY8ZRuljWEtmQknAJ/b1RAbH/AK896QCwg0ZIt+Z91s71DQhM4 X-Received: by 2002:ad4:5f4c:0:b0:5f7:a9e1:bbbf with SMTP id p12-20020ad45f4c000000b005f7a9e1bbbfmr61810554qvg.44.1684266688745; Tue, 16 May 2023 12:51:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266688; cv=none; d=google.com; s=arc-20160816; b=V0QDMjcnbr73OhI/5eIMO5ar/mAfeOGH2XBIk6ZW3xK2fSjeIzJum3z1TfGyhDHUxu QVMVYPvLfOxa8xhlF6Tj4FDoLB6cDZmyANZkW9sAidRtsSzGOv24rkTzyfyUivhB0kof BbYXHVv1sBdq1aUw87osiHPIO3nVKCjmJT468rezcGnlhImnAUWYCK922Jk9zcbB3ckI 1KA64GOaeTTeh4JwBNXt0ni+YO6VSY+6FzNy21T2xpvXYITi1EJP9uyY7QDcxLqiHTYl 2QlHEPEW1aGaOLWJUrN/Fq8ua++pDmV9ntgXhF6TVcHigKyO1yczd4hAbvjSP99ZPLwT SYmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=LPhN/AVmZfqDEN0zKKnHjQGokKYF75xLV5YHB0OEaoc=; b=iPxx4rtN7Mik+ZtC828i0XtMkb7f9lECPVSxsEI3ipHtT7CgvzpP6f/BMNMSTyaCOg dwZAMKf1ft9kkY6ZIM4TikolumzTFB8sV6a7PkmVteT2+or40Ap6D0cXGb6fwTl2ip0u ah6W7hZ+WYl1pbnoCFf0yg35MPGB6P3TO7ie8sSfURdl2LCiTbf4gmhAB+XDajGwZdiP EwPRZFvo9/7jUIugdeX2h4InI1SulT7JreyFz+3WEINogZS07iiHJ5FnWhPZHYbKWKU6 l7b11XL/OG0oJcUZJX3jbyXy1NIkPRPKhpjLuJeEh0jbYPlnCKuZCvGjsTRvhUUq4r1U /nkw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="M/N/6lTq"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id c19-20020ae9e213000000b0074ab068e8bcsi204439qkc.526.2023.05.16.12.51.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:51:28 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="M/N/6lTq"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0Z0-0006pW-UY; Tue, 16 May 2023 15:42:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0Z0-0006pA-02 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:26 -0400 Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0Yy-0002vF-9m for qemu-devel@nongnu.org; Tue, 16 May 2023 15:42:25 -0400 Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-64384274895so10368132b3a.2 for ; Tue, 16 May 2023 12:42:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266142; x=1686858142; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LPhN/AVmZfqDEN0zKKnHjQGokKYF75xLV5YHB0OEaoc=; b=M/N/6lTqV1CBQsGZmMZk88QpJF+qaMz2a3KhjtiDREn5P68V7PCJyfDecKteQH0m3o yrp5icQPqxfJXjwJGj13a4vstUTrLRq17RHdCSoAisJj7LitQMQ6IL/yjkVJp+BwMLxz bNTv/E3xO7OZWUsLizkIAO3KoDCziXwwbksElNKn0QzMa8l/lg1P6Oi74fSfsI18pCBH fzkSm/mTKLl/F62xV2Co7GS8JRm0mkmFT9Vb+yrqNPpYNDPpgnDNFdGxkn4D5hGIa5ta oB1DZTpAZIz/bw189IqP2oWxt60bddaZYzj3VGnuQz8hZCmtw3pd+tM8ZHT5hEE2UMs1 6otQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266142; x=1686858142; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LPhN/AVmZfqDEN0zKKnHjQGokKYF75xLV5YHB0OEaoc=; b=Zb1N5Kbsn2pDYF8SHfupL7eoQRGl482+I71d26TZ76h6ZN1syddzZs7WfJEB39GJ2Z +iQY9i00oT8QyKY14opKwgG1gyS6KQba5+7rX7i+XERKAjRSPWyC2H0dwygbLc+4VmCA lQlNnjl7bV5qIA/Ni7YNRhq9OSNtyow+BLUWywLumO7mYJMBQKNV7uukRGJVdXAu2oMf 3171KMBhQ0UNvR8mkCcDoAzLmCpJ2OtnlymHIhV5CWfsgRLozJ8e26+b+WiTiHEXzeEe rmbis9XYOR/5ZWp+oGwPyYsixbUyBnXT2Swibx0EP1OxvZiVrj56qhicwTSoN+YdvViZ H4Qg== X-Gm-Message-State: AC+VfDyAAd5lj4LxnlcVcMGikRx6QVc9Fs/ZOArD5Hq32PIIU5b2W4jP mj5zcrPlFYvaHrQtv8YjgS3weooijCmfel7zocE= X-Received: by 2002:a05:6a00:124c:b0:647:d698:56d2 with SMTP id u12-20020a056a00124c00b00647d69856d2mr30384765pfi.27.1684266141991; Tue, 16 May 2023 12:42:21 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id z21-20020aa791d5000000b006260526cf0csm13771165pfa.116.2023.05.16.12.42.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:42:21 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 39/80] tcg/aarch64: Use atom_and_align_for_opc Date: Tue, 16 May 2023 12:41:04 -0700 Message-Id: <20230516194145.1749305-40-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42a; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.c.inc | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index 0cc719d799..ea4108d59c 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -1593,6 +1593,7 @@ typedef struct { TCGReg base; TCGReg index; TCGType index_ext; + TCGAtomAlign aa; } HostAddress; bool tcg_target_has_memory_bswap(MemOp memop) @@ -1646,8 +1647,13 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, TCGType addr_type = TARGET_LONG_BITS == 64 ? TCG_TYPE_I64 : TCG_TYPE_I32; TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); - unsigned a_bits = get_alignment_bits(opc); - unsigned a_mask = (1u << a_bits) - 1; + unsigned a_mask; + + h->aa = atom_and_align_for_opc(s, opc, + have_lse2 ? MO_ATOM_WITHIN16 + : MO_ATOM_IFALIGN, + false); + a_mask = (1 << h->aa.align) - 1; #ifdef CONFIG_SOFTMMU unsigned s_bits = opc & MO_SIZE; @@ -1693,7 +1699,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, * bits within the address. For unaligned access, we check that we don't * cross pages using the address of the last byte of the access. */ - if (a_bits >= s_bits) { + if (a_mask >= s_mask) { x3 = addr_reg; } else { tcg_out_insn(s, 3401, ADDI, TARGET_LONG_BITS == 64, @@ -1713,11 +1719,9 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, ldst->label_ptr[0] = s->code_ptr; tcg_out_insn(s, 3202, B_C, TCG_COND_NE, 0); - *h = (HostAddress){ - .base = TCG_REG_X1, - .index = addr_reg, - .index_ext = addr_type - }; + h->base = TCG_REG_X1, + h->index = addr_reg; + h->index_ext = addr_type; #else if (a_mask) { ldst = new_ldst_label(s); @@ -1735,17 +1739,13 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, } if (USE_GUEST_BASE) { - *h = (HostAddress){ - .base = TCG_REG_GUEST_BASE, - .index = addr_reg, - .index_ext = addr_type - }; + h->base = TCG_REG_GUEST_BASE; + h->index = addr_reg; + h->index_ext = addr_type; } else { - *h = (HostAddress){ - .base = addr_reg, - .index = TCG_REG_XZR, - .index_ext = TCG_TYPE_I64 - }; + h->base = addr_reg; + h->index = TCG_REG_XZR; + h->index_ext = TCG_TYPE_I64; } #endif From patchwork Tue May 16 19:41:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682398 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp19707wrt; Tue, 16 May 2023 12:54:28 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ51BXPjMMa76GNzr2HpwjbRpxbpk+ePcCzzmiU5fnX6dMyvOf4VQJNMLdsKJG/TBEB+kYf/ X-Received: by 2002:a05:622a:350:b0:3f3:96c9:b071 with SMTP id r16-20020a05622a035000b003f396c9b071mr46311007qtw.26.1684266867944; Tue, 16 May 2023 12:54:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266867; cv=none; d=google.com; s=arc-20160816; b=d+ilx+PMRhdnrZMONyAVlVmvz4um+CzZXPy7x5+5FWgPfMk39OqXBdl3ECgZl+Ht5Q 9TqltYYpkVyHmuZcgUonzIJZsaKhQt4jOmD5g5g3i9Nce5GLWott9ezXvn3xvSNQIkch oQHSJAmugL05EDtrY2W9JqkTwcYkeOCFZLfV5WH7AFsq2VOX/X937oy+/ALold4aNSTJ JwSSqm9w4OJLOw92K/GxKROuXKP+5vKErEd6G1WkYFJ/zEx4FYE8n+SEQpDEIpc5YwLd kv8/YOKJ2vrsLTEepNzu+p2p7uio4W2l9iFV6wCh8Da3XaaIBqo+yAfTBjs4CbmJ1D8C 8Olg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=1qqlVmOWLkkD4N0TUKCkau5U1AXPhMOrCn0VUAaOFSs=; b=PkwRD2A96RynGkf1tgjaT6yEw1dBFPbx5+NZHA24N3/fUKRpOXIsuigExhRDWjmAMM 79wmr8YKjB8h2b31rGJ9eKp61q3W649aIp4JcK+XIm+ANyVyAh/EU8ROqjO1LfdOnriv 5qJrq0TxQy25EoXCNJ9hVG/hry1BAu3PkOyA63tShxBXX7O1xutA0i+yZFE3EyaPfbcO yMvSVif3VU2LC3g5a8KZi+CoyU5W6rTC1liGfF/AXg+4fdftfLwbzPE2NIXRrFEvs+fY bhyVRt/Y35bF7yk9GshUVbOwfVAEZw+QXwjjIJA5Iy8Sy9PpddbTz5hDk4oWTUf6MWr3 Gqag== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="VMVhWe/J"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t29-20020a37ea1d000000b0074debe9eb58si258971qkj.279.2023.05.16.12.54.27 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:54:27 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="VMVhWe/J"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0bz-0008Ox-Dk; Tue, 16 May 2023 15:45:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0bu-0008Hy-K0 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:26 -0400 Received: from mail-pg1-x52b.google.com ([2607:f8b0:4864:20::52b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0bs-0003eh-Ph for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:26 -0400 Received: by mail-pg1-x52b.google.com with SMTP id 41be03b00d2f7-52079a12451so10506643a12.3 for ; Tue, 16 May 2023 12:45:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266323; x=1686858323; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1qqlVmOWLkkD4N0TUKCkau5U1AXPhMOrCn0VUAaOFSs=; b=VMVhWe/JxWeftGcUftozwAjFobVowe+43QdO1ZTTHJ2HPAjNtEwuSw+kQhWhOiV6Lk 7+xw86dRuXgtLwfuWtOlszi74mZjyl9uKeaB12jO18IXy7ysfyNqEAVhfPuSuPjus1qW rk/qoyQp1Xzoy3DfjR+lSdqzlDGHlTzZhg2Vnqcex68Y/oD5dl4ChYGQU/L+AmYnLzL6 PSzn/qoIJ3wkkq85UkVxEwF28IV92xg6iDplbLP2iX7hw/pLEPGM6m8FWSxEHR6vt8yj FZ53Id4SHU2Sf1dyLkWXxkivh77CzJhe/+/dvMF6MR/HbabnA6toaVLBUSXFledZ5Rqg kQDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266323; x=1686858323; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1qqlVmOWLkkD4N0TUKCkau5U1AXPhMOrCn0VUAaOFSs=; b=XF9V9U+W5GbkAyCZ6eAkipEY/QP+5z9QR7ylY2ESXNBbZPoZfl6G5kN5h/gO905Egk kcs4ojOm9VZ6U5etCioFxKYVy1cUOUcWn8MtPt9bFKQ4Z780fCM5tDavjhjbLPhlVmB1 q/OfXtYD9IE8e/PLj6Pb9jNSMnU/CwZ9Gnk/ClT5Y+AcUyozzSQ7yJdcPQyCkqQncfi3 EhVGKnRlv0Sh4UgW7oOShPM8BlMIxqXHySOKCefExhXNArcqhLdUAFirqp9au3fGLv0B twviXhSECAcMEVw+NnoAXyqNliVutKeFuu+j2zBosBVQ+8Xb31t6EUuKbXgvV6hq2+KK 5oCw== X-Gm-Message-State: AC+VfDzIW5uoXe1RLJzOV7hyxBLQvVlO/XNC3CvePFuqcPFlVUwBs9vk 302gdZqky99zc6dnWuCDL+uh21nkJ/GJxVCcnaQ= X-Received: by 2002:a17:90a:e68b:b0:250:8f09:75f2 with SMTP id s11-20020a17090ae68b00b002508f0975f2mr28857460pjy.22.1684266323347; Tue, 16 May 2023 12:45:23 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:22 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 40/80] tcg/arm: Use atom_and_align_for_opc Date: Tue, 16 May 2023 12:41:05 -0700 Message-Id: <20230516194145.1749305-41-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52b; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org No change to the ultimate load/store routines yet, so some atomicity conditions not yet honored, but plumbs the change to alignment through the relevant functions. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/arm/tcg-target.c.inc | 39 ++++++++++++++++++++++----------------- 1 file changed, 22 insertions(+), 17 deletions(-) diff --git a/tcg/arm/tcg-target.c.inc b/tcg/arm/tcg-target.c.inc index e5aed03247..add8cc1fd5 100644 --- a/tcg/arm/tcg-target.c.inc +++ b/tcg/arm/tcg-target.c.inc @@ -1323,6 +1323,7 @@ typedef struct { TCGReg base; int index; bool index_scratch; + TCGAtomAlign aa; } HostAddress; bool tcg_target_has_memory_bswap(MemOp memop) @@ -1379,8 +1380,26 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, { TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); - MemOp a_bits = get_alignment_bits(opc); - unsigned a_mask = (1 << a_bits) - 1; + unsigned a_mask; + +#ifdef CONFIG_SOFTMMU + *h = (HostAddress){ + .cond = COND_AL, + .base = addrlo, + .index = TCG_REG_R1, + .index_scratch = true, + }; +#else + *h = (HostAddress){ + .cond = COND_AL, + .base = addrlo, + .index = guest_base ? TCG_REG_GUEST_BASE : -1, + .index_scratch = false, + }; +#endif + + h->aa = atom_and_align_for_opc(s, opc, MO_ATOM_IFALIGN, false); + a_mask = (1 << h->aa.align) - 1; #ifdef CONFIG_SOFTMMU int mem_index = get_mmuidx(oi); @@ -1469,13 +1488,6 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, if (TARGET_LONG_BITS == 64) { tcg_out_dat_reg(s, COND_EQ, ARITH_CMP, 0, TCG_REG_R3, addrhi, 0); } - - *h = (HostAddress){ - .cond = COND_AL, - .base = addrlo, - .index = TCG_REG_R1, - .index_scratch = true, - }; #else if (a_mask) { ldst = new_ldst_label(s); @@ -1484,18 +1496,11 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, ldst->addrlo_reg = addrlo; ldst->addrhi_reg = addrhi; - /* We are expecting a_bits to max out at 7 */ + /* We are expecting alignment to max out at 7 */ tcg_debug_assert(a_mask <= 0xff); /* tst addr, #mask */ tcg_out_dat_imm(s, COND_AL, ARITH_TST, 0, addrlo, a_mask); } - - *h = (HostAddress){ - .cond = COND_AL, - .base = addrlo, - .index = guest_base ? TCG_REG_GUEST_BASE : -1, - .index_scratch = false, - }; #endif return ldst; From patchwork Tue May 16 19:41:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682405 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp20454wrt; Tue, 16 May 2023 12:57:16 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5EUzGEsLSqvjPSjwotLFRH6Ve4R6gBS26wypSoQ0tlYrAJvwgWO7FcNAJxuidP6pWBKQ1D X-Received: by 2002:a05:622a:5c7:b0:3f3:845f:122c with SMTP id d7-20020a05622a05c700b003f3845f122cmr51689067qtb.34.1684267036031; Tue, 16 May 2023 12:57:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267036; cv=none; d=google.com; s=arc-20160816; b=p3ARUp9cU3Ue/iaVlBUqFSRv4j7xnulOghnaNKIFxTKN6bT/5iVZA+AKltEEsLuSrT lyptRtflzdCpd62kNR7hqny+3ECR6z4afqMNbXUTojLVbMqwGrXWsV5dIwx+byfvj91R hQx6Kn7N7EzfJVrSRP4v11QbrJrJ54+vt3Lr1XuWKiMXFgONWBEIzNRK8YbPNuZGMny/ H+XJbCOEDXVrOgNyEuVUp301VjxSDLguVYEGpkxrWQ5csNW4QHbS02evkYTNg8OP0mHl X739+ij05Dz/aQ3BH0PJ7XveGsf5C/SoA0sNljgsBPYTERSmUXPc82D7j89O1LuCePy2 zQ6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=ls4ciHximYxj1CuBwfXZE8IEsihukuIHW0VGJUoZUDc=; b=g7T9cx1XsAir1bYNEDEkKBMlMqkpCJgZzImCCo6GVWjs+Y6hfISuzoTQdeLtCydJ8C 0zzJPrAzf8P01r0Wv+tRLxzkUDiQGsOpY6L4GaiBTBogG071FO921oxioX2yXVvCwJHF c5F8kta6tJi+VkvgM7kF6+CtLeELbtiEdU3ek/Sy01fAMOmd9Vh5k69phbfA9rYd+Hod RiHJ5Lrgw28/0U53FAqNDf4e0PjjBXabJE42I1FtdcLgyCANJvn1FWTccQODhyWtmoz3 ymjCkLP2Vu7ir5L9VOaC0+hqUbvgKSb/0Un4bFGDEkTdWzOPTxvpHatC6AJuxiKPtQgy cLzA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="DK/zLicn"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z7-20020a05620a100700b00751324dbab4si200130qkj.687.2023.05.16.12.57.15 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:57:16 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="DK/zLicn"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0cK-0000J4-EY; Tue, 16 May 2023 15:45:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0bv-0008Jl-T8 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:28 -0400 Received: from mail-pj1-x1034.google.com ([2607:f8b0:4864:20::1034]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0bt-0003fR-UN for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:27 -0400 Received: by mail-pj1-x1034.google.com with SMTP id 98e67ed59e1d1-24e2b2a27ebso48345a91.3 for ; Tue, 16 May 2023 12:45:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266324; x=1686858324; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ls4ciHximYxj1CuBwfXZE8IEsihukuIHW0VGJUoZUDc=; b=DK/zLicnIRp3xgwKDUSda/vBu7hRVT5YmOqFi4tJI84PA2hcaphlW27/k8gqnITEs0 7KbDggdv14bJ5u+fCLW++YbR2ibXbtas+p72XpLe32hIGOAEGJI6U/udUvCQVK5s0JXw gTM9yPiRkXjD2CZmur5s8pdZV/FYgkD9SY7fuWTKfRi0k3a5ez6JWT/CO6A5zjtUzQPm ZQlF4wN2kKWUmGXDRsa87he0PI4Jftr0rgDCPE61l7ZlUVKrYHbGf1rEtAMXKOiteJEk G6pgxdarXe8tWVAYDE2o9q8kgnxdC56My+rFAUy2J/dblSQxMzDv7rvEBXz2L83OkBgy iI4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266324; x=1686858324; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ls4ciHximYxj1CuBwfXZE8IEsihukuIHW0VGJUoZUDc=; b=kuxTdkzIqQ+6QC8J58nlJ60XgL/8sNvSiktdepme1b8ZAh96mtuzFl+W6xv3jzr5X7 gLw07yYOMH9NoR7JIu8kcJV0iVqIffjmKI93CZ3gPGQpHI9VfGHZpsrTYwcNZ98N3YLb F2QiyfCcdzVJJPzsOAhrJqzjhiuGskbop7rq+fgsZ7Mq1cmwFfmWo7svHw1mwRYR3ZHn fMPrhUo8rGwelDGh/cTplQTwMWg9AVrwPEIUh2ip9Q/6a1l2iA9xX2iM893WGIBKPg7s LQfidgSUYP4Y+wCg2kSX7HiBZv1fyYf3vHtfytNwII2M7EUS9/69xmIKLTv5N/U6sbjp Eh6w== X-Gm-Message-State: AC+VfDxsxTUeggbhfS6MFKXKnwpB9T4dFweCGytI7JPVeDfp+7Xn3iig E0x0wN9FrSjqkLLExRZE+V1hpNU1+jGMUWWeQXE= X-Received: by 2002:a17:90a:3841:b0:24e:14a4:9b93 with SMTP id l1-20020a17090a384100b0024e14a49b93mr39022148pjf.43.1684266324261; Tue, 16 May 2023 12:45:24 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:23 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 41/80] tcg/loongarch64: Use atom_and_align_for_opc Date: Tue, 16 May 2023 12:41:06 -0700 Message-Id: <20230516194145.1749305-42-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1034; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1034.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/loongarch64/tcg-target.c.inc | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tcg/loongarch64/tcg-target.c.inc b/tcg/loongarch64/tcg-target.c.inc index d26174dde5..07d35f92fa 100644 --- a/tcg/loongarch64/tcg-target.c.inc +++ b/tcg/loongarch64/tcg-target.c.inc @@ -826,6 +826,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) typedef struct { TCGReg base; TCGReg index; + TCGAtomAlign aa; } HostAddress; bool tcg_target_has_memory_bswap(MemOp memop) @@ -845,7 +846,10 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, { TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); - unsigned a_bits = get_alignment_bits(opc); + MemOp a_bits; + + h->aa = atom_and_align_for_opc(s, opc, MO_ATOM_IFALIGN, false); + a_bits = h->aa.align; #ifdef CONFIG_SOFTMMU unsigned s_bits = opc & MO_SIZE; From patchwork Tue May 16 19:41:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682410 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp20538wrt; Tue, 16 May 2023 12:57:32 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7hEkBo6eaOhFv9CcIt+Lgr9KtLV4rAywMfU9du9TLApVrKGMr0ZjqZLdcQHYl6nkQOisPW X-Received: by 2002:a05:6214:3008:b0:621:53d1:3c66 with SMTP id ke8-20020a056214300800b0062153d13c66mr36240886qvb.18.1684267051934; Tue, 16 May 2023 12:57:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267051; cv=none; d=google.com; s=arc-20160816; b=DZHlwo9+2ae8qcMQtXSWTbzoTqige3zo6fE92g8HwBKK5anTuymJMG2yXIKyE52yKu AHi3bipZoEilr4DyqvhI22n3pn0yjqrkrsffGOFBFUAffNA6DCdSwqgqHzFDfQmmByjD 5lWoYShKIPcdq8NQzLRhK5dDmY07yPJpwZ1DLw4upw98QgbrylFV774nKYe/y4hUi5jS vkBmbatT+LlbJZHE+CZm8N4g+weVHPR1X5LwT3tJk9JbBIyM80POQ3P/POeQkFWYQSPR wavKk3vgxbWL8DLkyG0J0osU3u6DNU8PI0g1sAjAF48XAJxPbrZAnX6r1Ey4oCz36HSb YV1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=COeCBcDhyHAzyrUtiEtch7tCJeKP75ntgsjZMHrLAcw=; b=ifyF35N4TFVWtAzQCb4QZxNTJlBinI2grSEj4dxAmFl+eTO2DMp/SKkTwzMcKITxod E1OGxRt20Fc10HMcBPRKIiLIEfgVCGwAXWEDvmz2TrdWq2GW3Mu3WXMY+YbGC32X6spc g6pqLQwhoDiW6CV2lvXuYOon5HrLOiv5lunlPC2y7oex4FmYFZ/8J51yPZ/ksov8soGz SCsyHXqdAGBxucZNy5hkE7TpBb3yg/FP0sf/OOS2Ej9N/dofZwFEAzyzQ9DqMCD/S7Y4 vZ6VEF/WIIR98eYvMtyk8I+AcORI5dN85NaoR8uofXTjBW1aEoamQ3ngBbM8JiQIhT5D kz2g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="ST7KVI/X"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 8-20020a0562140d4800b0061b7310f27asi12524614qvr.582.2023.05.16.12.57.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:57:31 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="ST7KVI/X"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0c0-0008TD-Sj; Tue, 16 May 2023 15:45:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0bx-0008Ou-S0 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:30 -0400 Received: from mail-pj1-x1033.google.com ([2607:f8b0:4864:20::1033]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0bu-0003fy-RE for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:29 -0400 Received: by mail-pj1-x1033.google.com with SMTP id 98e67ed59e1d1-24dec03ad8fso49125a91.1 for ; Tue, 16 May 2023 12:45:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266325; x=1686858325; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=COeCBcDhyHAzyrUtiEtch7tCJeKP75ntgsjZMHrLAcw=; b=ST7KVI/X+uG6HJ8ZuY28QOQwO49NpfLn8kNqobVDey1otUh1nCB/V2qlokNOJsPX42 Z+UOwRTNY4PO74quE3k+eXt3SDCcMIBJHWAYmHVuKjR9QcQDsaq4K0NfJDM3L0RRp7Jm /6sCaIn6+bDvM0NNi4PCt/hyGZQfYoGVTyDxdgXXcXWrsWpentqUG/sqW2Lg7pPq/yES qAT3lpoLFPMb0aummrYd7VTpitJPhKtMeGXdGJacBP0NMLR6X8zsci+YManoPlvzNG9U 96ean41NrDzFo1wRwr8ZDhAE9dP5pvfUwzZcIbsGYw9siSDVAg5drovWwgGXzEFq7Jbr pt+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266325; x=1686858325; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=COeCBcDhyHAzyrUtiEtch7tCJeKP75ntgsjZMHrLAcw=; b=KNfXEuCErtywHYA/Rz2JYJiaithC9MlGdXt6eWh4Rcqx6DemBeuR5uw50BF+ygzneK qzR+VQ/q1qGDEhhhfgx8x8nfGHvozv2X1hWgy3GAxKmNUPJmQSGIMTnPgupsMHLTKOuR n48ncScQpbLhKBVPGFSjig57X3t0fdxsS+1p5hB8aL8z8VOqJalEyzYY3UkleBh6VOhr T1ObAql2uEsVOGnLKhaQdO1FVgC2mE9Y9P5w0E1mg9qimrB5mm6bs7AYRa2YBWB4ihCH IS5kF8IcCT+67L0yeMxYuhCYl5+TlLRTN3fzGdkx4cRVKqJTXgZ+JMQSvctAvXhdv9l5 UhYA== X-Gm-Message-State: AC+VfDzKOBGqYOubv4rZSPEiDePLgxd2hhfvgDr47sX7VjWd5sl8GVz4 MCOiHB6iJcViuKHApMgwOrgbwf3Xb6p03+8dkto= X-Received: by 2002:a17:90b:38cb:b0:252:b95c:a701 with SMTP id nn11-20020a17090b38cb00b00252b95ca701mr15376666pjb.32.1684266325267; Tue, 16 May 2023 12:45:25 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:24 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 42/80] tcg/mips: Use atom_and_align_for_opc Date: Tue, 16 May 2023 12:41:07 -0700 Message-Id: <20230516194145.1749305-43-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1033; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1033.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/mips/tcg-target.c.inc | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/tcg/mips/tcg-target.c.inc b/tcg/mips/tcg-target.c.inc index cd0254a0d7..3f3fe5b991 100644 --- a/tcg/mips/tcg-target.c.inc +++ b/tcg/mips/tcg-target.c.inc @@ -1138,7 +1138,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) typedef struct { TCGReg base; - MemOp align; + TCGAtomAlign aa; } HostAddress; bool tcg_target_has_memory_bswap(MemOp memop) @@ -1158,11 +1158,15 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, { TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); - unsigned a_bits = get_alignment_bits(opc); + MemOp a_bits; unsigned s_bits = opc & MO_SIZE; - unsigned a_mask = (1 << a_bits) - 1; + unsigned a_mask; TCGReg base; + h->aa = atom_and_align_for_opc(s, opc, MO_ATOM_IFALIGN, false); + a_bits = h->aa.align; + a_mask = (1 << a_bits) - 1; + #ifdef CONFIG_SOFTMMU unsigned s_mask = (1 << s_bits) - 1; int mem_index = get_mmuidx(oi); @@ -1281,7 +1285,6 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, #endif h->base = base; - h->align = a_bits; return ldst; } @@ -1394,7 +1397,7 @@ static void tcg_out_qemu_ld(TCGContext *s, TCGReg datalo, TCGReg datahi, ldst = prepare_host_addr(s, &h, addrlo, addrhi, oi, true); - if (use_mips32r6_instructions || h.align >= (opc & MO_SIZE)) { + if (use_mips32r6_instructions || h.aa.align >= (opc & MO_SIZE)) { tcg_out_qemu_ld_direct(s, datalo, datahi, h.base, opc, data_type); } else { tcg_out_qemu_ld_unalign(s, datalo, datahi, h.base, opc, data_type); @@ -1481,7 +1484,7 @@ static void tcg_out_qemu_st(TCGContext *s, TCGReg datalo, TCGReg datahi, ldst = prepare_host_addr(s, &h, addrlo, addrhi, oi, false); - if (use_mips32r6_instructions || h.align >= (opc & MO_SIZE)) { + if (use_mips32r6_instructions || h.aa.align >= (opc & MO_SIZE)) { tcg_out_qemu_st_direct(s, datalo, datahi, h.base, opc); } else { tcg_out_qemu_st_unalign(s, datalo, datahi, h.base, opc); From patchwork Tue May 16 19:41:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682404 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp20384wrt; Tue, 16 May 2023 12:57:03 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6nW9btK/Nys6rRWuFRcEOQxQ6xFvklLpOnRHoLARdyXBb4tc2S7gzknCx9piCwexpoByhh X-Received: by 2002:ac8:7d04:0:b0:3f3:96c9:b075 with SMTP id g4-20020ac87d04000000b003f396c9b075mr46503745qtb.31.1684267023259; Tue, 16 May 2023 12:57:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267023; cv=none; d=google.com; s=arc-20160816; b=PDpHq8Bc7sMjIc2TG1wwzRsPoeligI3Wcg9by6GvQLmtcop4zwWmkohslS4CaN75xf 9+qMuDXLD5GrBGA2IR5W3rBGV41btW3KRx7RIpz6cyld/AyuMRW5zQDcABGYxYRyPEFQ ceVlIxNkQPe3I+y5iZebtg4EkLDAWFpfZJDH1SLkzxuheKH9YJELN7bcYhpfW25a6uWc fgFM+d+ZFNjbWlfcMQM+ZhR6dn3P38D2Fh2T27bfAGkwd/YVzd8iX8N1MDoaID7COt98 R1tgbGPX9CXQYZKGDdw9WbiB2N6/PXrESCRy6dWwO+zfRfD+xX0IRFNtRsHZsMXWK4ol +F3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=UBFQEHcsaNEVnrjFS9YpHI6iX7jr4WbeYOSUWmY+ntw=; b=E1fv/wl9gIC9lfQkdMmPQbRAUz11jqd5tlCIOyNKTQDT5g5MKB7H3T3TKnwz4CB1HL AKnTBjG0VY4vFuSphuRvQeWFVK3NqnGIfBquFR3VDjsTHbzNx8opSUJO+/nrADjQ8aKE MT6ubtpBjdAscd8NNARDQnSN/VpMhA5MnEYZWl7/WyqI19UqOx0D6krqsWmVL0WiPJDu CqezG07XGPs9GiZyQYvXlsQ0/MxeJ043pYApEADl017kSNIRWO38QjtjZOktWTvAh1LI LpLZ0MZGhfxP4n7bFPFM4GoAM5PwmAZnkohF5PvFhTAd6JjBI7cWJYPPOA1t08pCsoPX 0cYg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=twvpdnb0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id dc7-20020a05620a520700b0075544fda9ffsi188814qkb.744.2023.05.16.12.57.03 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:57:03 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=twvpdnb0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0f7-00047J-9Y; Tue, 16 May 2023 15:48:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0c0-0008Rr-GS for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:32 -0400 Received: from mail-pl1-x635.google.com ([2607:f8b0:4864:20::635]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0bv-0003gk-M2 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:31 -0400 Received: by mail-pl1-x635.google.com with SMTP id d9443c01a7336-1aaebed5bd6so450625ad.1 for ; Tue, 16 May 2023 12:45:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266326; x=1686858326; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UBFQEHcsaNEVnrjFS9YpHI6iX7jr4WbeYOSUWmY+ntw=; b=twvpdnb0549R7e6OSvB6vlr90hhbIhdPooFvRFj1RdNK5XziHFwQu/URlzMNj2mwMR g3fgrCXOL9TLMf4Vo2fYtK/4fHUbBjQwLpXyLtmlZWQuJOoHeBY1ZCBfy2eRv+wysjnI Nr+utTvAAHUu5RbjvzZ4RUt8w6P6Pq0e0W9Z8sGHyd8iTkwWXfsUVoBJkbE+sTF3wuG1 nranBa/EuGs+DaY5HUVwueUvWIgjcrOKPQt/BzK0TZJnjhDvXR1xv7fzTxp641IKD//m wsQu9HY3qSIhN8N7bRHuAnqUqzCitroTSDW3HDZIVDuhcGV8VTm30EXE95jjE1Bl4BEa IQkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266326; x=1686858326; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UBFQEHcsaNEVnrjFS9YpHI6iX7jr4WbeYOSUWmY+ntw=; b=Q640oJp1i6gpstC9AM9yzasLIWRjhH2fqxlU0O0DNcYJXxwPp1tvG10d3EOiMXDo7u SiGpsohTTHDlsOy+5Ie/ltvbDponcywhPIcldT0qPlTtyR6dix9Hx8RGWXkQMQu6FtZo t++AVWyLDxd7FDgjtMpCOvaN0QFb+idRXhAAll981w8y5uauj41c06aImlaGqRDRtdWs MiIOP0NBaNWeQn4vUb/x4bCXO8PHKe0SQ2OpcNPXRiO0I2GCRbuAU27Kzx6KOBzA1bCu x7bQDoRCSlgF4SzuZfZbUXif/Qa6uTH7HUG8Q7rMfAtSLnLeqAJm5G9SO8WxzdFDT7np BxrA== X-Gm-Message-State: AC+VfDzT404IfcVP6yTc+9G5mVg2sFRavDNpn89pvDCeIR5XjNmk/K75 c1m/y5oVf5C8tdvLcxqDJt1dejrb/7RuSUyktOc= X-Received: by 2002:a17:902:ef96:b0:1aa:ce4d:c77d with SMTP id iz22-20020a170902ef9600b001aace4dc77dmr36128283plb.24.1684266326193; Tue, 16 May 2023 12:45:26 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:25 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 43/80] tcg/ppc: Use atom_and_align_for_opc Date: Tue, 16 May 2023 12:41:08 -0700 Message-Id: <20230516194145.1749305-44-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::635; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x635.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/ppc/tcg-target.c.inc | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc index b62a163014..b5c49895f3 100644 --- a/tcg/ppc/tcg-target.c.inc +++ b/tcg/ppc/tcg-target.c.inc @@ -2015,6 +2015,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) typedef struct { TCGReg base; TCGReg index; + TCGAtomAlign aa; } HostAddress; bool tcg_target_has_memory_bswap(MemOp memop) @@ -2034,7 +2035,23 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, { TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); - unsigned a_bits = get_alignment_bits(opc); + MemOp a_bits; + + /* + * Book II, Section 1.4, Single-Copy Atomicity, specifies: + * + * Before 3.0, "An access that is not atomic is performed as a set of + * smaller disjoint atomic accesses. In general, the number and alignment + * of these accesses are implementation-dependent." Thus MO_ATOM_IFALIGN. + * + * As of 3.0, "the non-atomic access is performed as described in + * the corresponding list", which matches MO_ATOM_SUBALIGN. + */ + h->aa = atom_and_align_for_opc(s, opc, + have_isa_3_00 ? MO_ATOM_SUBALIGN + : MO_ATOM_IFALIGN, + false); + a_bits = h->aa.align; #ifdef CONFIG_SOFTMMU int mem_index = get_mmuidx(oi); From patchwork Tue May 16 19:41:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682375 Delivered-To: patch@linaro.org Received: by 2002:a05:6504:558:b0:22d:db33:bb6d with SMTP id w24csp640394ltp; Tue, 16 May 2023 12:46:58 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4lG8Mz21AHl6p/JM41uA6SIu+Ug4jHnJH5ugt2iEp/8w6fBa/5di/Jy+MckEKWJIOGQcBI X-Received: by 2002:ad4:5b8f:0:b0:623:66d3:e538 with SMTP id 15-20020ad45b8f000000b0062366d3e538mr12098380qvp.29.1684266418208; Tue, 16 May 2023 12:46:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266418; cv=none; d=google.com; s=arc-20160816; b=g9c+OpEwDmDxh1uYKzNgi8YXPmNnMhgdpQBf69MAmSTz6/kvNBSS+q1ROeQ+tmuSjX X3UTmMYTbdy54hePClVDyVYfmdOrsyJi6O0xf4SuNURzdk6LJXmqlbUynnHu0NUoOFZs kbc8KJTnPNX/HRA1pJubdkxz9BOpTS49xKkYRwmdCz6VjcfFzbAicr3HLpYTDuiGQFfj uoyKF8KYXHE6yeWNcUsoRXfjtJJuB9TtPbB7BMU9Her1YFLqkud42Ln7W3lwXdhM05B+ OWMt1UKJ+FQw2xPsBwz5CuLD/nzyigEACf44rYNgY2AL5Qou5ii0Vf1z8Y64pjrEE0B3 8kfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=9oD9XgB0lT5e/0MS35T27z73KdHZgG8NiY9ZAZ4mWDU=; b=pdWbeY3qVywL4DhMY0/1XpqEn9N2YYQUzNwSMV5Vn4IZTQMXPy17BzOA806/TAGDNS 0sy8hwPH490G/NzllD2X+JMearJmeKTV6Gly0OnMhkfpA8Vqe6o2gwj6YMo9p2xLUIFK lwn3LQt8EbxOU6ERGGO92eadpu6av30EJ6VHpJ6kGBCrGvi70+06h0SN/Ri/8jtT6S2s eykz+Gmj220E2RB4S5xiTogBVLTvzA33IP14G+EKv1kkhYB3Xl3S/9FhI5fPInnMgwO5 4BWZbqx6URrQzBo2xmcWkp9sr+IMBmZZy3EVoXqrk3YtgIJcFZArJpqsotAHHdm2dKnv ZI7w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ceWK3sC0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id kj13-20020a056214528d00b0060383305b81si11806673qvb.22.2023.05.16.12.46.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:46:58 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ceWK3sC0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0cS-0000Y2-PD; Tue, 16 May 2023 15:46:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0by-0008Q6-Kd for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:30 -0400 Received: from mail-pl1-x632.google.com ([2607:f8b0:4864:20::632]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0bw-0003h0-S7 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:30 -0400 Received: by mail-pl1-x632.google.com with SMTP id d9443c01a7336-1aaea43def7so444675ad.2 for ; Tue, 16 May 2023 12:45:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266327; x=1686858327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9oD9XgB0lT5e/0MS35T27z73KdHZgG8NiY9ZAZ4mWDU=; b=ceWK3sC0V1Z8bJ2fNVIAULGuuGwHUajQ1bGat00F+WkEtv0qkvkudqaR17KeviPSEq YVv8oU5rWJlePwhLQZXUg2QDlfxa49Znji7WsMNVRhAxEdvpZiXg8y3d0dktW2lNERWa BlWVtsKTkl46/cJVBluowUalUmLucAhVe0YlV4B6477AsrqoadJhWs0/ZU/1aMZvfN/p mB3jlqZTVpx6EzgTdvRzxujtTMNpUhPMdMzBY/3PIFlYP0Mxj2Ku8IU2ZXcme7SahmvR oZXStAAO4UwFk1EhHLyNVpPV1KEWAs8u33lYZNelsRmWsFOs5DiY+YljQH6tCEkf18AK hAmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266327; x=1686858327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9oD9XgB0lT5e/0MS35T27z73KdHZgG8NiY9ZAZ4mWDU=; b=Da07OKv5WTHPqbNG5ouCTPPW/wK0jgYCaC/6407r3rZxdAeoWXLl4lEeLU37eESHDG VELwQ5HepWGz8qWwN4zK6pGj+6BEsPkQTea6HFosbaU0NIsf55rx4JHiT14l2XQvfY45 HmqanCyvGunJi6XMRSqRFum4AVasmD6l4s+rERoSzZRojGSVaHtVhfMsYm8wVVNiSbKV AYtdpEw0jZR6YN1OtL4eq7QW2HK7IeWjA6HhsaY9FHdKd9Yl303EEDxvu/Bfvr/2m3h/ Ql1bBCq5QRnIIY22l6SrGDbB0yD5yOH+Zb0qCgDwEsU1NTK9tXgEywdJwDbYCUk8MxwE BQ9g== X-Gm-Message-State: AC+VfDxIA94gJUcWzPUfEjNiambsme2VxnPyc6+q4f1jY0lHs9tcESOO BlDJlAO/5F71EWuZYNhGYpb3EkLzBtQgWq0pnPM= X-Received: by 2002:a17:902:ced0:b0:1ad:c736:209a with SMTP id d16-20020a170902ced000b001adc736209amr21467118plg.56.1684266326970; Tue, 16 May 2023 12:45:26 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:26 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 44/80] tcg/riscv: Use atom_and_align_for_opc Date: Tue, 16 May 2023 12:41:09 -0700 Message-Id: <20230516194145.1749305-45-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::632; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x632.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/riscv/tcg-target.c.inc | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/tcg/riscv/tcg-target.c.inc b/tcg/riscv/tcg-target.c.inc index 37870c89fc..1fc1a9199b 100644 --- a/tcg/riscv/tcg-target.c.inc +++ b/tcg/riscv/tcg-target.c.inc @@ -910,8 +910,11 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, TCGReg *pbase, { TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); - unsigned a_bits = get_alignment_bits(opc); - unsigned a_mask = (1u << a_bits) - 1; + TCGAtomAlign aa; + unsigned a_mask; + + aa = atom_and_align_for_opc(s, opc, MO_ATOM_IFALIGN, false); + a_mask = (1u << aa.align) - 1; #ifdef CONFIG_SOFTMMU unsigned s_bits = opc & MO_SIZE; @@ -944,7 +947,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, TCGReg *pbase, * cross pages using the address of the last byte of the access. */ addr_adj = addr_reg; - if (a_bits < s_bits) { + if (a_mask < s_mask) { addr_adj = TCG_REG_TMP0; tcg_out_opc_imm(s, TARGET_LONG_BITS == 32 ? OPC_ADDIW : OPC_ADDI, addr_adj, addr_reg, s_mask - a_mask); @@ -983,8 +986,8 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, TCGReg *pbase, ldst->oi = oi; ldst->addrlo_reg = addr_reg; - /* We are expecting a_bits max 7, so we can always use andi. */ - tcg_debug_assert(a_bits < 12); + /* We are expecting alignment max 7, so we can always use andi. */ + tcg_debug_assert(a_mask == sextreg(a_mask, 0, 12)); tcg_out_opc_imm(s, OPC_ANDI, TCG_REG_TMP1, addr_reg, a_mask); ldst->label_ptr[0] = s->code_ptr; From patchwork Tue May 16 19:41:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682386 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp18350wrt; Tue, 16 May 2023 12:49:56 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6bBa7yEPmZluWhqPvegfvRUl5hkOrKNu8Job0njpcRP8aAn0vH80br9I/1i+9fehp9ot0T X-Received: by 2002:ac8:5c10:0:b0:3f5:1bf2:8b90 with SMTP id i16-20020ac85c10000000b003f51bf28b90mr17365504qti.13.1684266596527; Tue, 16 May 2023 12:49:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266596; cv=none; d=google.com; s=arc-20160816; b=xXxGd9vmXusVyRyCGX80yMZ+aENN8k3a0j9fTK7dIqLyHkqhPQBHFDuuSDg+aXcJf9 UfwtXhvMk9Y88s7Wnx8/yF+zZw1kYEglQGPAFwWjjhiBYkGE83V4zKUfbjH8qcbfi6V8 6InQQWMQ2f8NL4YwPAu+RhTYmZJqHrksHxCBxSQ+YntM8Rhl37gT4aKpgR6L8pUZRQjk gdNqa50oBProbLVbJ3HvLmnu1pKNB1dtDJyZd+cXq0gEraVLCxawXXVoTeJza1s3VX6C cfXAy/NZA02V7Qq+/Mnewt+zX1X0X25PAH8BrEzveGIX7vjGzwWVsHWiAEf9x8ItXnU4 AiAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=NDknFLWtWBKU8G+ToympZ5KLvdGdhGS895Fc1ZFjry8=; b=ZJMDMzrOO8Ie+mSf3NM5s3NYzzo3Sw2oTebDh0I6h26iFF7lMpvuh4mvHqCTOo03TB dBxTReyxX+uVkr5t3+wEYktb59d8Xyoa03lZmNK4jw6IegjYe2jMaJTDE8QDC9Tf8BA9 PSMEq9AJly//lomlpKZg5myaCi2sODuOK+KSuSiCXA12jy6hEvFf3O3gotZEU3wBMr8R MB2z3Zm7ZGhdhyIKHtmaVunhZJM+TEA8xrbmYEXjrWGreyV4TmNhKaZuo6z8Wycv2ptL hRVcxREv3C6SPoDSzWf2BBHWIEcvrGK9yuwfjYAKRLysDyfIQONobh398Dpu67r+tQ3r /+nA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ssM1JCAN; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 15-20020ac8594f000000b003e38a0b8c73si11834159qtz.453.2023.05.16.12.49.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:49:56 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ssM1JCAN; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0cW-0000lV-IY; Tue, 16 May 2023 15:46:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0c1-0008V5-BF for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:33 -0400 Received: from mail-pj1-x102b.google.com ([2607:f8b0:4864:20::102b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0bx-0003hm-An for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:32 -0400 Received: by mail-pj1-x102b.google.com with SMTP id 98e67ed59e1d1-24decf5cc03so54197a91.0 for ; Tue, 16 May 2023 12:45:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266328; x=1686858328; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NDknFLWtWBKU8G+ToympZ5KLvdGdhGS895Fc1ZFjry8=; b=ssM1JCANf8OXdb6Dhy66GB+UqbtGtjMbhXbClq/mg5NrtGbGWILh1Hl5Z0zNYykRB5 fGzqYHAy+rtT7NXVqd9PW7mU0c4j0TCiSVDnQJv9XKiZdLqnp+EFGWZUCPWjz1ccN7aQ fi9gXRem5JXWjzPpBCMcewuFCEya6tAGgqltKXk1Epj4DwCF2g1fGYGfxqwCl+8125qs TIgH8buGcYX+CmmoAC3CinwIdffViEOLHTmxt8f8ro/vpIP4WI0orW/ylhDgjVFT6A/Y ng6Gl7sbSV0K9EibvRO90Y7AhPB4inrilLrroh/86yR2V9qPAKmUm577bsORI0zpiiql U1VA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266328; x=1686858328; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NDknFLWtWBKU8G+ToympZ5KLvdGdhGS895Fc1ZFjry8=; b=h/Kd2hgcsqoL2YM68WjGYEO4yRGAnBTb8Z9QcZUEhD0mnoLg+zI/IGuklKSuw4rCOC iVjnd8U8hUZEuEFIfh9qjLcF2IxfPJCZu8+VHLlTb8aItRg/DiIkZmieRbYsxyGYEfeH 6Qo32DHW6HULq+IBDvwOMqfaLnhAXUvrqdlBuqAXGcZc7Kqxk0hzcvdWrCxtc8qjhUFh M+aHUnKIWhsQd6v6uwKA9fXnEuxVi2NZ6ss1PVHWEAjzKoxEmRwtPTBlBJg9x4nCDNqy NLl+db4TX7krEQiQ6GOiBtePWxDIsumfBUNWVkX6BsJctfzncajFGOjSAc8Pa+q5tPpV 7I5w== X-Gm-Message-State: AC+VfDzT+QoCOBf/G0r8bpXvXWt6lFGvOn36SmKqty1dLwQmLC22Ya9r A5rtgjZ2l0R66U0cL/i/8Qwg5Hgn0EE4QaMSHPE= X-Received: by 2002:a17:90a:ac06:b0:253:36ca:7ea0 with SMTP id o6-20020a17090aac0600b0025336ca7ea0mr1236809pjq.27.1684266327792; Tue, 16 May 2023 12:45:27 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:27 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 45/80] tcg/s390x: Use atom_and_align_for_opc Date: Tue, 16 May 2023 12:41:10 -0700 Message-Id: <20230516194145.1749305-46-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102b; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/s390x/tcg-target.c.inc | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc index 22f0206b5a..8e34b214fc 100644 --- a/tcg/s390x/tcg-target.c.inc +++ b/tcg/s390x/tcg-target.c.inc @@ -1572,6 +1572,7 @@ typedef struct { TCGReg base; TCGReg index; int disp; + TCGAtomAlign aa; } HostAddress; bool tcg_target_has_memory_bswap(MemOp memop) @@ -1733,8 +1734,10 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, { TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); - unsigned a_bits = get_alignment_bits(opc); - unsigned a_mask = (1u << a_bits) - 1; + unsigned a_mask; + + h->aa = atom_and_align_for_opc(s, opc, MO_ATOM_IFALIGN, false); + a_mask = (1 << h->aa.align) - 1; #ifdef CONFIG_SOFTMMU unsigned s_bits = opc & MO_SIZE; @@ -1764,7 +1767,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, * bits within the address. For unaligned access, we check that we don't * cross pages using the address of the last byte of the access. */ - a_off = (a_bits >= s_bits ? 0 : s_mask - a_mask); + a_off = (a_mask >= s_mask ? 0 : s_mask - a_mask); tlb_mask = (uint64_t)TARGET_PAGE_MASK | a_mask; if (a_off == 0) { tgen_andi_risbg(s, TCG_REG_R0, addr_reg, tlb_mask); @@ -1806,7 +1809,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, ldst->addrlo_reg = addr_reg; /* We are expecting a_bits to max out at 7, much lower than TMLL. */ - tcg_debug_assert(a_bits < 16); + tcg_debug_assert(a_mask <= 0xffff); tcg_out_insn(s, RI, TMLL, addr_reg, a_mask); tcg_out16(s, RI_BRC | (7 << 4)); /* CC in {1,2,3} */ From patchwork Tue May 16 19:41:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682378 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp17981wrt; Tue, 16 May 2023 12:48:49 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6fK06c57rGkGXXxIJKMQqV3EZyG3I+WB2DCHzTBFzxWyRlr/oQY43t7dOgmV0P8w4d/s05 X-Received: by 2002:ac8:5a8b:0:b0:3ef:3711:e571 with SMTP id c11-20020ac85a8b000000b003ef3711e571mr54984576qtc.36.1684266529425; Tue, 16 May 2023 12:48:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266529; cv=none; d=google.com; s=arc-20160816; b=qMbC+LA0nl03+/LyvIqwCxQNUUEOrkoz1bvsz9k5OcrCGH2sdXrC+mMHoYPmHnG1xL WfY5eO3Y5vfukyw7G8/inNhh0sSajkEFp2pONKFW9cxjvLNPJOCdnC83d0bFTMnglMvN KwC/taJwvEkZCqXNL3TzptpE80/68NdV1zfYql6b6x0z+hNCZmsOEw4wOMZJ02/OUXii P81kzpoLfmruQiTbFKeTtDa30RRUayanebo0obkehcqJUGddQ0Jy2APU8jE3nGEhRWsr 5rT6j5lvtVLb4y/BgPhBgC8XOkTdX2jJl6MDTKL0ElLaI3lq+wShLcvP2jzPj6JaC+Rz wYeg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=peJEfVzHrcMF90ZjxGIbaEJp8Gj9YCBgCnVRbK/FL9k=; b=YJ5JWUnw1vBfd+ecKymt3z5XUJYxwIYHXXSftEBNChN7CSt3BMy+6cUmQ/W18cMoHU TFMeVffxe+m2Qo9tIVPONxsTly2bVX8+62HzVbMWM8lJ/ILJvQX2Ywyvqoi9EwTOvNbO AYj/QlSvTY7+MJZVjD96AMvoXlRjhGhU/JoqYGhrOZawAMabXKj005OT9K1Xy13a5jNn vnCehYkD9X/QB3TYsVLI9KfKOBmQPDAjCii9b+GFj9RaaurkEQ4l/UAuDwYVoxM2jou/ xsycbNRM2SQOWpx+TdIewZw4mxAYk2vyXfXUB6D1zXto9YKuxTT507HeUrjmOqnUpG/1 Jtdg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=LYReWlvm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y2-20020a05620a25c200b0075152afb056si200736qko.552.2023.05.16.12.48.49 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:48:49 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=LYReWlvm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0cC-000074-K3; Tue, 16 May 2023 15:45:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0c0-0008TU-Kz for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:32 -0400 Received: from mail-pg1-x52c.google.com ([2607:f8b0:4864:20::52c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0by-0003hv-Mu for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:32 -0400 Received: by mail-pg1-x52c.google.com with SMTP id 41be03b00d2f7-517ab9a4a13so13457882a12.1 for ; Tue, 16 May 2023 12:45:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266329; x=1686858329; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=peJEfVzHrcMF90ZjxGIbaEJp8Gj9YCBgCnVRbK/FL9k=; b=LYReWlvmgFOaI1hdUN/H6WF3Fso+et9COcYdU6xRrGX8+9C1JnR/+NYdfhStvE9IqM vN7fmXZZP5mp0yWD22COsrmjGIp0QiA40Hy0Jvckcn9+CEiEOwtzKGjH8fQwAvq5Gc/V gVyO1OhieRVeY+A2SnjTX/S+jhGhS6Chq08tQm/i9AJtrGhjxMmu2O9PmOT8rOVK8IQ/ yvoUBgrJ/FWFiQH/wcH/wxdUHWtufRvn6N4y9GsH1pxdoe3qs9B30xjUdoOwAjRMplL3 25IdB7OAqDT2OlkIRc9maY/xXDDiSLbrveqwrnDkc4kBrtCV0Q/iRhPO1Is2/RfD3OVq WkLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266329; x=1686858329; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=peJEfVzHrcMF90ZjxGIbaEJp8Gj9YCBgCnVRbK/FL9k=; b=Lk7zPOzIAmINttq1A0xRC2dLcL4oa7F7hRx5/4eZS+YptOK1mvNn7VqK4TX/C9v6GT cp4br0izTUtK7EAv5cAOs+XiCPDstoe5BJdXciNLyObKFhfqbzCyP2XjmPWfgD11WkQe /3P1vNKG4Ufrmujlyf9huv48VmvXpTaBHFXYTaAaqc1Dt8BzYjNKDJmAQsIp1up/4SdG WC8bVrrs/zWKAYYhqY13AVZnmqXvYXmIGP60aouv7hI+yEIZMnAcmqn7vrs5C5R7Rxkx w+EcxTxrenxmv2yctsIKmcgqIF09/tzCPrL6hrVR/xg3LGUzdjvkKn6+OkKL8D4H9DAw 0HOg== X-Gm-Message-State: AC+VfDxohKNaOsLooehEfQkRuShCGIEr4bSpD10hztBfYpU4jTAbE98z ztqH3xPZR0Tu5Sv0rfgYS+3vZHEdpfWgan9WCfU= X-Received: by 2002:a05:6a21:7891:b0:101:209e:bc57 with SMTP id bf17-20020a056a21789100b00101209ebc57mr35507622pzc.50.1684266328810; Tue, 16 May 2023 12:45:28 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:28 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 46/80] tcg/sparc64: Use atom_and_align_for_opc Date: Tue, 16 May 2023 12:41:11 -0700 Message-Id: <20230516194145.1749305-47-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52c; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/sparc64/tcg-target.c.inc | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index bb23038529..9676b745a2 100644 --- a/tcg/sparc64/tcg-target.c.inc +++ b/tcg/sparc64/tcg-target.c.inc @@ -1009,6 +1009,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) typedef struct { TCGReg base; TCGReg index; + TCGAtomAlign aa; } HostAddress; bool tcg_target_has_memory_bswap(MemOp memop) @@ -1028,13 +1029,13 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, { TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); - unsigned a_bits = get_alignment_bits(opc); - unsigned s_bits = opc & MO_SIZE; + MemOp s_bits = opc & MO_SIZE; unsigned a_mask; /* We don't support unaligned accesses. */ - a_bits = MAX(a_bits, s_bits); - a_mask = (1u << a_bits) - 1; + h->aa = atom_and_align_for_opc(s, opc, MO_ATOM_IFALIGN, false); + h->aa.align = MAX(h->aa.align, s_bits); + a_mask = (1u << h->aa.align) - 1; #ifdef CONFIG_SOFTMMU int mem_index = get_mmuidx(oi); @@ -1086,11 +1087,13 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, cc = TARGET_LONG_BITS == 64 ? BPCC_XCC : BPCC_ICC; tcg_out_bpcc0(s, COND_NE, BPCC_PN | cc, 0); #else - if (a_bits != s_bits) { - /* - * Test for at least natural alignment, and defer - * everything else to the helper functions. - */ + /* + * If the size equals the required alignment, we can skip the test + * and allow host SIGBUS to deliver SIGBUS to the guest. + * Otherwise, test for at least natural alignment and defer + * everything else to the helper functions. + */ + if (s_bits != get_alignment_bits(opc)) { tcg_debug_assert(check_fit_tl(a_mask, 13)); tcg_out_arithi(s, TCG_REG_G0, addr_reg, a_mask, ARITH_ANDCC); From patchwork Tue May 16 19:41:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682423 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp21075wrt; Tue, 16 May 2023 12:59:30 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ54gXai4tj/uthXSzivDCRfNSs7Nk0M+uubVL9w3xk0PBYP95Ig0ubkShAHma1tpGiYygKc X-Received: by 2002:a5d:55c2:0:b0:306:5149:3aa8 with SMTP id i2-20020a5d55c2000000b0030651493aa8mr27066444wrw.24.1684267170730; Tue, 16 May 2023 12:59:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267170; cv=none; d=google.com; s=arc-20160816; b=geZDsUCx2YwXXMPzSiPHao5S621UsGxHubMMpAfDtfjoJZ4i3oaReQSy3n34uBxaYm suA00CbTF5nCZ3i2mSbbVCd/uc47hZZvFD0kNq+5sKjzJy5ml31F+jo+c5vHBZ6gXifY +5GPBZr9HyRZthd6Sl67Tk0DvOuQfc5R2cqG81u0S7qwwSadTVywdSNHSnvUjjfJXAJr NrLYQwT+9X6nY8JdOTmXclKFpg5t5ZJKN3De73j7BQfeH+rt1zRWvtCFUj6XEPg6E9cI vd2WJ9Hdt7Fq+633YJbuvNDa47H9KwG8ECBmj5Hmhs3aL5a6If/cczKlTOfamklb+YMj z5HQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=e3K+79nBiSc5crGCl8vbMQJ8GibyAa5XexENyD2+9r4=; b=wu5ALMBdmykJzez5EuiLcuSJgbKQ4xBs8wdDWugDpSMN28eNk6BifU+B5MOE78UAQ/ gqXUf9UhxjLrexZCpsTJGIgvJBOFFSq7aFkQ9Yq6m5XMmMmsGEyAWSZyk5xHrcmhwlWi LU/kCqab9gzrCmehdgFs1fDQP/j5XeskEaSCXwbcqHIsIqNFzhlq7KYA/WHfu2Ox4s1q 6DDdt5508hXgqcnUAeI4fnfUDmd1iEqEtnVSsPtmqVPkGqH2KWxrGDxFpawYWC4R+AHs 9N2UUvydXTsvZhH3zsSJ7JPFpI9UgjqSzttTbBcroPxSgfFhVe/fBycxzKWnfpp2Z54d B8KQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ppBwYKMg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w4-20020adfde84000000b003079e5881d1si172917wrl.839.2023.05.16.12.59.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:59:30 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ppBwYKMg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0cM-0000S2-P2; Tue, 16 May 2023 15:45:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0c2-0008W0-5p for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:35 -0400 Received: from mail-pj1-x102e.google.com ([2607:f8b0:4864:20::102e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0c0-0003iP-3f for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:33 -0400 Received: by mail-pj1-x102e.google.com with SMTP id 98e67ed59e1d1-24df161f84bso38991a91.3 for ; Tue, 16 May 2023 12:45:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266329; x=1686858329; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=e3K+79nBiSc5crGCl8vbMQJ8GibyAa5XexENyD2+9r4=; b=ppBwYKMgiuaJj6tOojb90updO3NDRwOMNyEKFUMkcbT/HaF0UR+jEMa8lJiO2+c8QJ 6rBy01m6i34cCT3BpSmkaOkaeI9sz2/zHsGPnR02pykNPAalxFAesDtzdJ0EnrznGhU4 CjSBgI/RsBfJt/XLMWYbvmMd6fcQzZf2+4r7PsYhHR7yJoWbfE10Q2okmaE1ofKcZk4u ZijZ3opbS2il2mJWjQfRE7wu2QvUNfHazZ0WmlrAy9sVV+cVaeOHyXXk90BnuE8DpS6A 4amGUzGFln4x1PikePC5nOyCbXVjvr55Lz6jqF7UwT994gH2JzbPSKlbyXe7x3bbrsx8 QEeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266329; x=1686858329; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=e3K+79nBiSc5crGCl8vbMQJ8GibyAa5XexENyD2+9r4=; b=IQHC4WwT43aCxZjA9GKBWLjeEZ3CdqA53Nc+hlyWQwt1wtUZ/4d+002x4o8WDveQOr N0k4b9VHHlXxyfDSE8YQwCJ+iwjhZzlG6pL8w4Z5oDHaZ0u9sWVqfCb5agxxQEkhhEA5 1z/pBVGgBQ9brod1s5zyiWmkgfD+3UJBUDCtHggpYcqOn4sQ33zJSOD8iN/ufKwjkE5u PDuG60CGqi++wHoN6RBTzVlVpeQxvsCXAoEi1mL/i/EZQWTBXtOGRa16ZwugBFZfI0Ls vKR5yYYQd34ciRmAR1RIWhBwnibGdjhGbU/45m/3tjd5g8VdQQh/pLSvUm8Rkl0+52Da 9k7Q== X-Gm-Message-State: AC+VfDw7QhDkAgxvjsKAhDg3UO15tDU2vgwRV+0qDPVPRggUXFhJdPff X1StRFgTG3Kwm9Bx6uVr4Q1QoZNr1V3v+4BrvBU= X-Received: by 2002:a17:90a:9d87:b0:24d:f77c:71e7 with SMTP id k7-20020a17090a9d8700b0024df77c71e7mr36900436pjp.41.1684266329572; Tue, 16 May 2023 12:45:29 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:29 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 47/80] tcg/i386: Honor 64-bit atomicity in 32-bit mode Date: Tue, 16 May 2023 12:41:12 -0700 Message-Id: <20230516194145.1749305-48-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102e; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use the fpu to perform 64-bit loads and stores. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.c.inc | 44 +++++++++++++++++++++++++++++++++------ 1 file changed, 38 insertions(+), 6 deletions(-) diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index 3b8528e332..0415ca2a4c 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -468,6 +468,10 @@ static bool tcg_target_const_match(int64_t val, TCGType type, int ct) #define OPC_GRP5 (0xff) #define OPC_GRP14 (0x73 | P_EXT | P_DATA16) +#define OPC_ESCDF (0xdf) +#define ESCDF_FILD_m64 5 +#define ESCDF_FISTP_m64 7 + /* Group 1 opcode extensions for 0x80-0x83. These are also used as modifiers for OPC_ARITH. */ #define ARITH_ADD 0 @@ -2086,7 +2090,20 @@ static void tcg_out_qemu_ld_direct(TCGContext *s, TCGReg datalo, TCGReg datahi, datalo = datahi; datahi = t; } - if (h.base == datalo || h.index == datalo) { + if (h.aa.atom == MO_64) { + /* + * Atomicity requires that we use use a single 8-byte load. + * For simplicity and code size, always use the FPU for this. + * Similar insns using SSE/AVX are merely larger. + * Load from memory in one go, then store back to the stack, + * from whence we can load into the correct integer regs. + */ + tcg_out_modrm_sib_offset(s, OPC_ESCDF + h.seg, ESCDF_FILD_m64, + h.base, h.index, 0, h.ofs); + tcg_out_modrm_offset(s, OPC_ESCDF, ESCDF_FISTP_m64, TCG_REG_ESP, 0); + tcg_out_modrm_offset(s, movop, datalo, TCG_REG_ESP, 0); + tcg_out_modrm_offset(s, movop, datahi, TCG_REG_ESP, 4); + } else if (h.base == datalo || h.index == datalo) { tcg_out_modrm_sib_offset(s, OPC_LEA, datahi, h.base, h.index, 0, h.ofs); tcg_out_modrm_offset(s, movop + h.seg, datalo, datahi, 0); @@ -2156,12 +2173,27 @@ static void tcg_out_qemu_st_direct(TCGContext *s, TCGReg datalo, TCGReg datahi, if (TCG_TARGET_REG_BITS == 64) { tcg_out_modrm_sib_offset(s, movop + P_REXW + h.seg, datalo, h.base, h.index, 0, h.ofs); + break; + } + if (use_movbe) { + TCGReg t = datalo; + datalo = datahi; + datahi = t; + } + if (h.aa.atom == MO_64) { + /* + * Atomicity requires that we use use one 8-byte store. + * For simplicity, and code size, always use the FPU for this. + * Similar insns using SSE/AVX are merely larger. + * Assemble the 8-byte quantity in required endianness + * on the stack, load to coproc unit, and store. + */ + tcg_out_modrm_offset(s, movop, datalo, TCG_REG_ESP, 0); + tcg_out_modrm_offset(s, movop, datahi, TCG_REG_ESP, 4); + tcg_out_modrm_offset(s, OPC_ESCDF, ESCDF_FILD_m64, TCG_REG_ESP, 0); + tcg_out_modrm_sib_offset(s, OPC_ESCDF + h.seg, ESCDF_FISTP_m64, + h.base, h.index, 0, h.ofs); } else { - if (use_movbe) { - TCGReg t = datalo; - datalo = datahi; - datahi = t; - } tcg_out_modrm_sib_offset(s, movop + h.seg, datalo, h.base, h.index, 0, h.ofs); tcg_out_modrm_sib_offset(s, movop + h.seg, datahi, From patchwork Tue May 16 19:41:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682387 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp18645wrt; Tue, 16 May 2023 12:50:57 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7Z1vDzOHo85vAVRrqJfAPKOxqr/yiYy0KkHddQO6H5fWWVCJE4bxg2u3qL6vOwgWxXk1t8 X-Received: by 2002:a05:6214:5290:b0:5ef:8159:b9a9 with SMTP id kj16-20020a056214529000b005ef8159b9a9mr58912890qvb.21.1684266657058; Tue, 16 May 2023 12:50:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266657; cv=none; d=google.com; s=arc-20160816; b=SjU+ys69Dn32rQTDYuCSO3/yUegAgxVZpMmb9O0hUTs3ag1Qd9i72Hcb2POL36tsjI t24H3eVS51oZtx8rKMp6m5AZTq9KlvrCe/Yf+RjTWrxK9WgNEByMb5DqpODIoxrwy4Hf L5fPSKiwcnhfyAvoDCZflVr//f41P+5FGLuRzs1mrmOKZIvkaks1QPhVJFurEnGVyDsn RBDY0BGWRowoje4ZClOVBa5jx5AYuFeUINjFhIeFYf5v9rrgOocVA20pWv92WC0qBGXI 9b1ZX4aIh5t9nsFqRPOoqKEnK4SPnfAUJnk3pUcXceK+cWm2WbD6wb7WghER5saMrkLX Z7/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=AMpow73dBXvGA9oYAQFRwfnfJJlscn+XdJ+q82cdRac=; b=bHM7Mow4zwxjW6jGAtquMADHjpla/aqSLblIupgslzEzF9ZavfzXpz4CW5JJ1jZiMt jKf9kMMehHAFv9UrudKQ9xmwSYHhY5pwmy+ChUnaJaTDaJSuANvWbg8YQ0jIaTUMKGq/ iURtUHVskWu6FEHz8Ip6bjbmCLPnjMKzMPvPpsj3Pfmgr24dJhbeYitHnSrXjkmJscaa vCyxgk1i/rfwIKzIqYCb+ZZPoF/TZU2xswS2Zsa9/RQz7iZeI/Y0AaYpR1WIqMyOpve9 bRYeFSSGCPZevV62z0zyLYjxYQ6Ys6jRA6ibqFoqzlmOcpyDPWHrr79frIEmJFgeuz0W xM/g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xgAuzgNf; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v22-20020a05620a123600b00743c569812esi198289qkj.615.2023.05.16.12.50.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:50:57 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xgAuzgNf; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0cS-0000Yu-SS; Tue, 16 May 2023 15:46:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0c6-0000As-S2 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:42 -0400 Received: from mail-pl1-x62e.google.com ([2607:f8b0:4864:20::62e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0c0-0003ic-Tl for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:34 -0400 Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-1adc913094aso583765ad.0 for ; Tue, 16 May 2023 12:45:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266330; x=1686858330; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AMpow73dBXvGA9oYAQFRwfnfJJlscn+XdJ+q82cdRac=; b=xgAuzgNfbsiWQ6VdUoO8ug630/20Vr/OsFgH5R0/BdFzUaCAyHndivpq+Y8uqTaeno ezZlikzDoZ2ogLzIkUi1KwMGxUykW+32Neo5/icYqgesEpInzHOMWSis5zzdkrTuV3Xt 3LRKNSM606eJVH/dkKvkB3VJJ3i63q3l8L7VtesE8GAmBD9LFj9eQQzDEylRE6449gXM VLvwkJ5s1KfM+itJxkTBIkRlt+f4Vqj5SKQ/94jQ5ONQJd2EKPpOhuIHpfKYqTlW+0H0 YcEQma5VKW7O10SgCajnrQMXee61sjmHdeTka2Zqz2PaNSzUh6b7ajaZbbsYZKGP2boh Od5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266330; x=1686858330; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AMpow73dBXvGA9oYAQFRwfnfJJlscn+XdJ+q82cdRac=; b=VOeMIt0RIAUqWk2n03K0eCL0HZTgGmKHgrI90X/efRYXS2rEeshmRyYTtyRnyH1fXn IB/Ztnh/Ep/zxtDABfNTdjBf/IMFrh5XwX37M47btxg0AAYPU9hsCFZHQvrNXnWU2qbq isrLJ9vL63KGmTMTWEEGuF+kW/a89TTFuQNefr9nBLVlg/GKW9nXgjEvPnz2AP1ceAA3 V4gBT/puaI5p5cvAowtSx4W6kehJ/vrsyfnrXNh9lXcTdRy/VLC96h4iaGey+4589hCa BBx/DGxPfrAnVO1BXyUaW89MGfS2SmfLBy/ZcG9i7klGTBTcIXjnxzyrp5sg9AOE6COJ hgjQ== X-Gm-Message-State: AC+VfDy5EMjSvSprPIrv9NE2oUiTeEfmRyiCDSW3WHkVV7OeEIQ2tQHl qyhVlAk9RvJ5T3IkLXgtfxPQhu6v8JI9gpY4jsw= X-Received: by 2002:a17:90b:3a8c:b0:247:40f1:79d7 with SMTP id om12-20020a17090b3a8c00b0024740f179d7mr37840475pjb.34.1684266330416; Tue, 16 May 2023 12:45:30 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:30 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 48/80] tcg/i386: Support 128-bit load/store with have_atomic16 Date: Tue, 16 May 2023 12:41:13 -0700 Message-Id: <20230516194145.1749305-49-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62e; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.h | 3 +- tcg/i386/tcg-target.c.inc | 181 +++++++++++++++++++++++++++++++++++++- 2 files changed, 180 insertions(+), 4 deletions(-) diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index 943af6775e..7f69997e30 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -194,7 +194,8 @@ extern bool have_atomic16; #define TCG_TARGET_HAS_qemu_st8_i32 1 #endif -#define TCG_TARGET_HAS_qemu_ldst_i128 0 +#define TCG_TARGET_HAS_qemu_ldst_i128 \ + (TCG_TARGET_REG_BITS == 64 && have_atomic16) /* We do not support older SSE systems, only beginning with AVX1. */ #define TCG_TARGET_HAS_v64 have_avx1 diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index 0415ca2a4c..b66769952e 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -91,6 +91,8 @@ static const int tcg_target_reg_alloc_order[] = { #endif }; +#define TCG_TMP_VEC TCG_REG_XMM5 + static const int tcg_target_call_iarg_regs[] = { #if TCG_TARGET_REG_BITS == 64 #if defined(_WIN64) @@ -347,6 +349,8 @@ static bool tcg_target_const_match(int64_t val, TCGType type, int ct) #define OPC_PCMPGTW (0x65 | P_EXT | P_DATA16) #define OPC_PCMPGTD (0x66 | P_EXT | P_DATA16) #define OPC_PCMPGTQ (0x37 | P_EXT38 | P_DATA16) +#define OPC_PEXTRD (0x16 | P_EXT3A | P_DATA16) +#define OPC_PINSRD (0x22 | P_EXT3A | P_DATA16) #define OPC_PMAXSB (0x3c | P_EXT38 | P_DATA16) #define OPC_PMAXSW (0xee | P_EXT | P_DATA16) #define OPC_PMAXSD (0x3d | P_EXT38 | P_DATA16) @@ -1783,7 +1787,21 @@ typedef struct { bool tcg_target_has_memory_bswap(MemOp memop) { - return have_movbe; + TCGAtomAlign aa; + + if (!have_movbe) { + return false; + } + if ((memop & MO_SIZE) <= MO_64) { + return true; + } + + /* + * Reject 16-byte memop with 16-byte atomicity, i.e. VMOVDQA, + * but do allow a pair of 64-bit operations, i.e. MOVBEQ. + */ + aa = atom_and_align_for_opc(tcg_ctx, memop, MO_ATOM_IFALIGN, true); + return aa.atom <= MO_64; } /* @@ -1811,6 +1829,30 @@ static const TCGLdstHelperParam ldst_helper_param = { static const TCGLdstHelperParam ldst_helper_param = { }; #endif +static void tcg_out_vec_to_pair(TCGContext *s, TCGType type, + TCGReg l, TCGReg h, TCGReg v) +{ + int rexw = type == TCG_TYPE_I32 ? 0 : P_REXW; + + /* vpmov{d,q} %v, %l */ + tcg_out_vex_modrm(s, OPC_MOVD_EyVy + rexw, v, 0, l); + /* vpextr{d,q} $1, %v, %h */ + tcg_out_vex_modrm(s, OPC_PEXTRD + rexw, v, 0, h); + tcg_out8(s, 1); +} + +static void tcg_out_pair_to_vec(TCGContext *s, TCGType type, + TCGReg v, TCGReg l, TCGReg h) +{ + int rexw = type == TCG_TYPE_I32 ? 0 : P_REXW; + + /* vmov{d,q} %l, %v */ + tcg_out_vex_modrm(s, OPC_MOVD_VyEy + rexw, v, 0, l); + /* vpinsr{d,q} $1, %h, %v, %v */ + tcg_out_vex_modrm(s, OPC_PINSRD + rexw, v, v, h); + tcg_out8(s, 1); +} + /* * Generate code for the slow path for a load at the end of block */ @@ -1900,6 +1942,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, { TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); + MemOp s_bits = opc & MO_SIZE; unsigned a_mask; #ifdef CONFIG_SOFTMMU @@ -1910,7 +1953,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, *h = x86_guest_base; #endif h->base = addrlo; - h->aa = atom_and_align_for_opc(s, opc, MO_ATOM_IFALIGN, false); + h->aa = atom_and_align_for_opc(s, opc, MO_ATOM_IFALIGN, s_bits == MO_128); a_mask = (1 << h->aa.align) - 1; #ifdef CONFIG_SOFTMMU @@ -1920,7 +1963,6 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, TCGType tlbtype = TCG_TYPE_I32; int trexw = 0, hrexw = 0, tlbrexw = 0; unsigned mem_index = get_mmuidx(oi); - unsigned s_bits = opc & MO_SIZE; unsigned s_mask = (1 << s_bits) - 1; target_ulong tlb_mask; @@ -2115,6 +2157,69 @@ static void tcg_out_qemu_ld_direct(TCGContext *s, TCGReg datalo, TCGReg datahi, h.base, h.index, 0, h.ofs + 4); } break; + + case MO_128: + { + TCGLabel *l1 = NULL, *l2 = NULL; + bool use_pair = h.aa.atom < MO_128; + + tcg_debug_assert(TCG_TARGET_REG_BITS == 64); + + if (!use_pair) { + tcg_debug_assert(!use_movbe); + /* + * Atomicity requires that we use use VMOVDQA. + * If we've already checked for 16-byte alignment, that's all + * we need. If we arrive here with lesser alignment, then we + * have determined that less than 16-byte alignment can be + * satisfied with two 8-byte loads. + */ + if (h.aa.align < MO_128) { + use_pair = true; + l1 = gen_new_label(); + l2 = gen_new_label(); + + tcg_out_testi(s, h.base, 15); + tcg_out_jxx(s, JCC_JNE, l2, true); + } + + tcg_out_vex_modrm_sib_offset(s, OPC_MOVDQA_VxWx + h.seg, + TCG_TMP_VEC, 0, + h.base, h.index, 0, h.ofs); + tcg_out_vec_to_pair(s, TCG_TYPE_I64, datalo, + datahi, TCG_TMP_VEC); + + if (use_pair) { + tcg_out_jxx(s, JCC_JMP, l1, true); + tcg_out_label(s, l2); + } + } + if (use_pair) { + if (use_movbe) { + TCGReg t = datalo; + datalo = datahi; + datahi = t; + } + if (h.base == datalo || h.index == datalo) { + tcg_out_modrm_sib_offset(s, OPC_LEA + P_REXW, datahi, + h.base, h.index, 0, h.ofs); + tcg_out_modrm_offset(s, movop + P_REXW + h.seg, + datalo, datahi, 0); + tcg_out_modrm_offset(s, movop + P_REXW + h.seg, + datahi, datahi, 8); + } else { + tcg_out_modrm_sib_offset(s, movop + P_REXW + h.seg, datalo, + h.base, h.index, 0, h.ofs); + tcg_out_modrm_sib_offset(s, movop + P_REXW + h.seg, datahi, + h.base, h.index, 0, h.ofs + 8); + } + } + if (l1) { + tcg_out_label(s, l1); + } + } + break; + default: g_assert_not_reached(); } @@ -2200,6 +2305,60 @@ static void tcg_out_qemu_st_direct(TCGContext *s, TCGReg datalo, TCGReg datahi, h.base, h.index, 0, h.ofs + 4); } break; + + case MO_128: + { + TCGLabel *l1 = NULL, *l2 = NULL; + bool use_pair = h.aa.atom < MO_128; + + tcg_debug_assert(TCG_TARGET_REG_BITS == 64); + + if (!use_pair) { + tcg_debug_assert(!use_movbe); + /* + * Atomicity requires that we use use VMOVDQA. + * If we've already checked for 16-byte alignment, that's all + * we need. If we arrive here with lesser alignment, then we + * have determined that less that 16-byte alignment can be + * satisfied with two 8-byte loads. + */ + if (h.aa.align < MO_128) { + use_pair = true; + l1 = gen_new_label(); + l2 = gen_new_label(); + + tcg_out_testi(s, h.base, 15); + tcg_out_jxx(s, JCC_JNE, l2, true); + } + + tcg_out_pair_to_vec(s, TCG_TYPE_I64, TCG_TMP_VEC, + datalo, datahi); + tcg_out_vex_modrm_sib_offset(s, OPC_MOVDQA_WxVx + h.seg, + TCG_TMP_VEC, 0, + h.base, h.index, 0, h.ofs); + + if (use_pair) { + tcg_out_jxx(s, JCC_JMP, l1, true); + tcg_out_label(s, l2); + } + } + if (use_pair) { + if (use_movbe) { + TCGReg t = datalo; + datalo = datahi; + datahi = t; + } + tcg_out_modrm_sib_offset(s, movop + P_REXW + h.seg, datalo, + h.base, h.index, 0, h.ofs); + tcg_out_modrm_sib_offset(s, movop + P_REXW + h.seg, datahi, + h.base, h.index, 0, h.ofs + 8); + } + if (l1) { + tcg_out_label(s, l1); + } + } + break; + default: g_assert_not_reached(); } @@ -2523,6 +2682,10 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, tcg_out_qemu_ld(s, a0, a1, a2, args[3], args[4], TCG_TYPE_I64); } break; + case INDEX_op_qemu_ld_i128: + tcg_debug_assert(TCG_TARGET_REG_BITS == 64); + tcg_out_qemu_ld(s, a0, a1, a2, -1, args[3], TCG_TYPE_I128); + break; case INDEX_op_qemu_st_i32: case INDEX_op_qemu_st8_i32: if (TCG_TARGET_REG_BITS >= TARGET_LONG_BITS) { @@ -2540,6 +2703,10 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, tcg_out_qemu_st(s, a0, a1, a2, args[3], args[4], TCG_TYPE_I64); } break; + case INDEX_op_qemu_st_i128: + tcg_debug_assert(TCG_TARGET_REG_BITS == 64); + tcg_out_qemu_st(s, a0, a1, a2, -1, args[3], TCG_TYPE_I128); + break; OP_32_64(mulu2): tcg_out_modrm(s, OPC_GRP3_Ev + rexw, EXT3_MUL, args[3]); @@ -3234,6 +3401,13 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) : TARGET_LONG_BITS <= TCG_TARGET_REG_BITS ? C_O0_I3(L, L, L) : C_O0_I4(L, L, L, L)); + case INDEX_op_qemu_ld_i128: + tcg_debug_assert(TCG_TARGET_REG_BITS == 64); + return C_O2_I1(r, r, L); + case INDEX_op_qemu_st_i128: + tcg_debug_assert(TCG_TARGET_REG_BITS == 64); + return C_O0_I3(L, L, L); + case INDEX_op_brcond2_i32: return C_O0_I4(r, r, ri, ri); @@ -4090,6 +4264,7 @@ static void tcg_target_init(TCGContext *s) s->reserved_regs = 0; tcg_regset_set_reg(s->reserved_regs, TCG_REG_CALL_STACK); + tcg_regset_set_reg(s->reserved_regs, TCG_TMP_VEC); #ifdef _WIN64 /* These are call saved, and we don't save them, so don't use them. */ tcg_regset_set_reg(s->reserved_regs, TCG_REG_XMM6); From patchwork Tue May 16 19:41:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682384 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp18023wrt; Tue, 16 May 2023 12:48:53 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5fygF/IPgTNfkTBJpy7WLVgi8wU8yJAWhznuaZcL54yBarWqb6QqBLDfLvXTynmh8hAcPK X-Received: by 2002:a05:622a:20e:b0:3f3:669f:473a with SMTP id b14-20020a05622a020e00b003f3669f473amr63221344qtx.57.1684266532964; Tue, 16 May 2023 12:48:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266532; cv=none; d=google.com; s=arc-20160816; b=hLt1dRMcnOSOS8JDecYCH9/KKS8zzkPJhkcV5Cn1Ddoxby4N9M7GwtwdKsSA9fq4wW XUwZxuWlmN6WbEpkNUrVvaZZKK8lK2p2zHehRgPpZ389XLMqYXQtBHVFJ4sJMpDEtuAE ccEf5yqJbcExdU1+TlYg9z+AxPfh+Zb4VETsv6Sn24ALIFAuf9PYXHjWRWDgn4UtKKhU JBqwHjj8EkMz39bE0bOxRpTVCryLHxnBkp33AtfKB6WnCZAjhhDq6CYjo+yyAYp6CRw5 3Y3SmX36oIze0bx6iRecYEocNAHMgCKB5eC9sd6vm1jY1HJ5dz8tJTag1/X+Fetyae8B fsXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=s0UJEF5x7D/tJuynxyknIi5yS26S+uTp0Nck/e1vPAw=; b=qTMzF/Jhf47oE2N9Mz/xnFqTag1ZhK7+lfqvTD9Hr7cQQaOrpaYALrVqeKMbx6AQMZ M6ei6T3ZvZHgeM0EjxAT8fAL2hhWPeHdwFzRW25srg19Spz/fP6dJUwZS7FkAYZHPuI0 kCaLgxPOUpa8ENtK8fsIi5oYJb3Mc4JQJz3U0+hTTqfEjnVp5M2e95TJFHDcgCD/F2Sl BcVHM54i9iQ7uGvPlTLGNWD/W8URJTa7kIJxRrNxx2MZzKA6/pchVEiHwuX4xSQGkrPJ z5Tsg+7FRmkk4NrFAg324kXjy4HO7RtWaJqMb1B2+dbmle2aLkbaNPKAB6IP4AAOKMkd 7LDg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cqU6Y6Ml; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f40-20020a05622a1a2800b003f211db9651si11888624qtb.171.2023.05.16.12.48.52 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:48:52 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cqU6Y6Ml; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0cY-0000pN-Dh; Tue, 16 May 2023 15:46:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0c5-000075-0T for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:40 -0400 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0c0-0003iw-QK for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:34 -0400 Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-1ae4be0b1f3so542865ad.0 for ; Tue, 16 May 2023 12:45:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266331; x=1686858331; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=s0UJEF5x7D/tJuynxyknIi5yS26S+uTp0Nck/e1vPAw=; b=cqU6Y6Mlxf981nmwHHerhSrJHJiQS4Cn9vg31bfxFadBRz9f8YE+cL+IfrHAZIyHBl KRkW/g8GB2JTtB5bR0c5dFUUl2Ld4CvRECTEPTWg4Ipt/qlqjCrnvXINCgZjUAi+bW1p kfCCChsTCLgd6076DMGgLmUk7hV55de5xfhAnQB8KfKs7GCTjPByoaGCE2er/se4VVN4 NKc8b3nRimEa7qz42WfqMYKBvdmSrOoRDlDfvkgZUjjCMOEdpXmreXTFwSGsQ4QPKbYV lg5IOFWuXXhsmgMmRujYqLK8rDMUgRxXIGSXHc+9aTsQMqYeHFf7NI1q94TPaQ8AiYbn CnGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266331; x=1686858331; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=s0UJEF5x7D/tJuynxyknIi5yS26S+uTp0Nck/e1vPAw=; b=KCkTgkfnk3TxJm7/Fo/QIBidNMnxZO6CnV2+CaThk6JMX/xOjK2xpdp2Vq0XcXmLDQ U42EcnpvaXhgN/TdDcFOnqPNOCRFKklOvBkIqryO5XWbDSaYKXGta4etNAjyZEE9Yb9a YmqYJAEg63wx1Vdsa93jtKI50IBarsoPyhAJaaSovBucSwRwIMbz2dhwTTbQpsCGeU/3 vk0y3lEiiHqgldl5XIbi1UTuJpFkd4+wLg5uLg0IXQb+ge7dVYt/9XDyxFwPMNmQ5jWg LH1mxDJxNpXMa24ivPWybV0DFxd8TuWKwUFGLIaWtgJTZaABO606rLQbaL5nMuGP4MEU X/3A== X-Gm-Message-State: AC+VfDxBGkuWBjxXF1/NtZN9ull/Ur3QDDTJ3Iy9uht2NVi4IVeG15hi 6y2VjfPd9TRU8g8NU7NU5M3Ir1E2yK5tCQ5sY7g= X-Received: by 2002:a17:90b:fd8:b0:24e:4231:ec6b with SMTP id gd24-20020a17090b0fd800b0024e4231ec6bmr37281620pjb.21.1684266331365; Tue, 16 May 2023 12:45:31 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:30 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 49/80] tcg/aarch64: Rename temporaries Date: Tue, 16 May 2023 12:41:14 -0700 Message-Id: <20230516194145.1749305-50-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62d; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org We will need to allocate a second general-purpose temporary. Rename the existing temps to add a distinguishing number. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.c.inc | 50 ++++++++++++++++++------------------ 1 file changed, 25 insertions(+), 25 deletions(-) diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index ea4108d59c..1ed5be2c00 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -80,8 +80,8 @@ static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) bool have_lse; bool have_lse2; -#define TCG_REG_TMP TCG_REG_X30 -#define TCG_VEC_TMP TCG_REG_V31 +#define TCG_REG_TMP0 TCG_REG_X30 +#define TCG_VEC_TMP0 TCG_REG_V31 #ifndef CONFIG_SOFTMMU /* Note that XZR cannot be encoded in the address base register slot, @@ -998,7 +998,7 @@ static bool tcg_out_dup_vec(TCGContext *s, TCGType type, unsigned vece, static bool tcg_out_dupm_vec(TCGContext *s, TCGType type, unsigned vece, TCGReg r, TCGReg base, intptr_t offset) { - TCGReg temp = TCG_REG_TMP; + TCGReg temp = TCG_REG_TMP0; if (offset < -0xffffff || offset > 0xffffff) { tcg_out_movi(s, TCG_TYPE_PTR, temp, offset); @@ -1150,8 +1150,8 @@ static void tcg_out_ldst(TCGContext *s, AArch64Insn insn, TCGReg rd, } /* Worst-case scenario, move offset to temp register, use reg offset. */ - tcg_out_movi(s, TCG_TYPE_I64, TCG_REG_TMP, offset); - tcg_out_ldst_r(s, insn, rd, rn, TCG_TYPE_I64, TCG_REG_TMP); + tcg_out_movi(s, TCG_TYPE_I64, TCG_REG_TMP0, offset); + tcg_out_ldst_r(s, insn, rd, rn, TCG_TYPE_I64, TCG_REG_TMP0); } static bool tcg_out_mov(TCGContext *s, TCGType type, TCGReg ret, TCGReg arg) @@ -1367,8 +1367,8 @@ static void tcg_out_call_int(TCGContext *s, const tcg_insn_unit *target) if (offset == sextract64(offset, 0, 26)) { tcg_out_insn(s, 3206, BL, offset); } else { - tcg_out_movi(s, TCG_TYPE_I64, TCG_REG_TMP, (intptr_t)target); - tcg_out_insn(s, 3207, BLR, TCG_REG_TMP); + tcg_out_movi(s, TCG_TYPE_I64, TCG_REG_TMP0, (intptr_t)target); + tcg_out_insn(s, 3207, BLR, TCG_REG_TMP0); } } @@ -1505,7 +1505,7 @@ static void tcg_out_addsub2(TCGContext *s, TCGType ext, TCGReg rl, AArch64Insn insn; if (rl == ah || (!const_bh && rl == bh)) { - rl = TCG_REG_TMP; + rl = TCG_REG_TMP0; } if (const_bl) { @@ -1522,7 +1522,7 @@ static void tcg_out_addsub2(TCGContext *s, TCGType ext, TCGReg rl, possibility of adding 0+const in the low part, and the immediate add instructions encode XSP not XZR. Don't try anything more elaborate here than loading another zero. */ - al = TCG_REG_TMP; + al = TCG_REG_TMP0; tcg_out_movi(s, ext, al, 0); } tcg_out_insn_3401(s, insn, ext, rl, al, bl); @@ -1563,7 +1563,7 @@ static void tcg_out_cltz(TCGContext *s, TCGType ext, TCGReg d, { TCGReg a1 = a0; if (is_ctz) { - a1 = TCG_REG_TMP; + a1 = TCG_REG_TMP0; tcg_out_insn(s, 3507, RBIT, ext, a1, a0); } if (const_b && b == (ext ? 64 : 32)) { @@ -1572,7 +1572,7 @@ static void tcg_out_cltz(TCGContext *s, TCGType ext, TCGReg d, AArch64Insn sel = I3506_CSEL; tcg_out_cmp(s, ext, a0, 0, 1); - tcg_out_insn(s, 3507, CLZ, ext, TCG_REG_TMP, a1); + tcg_out_insn(s, 3507, CLZ, ext, TCG_REG_TMP0, a1); if (const_b) { if (b == -1) { @@ -1585,7 +1585,7 @@ static void tcg_out_cltz(TCGContext *s, TCGType ext, TCGReg d, b = d; } } - tcg_out_insn_3506(s, sel, ext, d, TCG_REG_TMP, b, TCG_COND_NE); + tcg_out_insn_3506(s, sel, ext, d, TCG_REG_TMP0, b, TCG_COND_NE); } } @@ -1602,7 +1602,7 @@ bool tcg_target_has_memory_bswap(MemOp memop) } static const TCGLdstHelperParam ldst_helper_param = { - .ntmp = 1, .tmp = { TCG_REG_TMP } + .ntmp = 1, .tmp = { TCG_REG_TMP0 } }; static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) @@ -1862,7 +1862,7 @@ static void tcg_out_goto_tb(TCGContext *s, int which) set_jmp_insn_offset(s, which); tcg_out32(s, I3206_B); - tcg_out_insn(s, 3207, BR, TCG_REG_TMP); + tcg_out_insn(s, 3207, BR, TCG_REG_TMP0); set_jmp_reset_offset(s, which); } @@ -1881,7 +1881,7 @@ void tb_target_set_jmp_target(const TranslationBlock *tb, int n, ptrdiff_t i_offset = i_addr - jmp_rx; /* Note that we asserted this in range in tcg_out_goto_tb. */ - insn = deposit32(I3305_LDR | TCG_REG_TMP, 5, 19, i_offset >> 2); + insn = deposit32(I3305_LDR | TCG_REG_TMP0, 5, 19, i_offset >> 2); } qatomic_set((uint32_t *)jmp_rw, insn); flush_idcache_range(jmp_rx, jmp_rw, 4); @@ -2075,13 +2075,13 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_rem_i64: case INDEX_op_rem_i32: - tcg_out_insn(s, 3508, SDIV, ext, TCG_REG_TMP, a1, a2); - tcg_out_insn(s, 3509, MSUB, ext, a0, TCG_REG_TMP, a2, a1); + tcg_out_insn(s, 3508, SDIV, ext, TCG_REG_TMP0, a1, a2); + tcg_out_insn(s, 3509, MSUB, ext, a0, TCG_REG_TMP0, a2, a1); break; case INDEX_op_remu_i64: case INDEX_op_remu_i32: - tcg_out_insn(s, 3508, UDIV, ext, TCG_REG_TMP, a1, a2); - tcg_out_insn(s, 3509, MSUB, ext, a0, TCG_REG_TMP, a2, a1); + tcg_out_insn(s, 3508, UDIV, ext, TCG_REG_TMP0, a1, a2); + tcg_out_insn(s, 3509, MSUB, ext, a0, TCG_REG_TMP0, a2, a1); break; case INDEX_op_shl_i64: @@ -2125,8 +2125,8 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, if (c2) { tcg_out_rotl(s, ext, a0, a1, a2); } else { - tcg_out_insn(s, 3502, SUB, 0, TCG_REG_TMP, TCG_REG_XZR, a2); - tcg_out_insn(s, 3508, RORV, ext, a0, a1, TCG_REG_TMP); + tcg_out_insn(s, 3502, SUB, 0, TCG_REG_TMP0, TCG_REG_XZR, a2); + tcg_out_insn(s, 3508, RORV, ext, a0, a1, TCG_REG_TMP0); } break; @@ -2528,8 +2528,8 @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc, break; } } - tcg_out_dupi_vec(s, type, MO_8, TCG_VEC_TMP, 0); - a2 = TCG_VEC_TMP; + tcg_out_dupi_vec(s, type, MO_8, TCG_VEC_TMP0, 0); + a2 = TCG_VEC_TMP0; } if (is_scalar) { insn = cmp_scalar_insn[cond]; @@ -2938,9 +2938,9 @@ static void tcg_target_init(TCGContext *s) s->reserved_regs = 0; tcg_regset_set_reg(s->reserved_regs, TCG_REG_SP); tcg_regset_set_reg(s->reserved_regs, TCG_REG_FP); - tcg_regset_set_reg(s->reserved_regs, TCG_REG_TMP); tcg_regset_set_reg(s->reserved_regs, TCG_REG_X18); /* platform register */ - tcg_regset_set_reg(s->reserved_regs, TCG_VEC_TMP); + tcg_regset_set_reg(s->reserved_regs, TCG_REG_TMP0); + tcg_regset_set_reg(s->reserved_regs, TCG_VEC_TMP0); } /* Saving pairs: (X19, X20) .. (X27, X28), (X29(fp), X30(lr)). */ From patchwork Tue May 16 19:41:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682429 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp21413wrt; Tue, 16 May 2023 13:00:21 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4d1v+O1W3oDawsGQuNVfZ+7fwoqM5sNhq7EPlHRvegaixNmMPJHQV+FuD5zHM4tTmrVT70 X-Received: by 2002:a7b:cc05:0:b0:3f1:7581:eaaf with SMTP id f5-20020a7bcc05000000b003f17581eaafmr27065370wmh.4.1684267221766; Tue, 16 May 2023 13:00:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267221; cv=none; d=google.com; s=arc-20160816; b=hI8awRVYuKDqF9aLpyuB2vCeXkvobW9uAr/JtZvEjJEpVa7FjU1hxewBoIO++T8ZRF B4a15m9EqxqirI0qWjNW/r8QTRW9fOS+He3ngH+yMgDp0WZ05onP6ii7Qutu9vKiiqRj J0Bp4qn32p/DUuTEA215efyDvTRMtdEzwHRk0BehXp4PnffPGYOrMExSbDWTmavJ+mi2 k+Z5r1UZyoscUvGTBpBWo0cid6zXumADz6zKdAIrwQ627hCxzVPeBeUGiKDtjL1kxYKh tSy91OmMkXGU/4si6HTH6aDb3VfaWLyUwa65/B5w0+Q8XZLkb/y8Hd5X2KhGfLO2vAlm fcHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=S6zv6CFBl/KpfAciWHv+fKXujm15Kfs396M2WZzhU80=; b=otMcu/nTcC54rY6MBcQWuQeWAtZDR8r8AwDl6oj1cTehAFvJa8TcSRYW6ChtxzLy6s jvw6LaZYUUMN1yfJ++VA+PRi3vG+AllZyZX8K3FPojkMpAE5p5gEMz1xvFTIRB1PCutI iOuzEsfkUwM6C4FAXGnbQapctvlmLI+bxFGZAvBmSqhrszhjKg4sWVSiuboHUQlhL4K5 Lcu5CtydGlrLVtmVPircWIvUdR9BsSZjb0xaqjKeknVwhgQkMIiVoW23Mzvx3dGbw/+I tEt/GLQqKyAxCoOlOAK7Q8IEzwiBgAVabSm9QPWsN1mFb94W38owjVk7dufK6p0VEPZ/ hvlg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=icy1VDCW; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f14-20020adff8ce000000b002fd8a643604si128949wrq.1079.2023.05.16.13.00.21 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 13:00:21 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=icy1VDCW; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0eo-00035P-6p; Tue, 16 May 2023 15:48:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0c7-0000Ba-9O for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:44 -0400 Received: from mail-pj1-x1030.google.com ([2607:f8b0:4864:20::1030]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0c1-0003jH-Ln for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:35 -0400 Received: by mail-pj1-x1030.google.com with SMTP id 98e67ed59e1d1-24e3a0aa408so58595a91.1 for ; Tue, 16 May 2023 12:45:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266332; x=1686858332; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=S6zv6CFBl/KpfAciWHv+fKXujm15Kfs396M2WZzhU80=; b=icy1VDCWgA3RxUGOdwxBJHDIhqtfvcW1ZHPQEKHvet8RDKxEMf4MaaTVoAfXLrSBQq Np8FY5eBl3Ak1iJ+uNG8yC/SBxj7BS4Mgxi8rEepuD/LKN4YtJM6qbUvxMp3qiUbES0G g1c02x3dzBo9gqoS2D1S2kthVrrboVOvDnwEqch93TDqxA49LPt7dDmA6j9NV18JIcLw T8LtAcsviO0+3m76h+U5K43hhIfzXb/zKOgIlw8plwCBD4Ic4SSPF+8dT7Wa0dOn1IRl pZXfZYpJVIsyLx2aDQku0PP97wl5wbzT7DPenGoX08FmOGQbyFvIL7lNaQ7HHbPj2FvJ HSIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266332; x=1686858332; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S6zv6CFBl/KpfAciWHv+fKXujm15Kfs396M2WZzhU80=; b=YnYLhCnAsOD/DZV+63gkLWANm3x4s2bp/VBbFk2gnmyOd9Ha1z/m0YxmD7/7YM+oCS tCoWMH7W3h1qR1YwXC2g8KG5+lW353USnxA8z2oP2QlUS/OIH+sdnAdJJRhKtEJ42oMN cbynr3wPeVHp74Xo65iV3ddj7k1LRHlFjQ+VyeH+JTrlBt+CKVIp320W+xdUh8aSCdkO 4RKV4zZdgZ5rb/t6myLKyAAYOQDP9VCtAElrNL+jLvngvaf20Nfkod8s4Qo/L97AF1eU vIBXlrftTwCOKK0AF2fYq89uyfbUmQdWX0RqinFFAPdvJEnbMD66bpw2q/76EJAzFCDf P1ow== X-Gm-Message-State: AC+VfDxotP9oD8sL9YQzMQ9+FLYmtFvrV4f5YufJcNz7btWxloJ+szVU p6Ev0Y1sB4fVdkW1EzZgkQaGiS3nhTF/CFrYa4E= X-Received: by 2002:a17:90a:65c6:b0:24e:4b1c:74d2 with SMTP id i6-20020a17090a65c600b0024e4b1c74d2mr38679667pjs.32.1684266332206; Tue, 16 May 2023 12:45:32 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:31 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 50/80] tcg/aarch64: Support 128-bit load/store Date: Tue, 16 May 2023 12:41:15 -0700 Message-Id: <20230516194145.1749305-51-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1030; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1030.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use LDXP+STXP when LSE2 is not present and 16-byte atomicity is required, and LDP/STP otherwise. This requires allocating a second general-purpose temporary, as Rs cannot overlap Rn in STXP. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target-con-set.h | 2 + tcg/aarch64/tcg-target.h | 11 +- tcg/aarch64/tcg-target.c.inc | 179 ++++++++++++++++++++++++++++++- 3 files changed, 189 insertions(+), 3 deletions(-) diff --git a/tcg/aarch64/tcg-target-con-set.h b/tcg/aarch64/tcg-target-con-set.h index d6c6866878..74065c7098 100644 --- a/tcg/aarch64/tcg-target-con-set.h +++ b/tcg/aarch64/tcg-target-con-set.h @@ -14,6 +14,7 @@ C_O0_I2(lZ, l) C_O0_I2(r, rA) C_O0_I2(rZ, r) C_O0_I2(w, r) +C_O0_I3(lZ, lZ, l) C_O1_I1(r, l) C_O1_I1(r, r) C_O1_I1(w, r) @@ -33,4 +34,5 @@ C_O1_I2(w, w, wO) C_O1_I2(w, w, wZ) C_O1_I3(w, w, w, w) C_O1_I4(r, r, rA, rZ, rZ) +C_O2_I1(r, r, l) C_O2_I4(r, r, rZ, rZ, rA, rMZ) diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index 74ee2ed255..2c079f21c2 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -129,7 +129,16 @@ extern bool have_lse2; #define TCG_TARGET_HAS_muluh_i64 1 #define TCG_TARGET_HAS_mulsh_i64 1 -#define TCG_TARGET_HAS_qemu_ldst_i128 0 +/* + * Without FEAT_LSE2, we must use LDXP+STXP to implement atomic 128-bit load, + * which requires writable pages. We must defer to the helper for user-only, + * but in system mode all ram is writable for the host. + */ +#ifdef CONFIG_USER_ONLY +#define TCG_TARGET_HAS_qemu_ldst_i128 have_lse2 +#else +#define TCG_TARGET_HAS_qemu_ldst_i128 1 +#endif #define TCG_TARGET_HAS_v64 1 #define TCG_TARGET_HAS_v128 1 diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index 1ed5be2c00..893b3514bb 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -81,6 +81,7 @@ bool have_lse; bool have_lse2; #define TCG_REG_TMP0 TCG_REG_X30 +#define TCG_REG_TMP1 TCG_REG_X17 #define TCG_VEC_TMP0 TCG_REG_V31 #ifndef CONFIG_SOFTMMU @@ -404,6 +405,10 @@ typedef enum { I3305_LDR_v64 = 0x5c000000, I3305_LDR_v128 = 0x9c000000, + /* Load/store exclusive. */ + I3306_LDXP = 0xc8600000, + I3306_STXP = 0xc8200000, + /* Load/store register. Described here as 3.3.12, but the helper that emits them can transform to 3.3.10 or 3.3.13. */ I3312_STRB = 0x38000000 | LDST_ST << 22 | MO_8 << 30, @@ -468,6 +473,9 @@ typedef enum { I3406_ADR = 0x10000000, I3406_ADRP = 0x90000000, + /* Add/subtract extended register instructions. */ + I3501_ADD = 0x0b200000, + /* Add/subtract shifted register instructions (without a shift). */ I3502_ADD = 0x0b000000, I3502_ADDS = 0x2b000000, @@ -638,6 +646,12 @@ static void tcg_out_insn_3305(TCGContext *s, AArch64Insn insn, tcg_out32(s, insn | (imm19 & 0x7ffff) << 5 | rt); } +static void tcg_out_insn_3306(TCGContext *s, AArch64Insn insn, TCGReg rs, + TCGReg rt, TCGReg rt2, TCGReg rn) +{ + tcg_out32(s, insn | rs << 16 | rt2 << 10 | rn << 5 | rt); +} + static void tcg_out_insn_3201(TCGContext *s, AArch64Insn insn, TCGType ext, TCGReg rt, int imm19) { @@ -720,6 +734,14 @@ static void tcg_out_insn_3406(TCGContext *s, AArch64Insn insn, tcg_out32(s, insn | (disp & 3) << 29 | (disp & 0x1ffffc) << (5 - 2) | rd); } +static inline void tcg_out_insn_3501(TCGContext *s, AArch64Insn insn, + TCGType sf, TCGReg rd, TCGReg rn, + TCGReg rm, int opt, int imm3) +{ + tcg_out32(s, insn | sf << 31 | rm << 16 | opt << 13 | + imm3 << 10 | rn << 5 | rd); +} + /* This function is for both 3.5.2 (Add/Subtract shifted register), for the rare occasion when we actually want to supply a shift amount. */ static inline void tcg_out_insn_3502S(TCGContext *s, AArch64Insn insn, @@ -1647,16 +1669,16 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, TCGType addr_type = TARGET_LONG_BITS == 64 ? TCG_TYPE_I64 : TCG_TYPE_I32; TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); + MemOp s_bits = opc & MO_SIZE; unsigned a_mask; h->aa = atom_and_align_for_opc(s, opc, have_lse2 ? MO_ATOM_WITHIN16 : MO_ATOM_IFALIGN, - false); + s_bits == MO_128); a_mask = (1 << h->aa.align) - 1; #ifdef CONFIG_SOFTMMU - unsigned s_bits = opc & MO_SIZE; unsigned s_mask = (1u << s_bits) - 1; unsigned mem_index = get_mmuidx(oi); TCGReg x3; @@ -1837,6 +1859,148 @@ static void tcg_out_qemu_st(TCGContext *s, TCGReg data_reg, TCGReg addr_reg, } } +static TCGLabelQemuLdst * +prepare_host_addr_base_only(TCGContext *s, HostAddress *h, TCGReg addr_reg, + MemOpIdx oi, bool is_ld) +{ + TCGLabelQemuLdst *ldst; + + ldst = prepare_host_addr(s, h, addr_reg, oi, true); + + /* Compose the final address, as LDP/STP have no indexing. */ + if (h->index != TCG_REG_XZR) { + tcg_out_insn(s, 3501, ADD, TCG_TYPE_I64, TCG_REG_TMP0, + h->base, h->index, + h->index_ext == TCG_TYPE_I32 ? MO_32 : MO_64, 0); + h->base = TCG_REG_TMP0; + h->index = TCG_REG_XZR; + h->index_ext = TCG_TYPE_I64; + } + + return ldst; +} + +static void tcg_out_qemu_ld128(TCGContext *s, TCGReg datalo, TCGReg datahi, + TCGReg addr_reg, MemOpIdx oi) +{ + TCGLabelQemuLdst *ldst; + HostAddress h; + + ldst = prepare_host_addr_base_only(s, &h, addr_reg, oi, true); + + if (h.aa.atom < MO_128 || have_lse2) { + tcg_out_insn(s, 3314, LDP, datalo, datahi, h.base, 0, 0, 0); + } else { + TCGLabel *l0, *l1 = NULL; + + /* + * 16-byte atomicity without LSE2 requires LDXP+STXP loop: + * 1: ldxp lo,hi,[addr] + * stxp tmp1,lo,hi,[addr] + * cbnz tmp1, 1b + * + * If we have already checked for 16-byte alignment, that's all + * we need. Otherwise we have determined that misaligned atomicity + * may be handled with two 8-byte loads. + */ + if (h.aa.align < MO_128) { + /* + * TODO: align should be MO_64, so we only need test bit 3, + * which means we could use TBNZ instead of AND+CBNE. + */ + l1 = gen_new_label(); + tcg_out_logicali(s, I3404_ANDI, 0, TCG_REG_TMP1, addr_reg, 15); + tcg_out_brcond(s, TCG_TYPE_I32, TCG_COND_NE, + TCG_REG_TMP1, 0, 1, l1); + } + + l0 = gen_new_label(); + tcg_out_label(s, l0); + + tcg_out_insn(s, 3306, LDXP, TCG_REG_XZR, datalo, datahi, h.base); + tcg_out_insn(s, 3306, STXP, TCG_REG_TMP1, datalo, datahi, h.base); + tcg_out_brcond(s, TCG_TYPE_I32, TCG_COND_NE, TCG_REG_TMP1, 0, 1, l0); + + if (l1) { + TCGLabel *l2 = gen_new_label(); + tcg_out_goto_label(s, l2); + + tcg_out_label(s, l1); + tcg_out_insn(s, 3314, LDP, datalo, datahi, h.base, 0, 0, 0); + + tcg_out_label(s, l2); + } + } + + if (ldst) { + ldst->type = TCG_TYPE_I128; + ldst->datalo_reg = datalo; + ldst->datahi_reg = datahi; + ldst->raddr = tcg_splitwx_to_rx(s->code_ptr); + } +} + +static void tcg_out_qemu_st128(TCGContext *s, TCGReg datalo, TCGReg datahi, + TCGReg addr_reg, MemOpIdx oi) +{ + TCGLabelQemuLdst *ldst; + HostAddress h; + + ldst = prepare_host_addr_base_only(s, &h, addr_reg, oi, false); + + if (h.aa.atom < MO_128 || have_lse2) { + tcg_out_insn(s, 3314, STP, datalo, datahi, h.base, 0, 0, 0); + } else { + TCGLabel *l0, *l1 = NULL; + + /* + * 16-byte atomicity without LSE2 requires LDXP+STXP loop: + * 1: ldxp xzr,tmp1,[addr] + * stxp tmp1,lo,hi,[addr] + * cbnz tmp1, 1b + * + * If we have already checked for 16-byte alignment, that's all + * we need. Otherwise we have determined that misaligned atomicity + * may be handled with two 8-byte stores. + */ + if (h.aa.align < MO_128) { + /* + * TODO: align should be MO_64, so we only need test bit 3, + * which means we could use TBNZ instead of AND+CBNE. + */ + l1 = gen_new_label(); + tcg_out_logicali(s, I3404_ANDI, 0, TCG_REG_TMP1, addr_reg, 15); + tcg_out_brcond(s, TCG_TYPE_I32, TCG_COND_NE, + TCG_REG_TMP1, 0, 1, l1); + } + + l0 = gen_new_label(); + tcg_out_label(s, l0); + + tcg_out_insn(s, 3306, LDXP, TCG_REG_XZR, + TCG_REG_XZR, TCG_REG_TMP1, h.base); + tcg_out_insn(s, 3306, STXP, TCG_REG_TMP1, datalo, datahi, h.base); + tcg_out_brcond(s, TCG_TYPE_I32, TCG_COND_NE, TCG_REG_TMP1, 0, 1, l0); + + if (l1) { + TCGLabel *l2 = gen_new_label(); + tcg_out_goto_label(s, l2); + + tcg_out_label(s, l1); + tcg_out_insn(s, 3314, STP, datalo, datahi, h.base, 0, 0, 0); + + tcg_out_label(s, l2); + } + } + + if (ldst) { + ldst->type = TCG_TYPE_I128; + ldst->datalo_reg = datalo; + ldst->datahi_reg = datahi; + ldst->raddr = tcg_splitwx_to_rx(s->code_ptr); + } +} + static const tcg_insn_unit *tb_ret_addr; static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) @@ -2172,6 +2336,12 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_qemu_st_i64: tcg_out_qemu_st(s, REG0(0), a1, a2, ext); break; + case INDEX_op_qemu_ld_i128: + tcg_out_qemu_ld128(s, a0, a1, a2, args[3]); + break; + case INDEX_op_qemu_st_i128: + tcg_out_qemu_st128(s, REG0(0), REG0(1), a2, args[3]); + break; case INDEX_op_bswap64_i64: tcg_out_rev(s, TCG_TYPE_I64, MO_64, a0, a1); @@ -2809,9 +2979,13 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_qemu_ld_i32: case INDEX_op_qemu_ld_i64: return C_O1_I1(r, l); + case INDEX_op_qemu_ld_i128: + return C_O2_I1(r, r, l); case INDEX_op_qemu_st_i32: case INDEX_op_qemu_st_i64: return C_O0_I2(lZ, l); + case INDEX_op_qemu_st_i128: + return C_O0_I3(lZ, lZ, l); case INDEX_op_deposit_i32: case INDEX_op_deposit_i64: @@ -2940,6 +3114,7 @@ static void tcg_target_init(TCGContext *s) tcg_regset_set_reg(s->reserved_regs, TCG_REG_FP); tcg_regset_set_reg(s->reserved_regs, TCG_REG_X18); /* platform register */ tcg_regset_set_reg(s->reserved_regs, TCG_REG_TMP0); + tcg_regset_set_reg(s->reserved_regs, TCG_REG_TMP1); tcg_regset_set_reg(s->reserved_regs, TCG_VEC_TMP0); } From patchwork Tue May 16 19:41:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682416 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp20972wrt; Tue, 16 May 2023 12:59:10 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7WDtsiS5qUJW3AlyJ9XVkdQW8QHSK3NKH+Ws7BL0hWVjmnTttQm7442mpP1k7y4rh5vMXC X-Received: by 2002:a05:6214:194b:b0:621:41c:75a7 with SMTP id q11-20020a056214194b00b00621041c75a7mr65617078qvk.30.1684267036425; Tue, 16 May 2023 12:57:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267036; cv=none; d=google.com; s=arc-20160816; b=voT9rsAKmdmyPxV+GF5RLYod+dcklEOV7zzTBOnA2Ozc64/caq8+xq1IhuBZhTkxud mVQPknpiCfyxhQ58nvlc9ievWgg5IS66EVMMCfgb7BvIT8vEUJ4jdYK/Etc/LMOrYqc1 0ubP+JL2AMEjq7vS9TVUuwTbZGhaukeRp8v7jQs8tIkjGOjXio2R6OaCQHbPIoA6HkU2 MGRysomkyuII6ycNyLiDPZlT2gY2m9QdjdCY6HJj0dK8EWtQvYJlfeLfIh9wxiL+DOms pCm/tYl3BGv2+Bos5zJ8xDhvAeFyXKw62vdDvyzOlMbITw0zwqAVyGkXVG/We87npjuN hL4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=a3C62E7I5KSH6hopm1N3+kJjnyiF8W6DIOatywqt9qE=; b=uWZ/ioUqqgEMMnWA8BG08aj3JaS9I3dslRgaBcRYw+uS6RsiA08f8S2GuXIKdf0kaH fA+4jeuwb7lH9/9kWqBExX8kJZXYff9lYARIpuUZiWq3+qLAD+wK929/eX2oToOtuuF8 4AAGHzsQIWaLyLiekENmbjVaFCgHiM3h4Dwv+6pkBH97S7kcWMHZjmPPmZT3N3e0c2Pk 7rbJdQ0QER5sCdRsVD9NgRdQjJlvktEkwmibeWpldIVW31CYIkDJFhA/3xCcQJgvY/Zt vyDCfKSxNMzfSuFzD9DfMVuRhrH8fHWtxQnV19n5A4rAWMzHMKQMiddv1w3eoiOqvTgG PCWg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hioE5TVG; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id dv7-20020ad44ee7000000b0056f2ba3d7c6si12206104qvb.33.2023.05.16.12.57.16 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:57:16 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hioE5TVG; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0eM-0002mO-Kn; Tue, 16 May 2023 15:47:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0c8-0000E1-UF for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:46 -0400 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0c2-0003jU-HQ for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:36 -0400 Received: by mail-pj1-x102a.google.com with SMTP id 98e67ed59e1d1-24e015fcf3dso40506a91.3 for ; Tue, 16 May 2023 12:45:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266333; x=1686858333; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=a3C62E7I5KSH6hopm1N3+kJjnyiF8W6DIOatywqt9qE=; b=hioE5TVG9YT7/E1qGMhj6V+n9ItFr8Wdaj6sJeeOOmtTD1lpk94s6RgXnI14FBWdnM qUEWtVMo27iIMfBehhzDG0kuHjeGupfX3pfaLIYplWd7U28TBhj9yOZdddZ89NlVSPWk 2jxd2wjPFlXXSTyHDJX4TkZdoz0YS8RPHt+5t4NcnhRVGvNkMg3RW21/h/03RbqYfIDZ GzbzbTUFnJE2GKggay6yYz6nNu+zC9svzCPtiT1S5XiV68pos/0mfmk/rTaD5IB+dMqo LHZC6Z9fsVrIwvNq2QAzTVUvZFTUTt6UkCVvw+d4YB/P/cjSksUJG5ODjJBZlbytCZqV pm4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266333; x=1686858333; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=a3C62E7I5KSH6hopm1N3+kJjnyiF8W6DIOatywqt9qE=; b=FOSCqSY9wLrT2BxiReeMTJ1ok9T9vVe5MYzedXTngYNnF1mdatTj6EHsisy5awZOIR eAdldO69Xl8Hl47IZqzVbSjFifjgURPWyu6beF3/hdkUtAM3d44es5pf4o/fYyH54cUI /i/0z/f9tWReOdHOsaMQcm3oqcQdyfNmSP+POE7WdQOvUBtBcUfWJK7pR4oPM04ezBNC ysYhzS42UIAxjfJZpE/MEEnmyM3c7mWgK8nbCEzMQqCkj7AJZUmhdXBK0QZXIOFLMbux WKK1b/41sl+R7vz7S9P33/6pOejvQhxXTAaLtj61JBZnnAXGPkpn54fHjzmroxVu4Yiy xqHw== X-Gm-Message-State: AC+VfDzQL+1UDqV3SFqKT16P0aUdnhs9g8TRHdFsaOPDp12QQ/PZoYYa QDGyOx6EA8kpxfmkUdU7eTBXo962yxhNZTu9z5E= X-Received: by 2002:a17:90a:1141:b0:250:43a8:6551 with SMTP id d1-20020a17090a114100b0025043a86551mr36373020pje.35.1684266333046; Tue, 16 May 2023 12:45:33 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:32 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Daniel Henrique Barboza Subject: [PULL 51/80] tcg/ppc: Support 128-bit load/store Date: Tue, 16 May 2023 12:41:16 -0700 Message-Id: <20230516194145.1749305-52-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102a; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use LQ/STQ with ISA v2.07, and 16-byte atomicity is required. Note that these instructions do not require 16-byte alignment. Reviewed-by: Daniel Henrique Barboza Signed-off-by: Richard Henderson --- tcg/ppc/tcg-target-con-set.h | 2 + tcg/ppc/tcg-target-con-str.h | 1 + tcg/ppc/tcg-target.h | 3 +- tcg/ppc/tcg-target.c.inc | 115 +++++++++++++++++++++++++++++++---- 4 files changed, 108 insertions(+), 13 deletions(-) diff --git a/tcg/ppc/tcg-target-con-set.h b/tcg/ppc/tcg-target-con-set.h index f206b29205..bbd7b21247 100644 --- a/tcg/ppc/tcg-target-con-set.h +++ b/tcg/ppc/tcg-target-con-set.h @@ -14,6 +14,7 @@ C_O0_I2(r, r) C_O0_I2(r, ri) C_O0_I2(v, r) C_O0_I3(r, r, r) +C_O0_I3(o, m, r) C_O0_I4(r, r, ri, ri) C_O0_I4(r, r, r, r) C_O1_I1(r, r) @@ -34,6 +35,7 @@ C_O1_I3(v, v, v, v) C_O1_I4(r, r, ri, rZ, rZ) C_O1_I4(r, r, r, ri, ri) C_O2_I1(r, r, r) +C_O2_I1(o, m, r) C_O2_I2(r, r, r, r) C_O2_I4(r, r, rI, rZM, r, r) C_O2_I4(r, r, r, r, rI, rZM) diff --git a/tcg/ppc/tcg-target-con-str.h b/tcg/ppc/tcg-target-con-str.h index 094613cbcb..20846901de 100644 --- a/tcg/ppc/tcg-target-con-str.h +++ b/tcg/ppc/tcg-target-con-str.h @@ -9,6 +9,7 @@ * REGS(letter, register_mask) */ REGS('r', ALL_GENERAL_REGS) +REGS('o', ALL_GENERAL_REGS & 0xAAAAAAAAu) /* odd registers */ REGS('v', ALL_VECTOR_REGS) /* diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h index 0914380bd7..204b70f86a 100644 --- a/tcg/ppc/tcg-target.h +++ b/tcg/ppc/tcg-target.h @@ -149,7 +149,8 @@ extern bool have_vsx; #define TCG_TARGET_HAS_mulsh_i64 1 #endif -#define TCG_TARGET_HAS_qemu_ldst_i128 0 +#define TCG_TARGET_HAS_qemu_ldst_i128 \ + (TCG_TARGET_REG_BITS == 64 && have_isa_2_07) /* * While technically Altivec could support V64, it has no 64-bit store diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc index b5c49895f3..c3a1527856 100644 --- a/tcg/ppc/tcg-target.c.inc +++ b/tcg/ppc/tcg-target.c.inc @@ -295,25 +295,27 @@ static bool tcg_target_const_match(int64_t val, TCGType type, int ct) #define B OPCD( 18) #define BC OPCD( 16) + #define LBZ OPCD( 34) #define LHZ OPCD( 40) #define LHA OPCD( 42) #define LWZ OPCD( 32) #define LWZUX XO31( 55) -#define STB OPCD( 38) -#define STH OPCD( 44) -#define STW OPCD( 36) - -#define STD XO62( 0) -#define STDU XO62( 1) -#define STDX XO31(149) - #define LD XO58( 0) #define LDX XO31( 21) #define LDU XO58( 1) #define LDUX XO31( 53) #define LWA XO58( 2) #define LWAX XO31(341) +#define LQ OPCD( 56) + +#define STB OPCD( 38) +#define STH OPCD( 44) +#define STW OPCD( 36) +#define STD XO62( 0) +#define STDU XO62( 1) +#define STDX XO31(149) +#define STQ XO62( 2) #define ADDIC OPCD( 12) #define ADDI OPCD( 14) @@ -2020,7 +2022,18 @@ typedef struct { bool tcg_target_has_memory_bswap(MemOp memop) { - return true; + TCGAtomAlign aa; + + if ((memop & MO_SIZE) <= MO_64) { + return true; + } + + /* + * Reject 16-byte memop with 16-byte atomicity, + * but do allow a pair of 64-bit operations. + */ + aa = atom_and_align_for_opc(tcg_ctx, memop, MO_ATOM_IFALIGN, true); + return aa.atom <= MO_64; } /* @@ -2035,7 +2048,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, { TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); - MemOp a_bits; + MemOp a_bits, s_bits; /* * Book II, Section 1.4, Single-Copy Atomicity, specifies: @@ -2047,10 +2060,11 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, * As of 3.0, "the non-atomic access is performed as described in * the corresponding list", which matches MO_ATOM_SUBALIGN. */ + s_bits = opc & MO_SIZE; h->aa = atom_and_align_for_opc(s, opc, have_isa_3_00 ? MO_ATOM_SUBALIGN : MO_ATOM_IFALIGN, - false); + s_bits == MO_128); a_bits = h->aa.align; #ifdef CONFIG_SOFTMMU @@ -2060,7 +2074,6 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, int fast_off = TLB_MASK_TABLE_OFS(mem_index); int mask_off = fast_off + offsetof(CPUTLBDescFast, mask); int table_off = fast_off + offsetof(CPUTLBDescFast, table); - unsigned s_bits = opc & MO_SIZE; ldst = new_ldst_label(s); ldst->is_ld = is_ld; @@ -2303,6 +2316,70 @@ static void tcg_out_qemu_st(TCGContext *s, TCGReg datalo, TCGReg datahi, } } +static TCGLabelQemuLdst * +prepare_host_addr_index_only(TCGContext *s, HostAddress *h, TCGReg addr_reg, + MemOpIdx oi, bool is_ld) +{ + TCGLabelQemuLdst *ldst; + + ldst = prepare_host_addr(s, h, addr_reg, -1, oi, true); + + /* Compose the final address, as LQ/STQ have no indexing. */ + if (h->base != 0) { + tcg_out32(s, ADD | TAB(TCG_REG_TMP1, h->base, h->index)); + h->index = TCG_REG_TMP1; + h->base = 0; + } + + return ldst; +} + +static void tcg_out_qemu_ldst_i128(TCGContext *s, TCGReg datalo, TCGReg datahi, + TCGReg addr_reg, MemOpIdx oi, bool is_ld) +{ + TCGLabelQemuLdst *ldst; + HostAddress h; + bool need_bswap; + uint32_t insn; + + ldst = prepare_host_addr_index_only(s, &h, addr_reg, oi, is_ld); + need_bswap = get_memop(oi) & MO_BSWAP; + + if (h.aa.atom == MO_128) { + tcg_debug_assert(!need_bswap); + tcg_debug_assert(datalo & 1); + tcg_debug_assert(datahi == datalo - 1); + insn = is_ld ? LQ : STQ; + tcg_out32(s, insn | TAI(datahi, h.index, 0)); + } else { + TCGReg d1, d2; + + if (HOST_BIG_ENDIAN ^ need_bswap) { + d1 = datahi, d2 = datalo; + } else { + d1 = datalo, d2 = datahi; + } + + if (need_bswap) { + tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_R0, 8); + insn = is_ld ? LDBRX : STDBRX; + tcg_out32(s, insn | TAB(d1, 0, h.index)); + tcg_out32(s, insn | TAB(d2, h.index, TCG_REG_R0)); + } else { + insn = is_ld ? LD : STD; + tcg_out32(s, insn | TAI(d1, h.index, 0)); + tcg_out32(s, insn | TAI(d2, h.index, 8)); + } + } + + if (ldst) { + ldst->type = TCG_TYPE_I128; + ldst->datalo_reg = datalo; + ldst->datahi_reg = datahi; + ldst->raddr = tcg_splitwx_to_rx(s->code_ptr); + } +} + static void tcg_out_nop_fill(tcg_insn_unit *p, int count) { int i; @@ -2853,6 +2930,11 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, args[4], TCG_TYPE_I64); } break; + case INDEX_op_qemu_ld_i128: + tcg_debug_assert(TCG_TARGET_REG_BITS == 64); + tcg_out_qemu_ldst_i128(s, args[0], args[1], args[2], args[3], true); + break; + case INDEX_op_qemu_st_i32: if (TCG_TARGET_REG_BITS >= TARGET_LONG_BITS) { tcg_out_qemu_st(s, args[0], -1, args[1], -1, @@ -2874,6 +2956,10 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, args[4], TCG_TYPE_I64); } break; + case INDEX_op_qemu_st_i128: + tcg_debug_assert(TCG_TARGET_REG_BITS == 64); + tcg_out_qemu_ldst_i128(s, args[0], args[1], args[2], args[3], false); + break; case INDEX_op_setcond_i32: tcg_out_setcond(s, TCG_TYPE_I32, args[3], args[0], args[1], args[2], @@ -3709,6 +3795,11 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) : TARGET_LONG_BITS == 32 ? C_O0_I3(r, r, r) : C_O0_I4(r, r, r, r)); + case INDEX_op_qemu_ld_i128: + return C_O2_I1(o, m, r); + case INDEX_op_qemu_st_i128: + return C_O0_I3(o, m, r); + case INDEX_op_add_vec: case INDEX_op_sub_vec: case INDEX_op_mul_vec: From patchwork Tue May 16 19:41:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682414 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp20841wrt; Tue, 16 May 2023 12:58:40 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4BtE8CtQ3QEAAv3WIp/Se9AiB9Cjqh6kmoB4Xf8d5HevP2p7wZliJffKs2yVdrERVa99rR X-Received: by 2002:adf:f544:0:b0:307:7959:6461 with SMTP id j4-20020adff544000000b0030779596461mr25133997wrp.31.1684267120076; Tue, 16 May 2023 12:58:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267120; cv=none; d=google.com; s=arc-20160816; b=cZJCXEX/M4CtS0AtTMNjofYkQiHjjnT/6n6UB5zo+Q9BhbU4a5327v2+m4DbBmpbpF hSDi7HYot7VcsUU8X9PaptyKturEt6Xm4ZwF5990S+cKfgkKk9p177dAoa/B7nYC1Fr+ IfaT0ughJXhAJRd0wsa73LyxS3SR8PgqiL9caZuI2358tX3H0XLG9xhFpUXEHgdlpR19 hMbaKzyxhg820MHYH+5oACRJDArdxklrUf9P7LomhQNU8ibFrBl1H352rm2/nak/AEFk oGOUIjPZi+6rq/+tf3uZ5ow9zkk+Q25BMDdj1e9uAQiHbkIsQtQ7pVcTYJiJ+hprn2Vi sW+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=J7XEtwm+OcbKBkD8wOKb8qxC2YJxosgTJywZE8mVEkM=; b=F4xMCfd633nnw2XYdASOSYMNlqPzlI/PirmDj4rijaqUg428ft1yUjs3PJIIc1yYET 5sc27cmkeuD+5b9vAGV/wkITsy27+uTwMuM8Lxt6lfb77ARS5pRbdZAlSmpLqJJy3l8Y 85/t7aItui0FPr0diDt5D02IITWS3827g3PDNBJw82/fEWnM7xa5giUMH6cPZR8ciU0d MulOHrN1f9lqh5SxEEaGSE8FC/acwGa6zdnTDITyGruXYrvvQZo3l0XdfIJ/srNNQCzL YqTXYvZsC/xlXb5okY2MAsT5rmex67yl9KOUEEe+umJ6FLdF/o1C9RHqYbRUDiJAyMsO x+Ag== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lWEU0A4u; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n11-20020a5d484b000000b0030642f5da36si158240wrs.752.2023.05.16.12.58.39 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:58:40 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lWEU0A4u; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0ca-0000t5-9v; Tue, 16 May 2023 15:46:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cC-0000Eo-8Y for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:46 -0400 Received: from mail-pj1-x1036.google.com ([2607:f8b0:4864:20::1036]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0c3-0003ji-Gb for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:40 -0400 Received: by mail-pj1-x1036.google.com with SMTP id 98e67ed59e1d1-2502346bea0so34190a91.2 for ; Tue, 16 May 2023 12:45:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266334; x=1686858334; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=J7XEtwm+OcbKBkD8wOKb8qxC2YJxosgTJywZE8mVEkM=; b=lWEU0A4uzhlFox56/qk5cMlxgo3RKqBd8qPgqJTd4WEnELAcoTVC31/oUE8/WjDOrm /7c3V6zYOTnAB1DJdJYgMDQxxPlA0lOnjB+R3xj7QjLCAD2RyNyyztmR96rS/OrOKDKz p9jxZWIaMYHix5Ot+T8nTfyMxAst4UQIInqKX1z0hwuBjLzkoQFOQoLiitbeVdpxp3SX x017wiVJphwHQ+b30UKR1xHoHhV9SVCHbPKDETe/vkfPgTN2e415HFf/TEj8mwfAJysE Us3QbHdntWsnMQ+uzPq1s+8hSB3fRtYG5LgWNjzcybyYR0FxNcHaiJskElxkroC4PY5/ LmDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266334; x=1686858334; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=J7XEtwm+OcbKBkD8wOKb8qxC2YJxosgTJywZE8mVEkM=; b=Rreg7rTkUVlotTj0VpRYorQG2ISh8Yb+3OjeiDTFS+WDZPENJdrMp9ddQbCgWFw+Qd YGRj6jSv1QRACetZ2SbU8nqICiSxsreB+OB089nYgPDnSNaRatjUtOxUHzOR7hY6ZitX XbWjvdojMGYUfayqeZ3CgBYu5ZWfnzWyNzvPjD4BZsleYh+5yPy7xmuE++ardyLaJUKc OxvFhHUqt07r1z11I0DSqTgt4daBl6nCDBSuOmZIt7Y9BLFhfFviqRgvXZCpZKasLDon HZM2C/RNovgzMfrqUGOi127IGLDEyw0KS/cjNS+RFjgrV8ipQH4Shd/cIMx1Wk9jWjxh DO/w== X-Gm-Message-State: AC+VfDyP8KWUvBZC3Ka/41CdgG1nyVMrVeKYbr9X7DW6ZZa/4MaJzz4d P6bvhXFEp9Ce2NgpDP8BhRXUy+6xi8VE6q0cnLY= X-Received: by 2002:a17:90b:1d87:b0:24f:13e7:e42a with SMTP id pf7-20020a17090b1d8700b0024f13e7e42amr39106837pjb.28.1684266333927; Tue, 16 May 2023 12:45:33 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:33 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PULL 52/80] tcg/s390x: Support 128-bit load/store Date: Tue, 16 May 2023 12:41:17 -0700 Message-Id: <20230516194145.1749305-53-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1036; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1036.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use LPQ/STPQ when 16-byte atomicity is required. Note that these instructions require 16-byte alignment. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/s390x/tcg-target-con-set.h | 2 + tcg/s390x/tcg-target.h | 2 +- tcg/s390x/tcg-target.c.inc | 103 ++++++++++++++++++++++++++++++++- 3 files changed, 103 insertions(+), 4 deletions(-) diff --git a/tcg/s390x/tcg-target-con-set.h b/tcg/s390x/tcg-target-con-set.h index ecc079bb6d..cbad91b2b5 100644 --- a/tcg/s390x/tcg-target-con-set.h +++ b/tcg/s390x/tcg-target-con-set.h @@ -14,6 +14,7 @@ C_O0_I2(r, r) C_O0_I2(r, ri) C_O0_I2(r, rA) C_O0_I2(v, r) +C_O0_I3(o, m, r) C_O1_I1(r, r) C_O1_I1(v, r) C_O1_I1(v, v) @@ -36,6 +37,7 @@ C_O1_I2(v, v, v) C_O1_I3(v, v, v, v) C_O1_I4(r, r, ri, rI, r) C_O1_I4(r, r, rA, rI, r) +C_O2_I1(o, m, r) C_O2_I2(o, m, 0, r) C_O2_I2(o, m, r, r) C_O2_I3(o, m, 0, 1, r) diff --git a/tcg/s390x/tcg-target.h b/tcg/s390x/tcg-target.h index 170007bea5..ec96952172 100644 --- a/tcg/s390x/tcg-target.h +++ b/tcg/s390x/tcg-target.h @@ -140,7 +140,7 @@ extern uint64_t s390_facilities[3]; #define TCG_TARGET_HAS_muluh_i64 0 #define TCG_TARGET_HAS_mulsh_i64 0 -#define TCG_TARGET_HAS_qemu_ldst_i128 0 +#define TCG_TARGET_HAS_qemu_ldst_i128 1 #define TCG_TARGET_HAS_v64 HAVE_FACILITY(VECTOR) #define TCG_TARGET_HAS_v128 HAVE_FACILITY(VECTOR) diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc index 8e34b214fc..835daa51fa 100644 --- a/tcg/s390x/tcg-target.c.inc +++ b/tcg/s390x/tcg-target.c.inc @@ -243,6 +243,7 @@ typedef enum S390Opcode { RXY_LLGF = 0xe316, RXY_LLGH = 0xe391, RXY_LMG = 0xeb04, + RXY_LPQ = 0xe38f, RXY_LRV = 0xe31e, RXY_LRVG = 0xe30f, RXY_LRVH = 0xe31f, @@ -253,6 +254,7 @@ typedef enum S390Opcode { RXY_STG = 0xe324, RXY_STHY = 0xe370, RXY_STMG = 0xeb24, + RXY_STPQ = 0xe38e, RXY_STRV = 0xe33e, RXY_STRVG = 0xe32f, RXY_STRVH = 0xe33f, @@ -1577,7 +1579,18 @@ typedef struct { bool tcg_target_has_memory_bswap(MemOp memop) { - return true; + TCGAtomAlign aa; + + if ((memop & MO_SIZE) <= MO_64) { + return true; + } + + /* + * Reject 16-byte memop with 16-byte atomicity, + * but do allow a pair of 64-bit operations. + */ + aa = atom_and_align_for_opc(tcg_ctx, memop, MO_ATOM_IFALIGN, true); + return aa.atom <= MO_64; } static void tcg_out_qemu_ld_direct(TCGContext *s, MemOp opc, TCGReg data, @@ -1734,13 +1747,13 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, { TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); + MemOp s_bits = opc & MO_SIZE; unsigned a_mask; - h->aa = atom_and_align_for_opc(s, opc, MO_ATOM_IFALIGN, false); + h->aa = atom_and_align_for_opc(s, opc, MO_ATOM_IFALIGN, s_bits == MO_128); a_mask = (1 << h->aa.align) - 1; #ifdef CONFIG_SOFTMMU - unsigned s_bits = opc & MO_SIZE; unsigned s_mask = (1 << s_bits) - 1; int mem_index = get_mmuidx(oi); int fast_off = TLB_MASK_TABLE_OFS(mem_index); @@ -1865,6 +1878,80 @@ static void tcg_out_qemu_st(TCGContext* s, TCGReg data_reg, TCGReg addr_reg, } } +static void tcg_out_qemu_ldst_i128(TCGContext *s, TCGReg datalo, TCGReg datahi, + TCGReg addr_reg, MemOpIdx oi, bool is_ld) +{ + TCGLabel *l1 = NULL, *l2 = NULL; + TCGLabelQemuLdst *ldst; + HostAddress h; + bool need_bswap; + bool use_pair; + S390Opcode insn; + + ldst = prepare_host_addr(s, &h, addr_reg, oi, is_ld); + + use_pair = h.aa.atom < MO_128; + need_bswap = get_memop(oi) & MO_BSWAP; + + if (!use_pair) { + /* + * Atomicity requires we use LPQ. If we've already checked for + * 16-byte alignment, that's all we need. If we arrive with + * lesser alignment, we have determined that less than 16-byte + * alignment can be satisfied with two 8-byte loads. + */ + if (h.aa.align < MO_128) { + use_pair = true; + l1 = gen_new_label(); + l2 = gen_new_label(); + + tcg_out_insn(s, RI, TMLL, addr_reg, 15); + tgen_branch(s, 7, l1); /* CC in {1,2,3} */ + } + + tcg_debug_assert(!need_bswap); + tcg_debug_assert(datalo & 1); + tcg_debug_assert(datahi == datalo - 1); + insn = is_ld ? RXY_LPQ : RXY_STPQ; + tcg_out_insn_RXY(s, insn, datahi, h.base, h.index, h.disp); + + if (use_pair) { + tgen_branch(s, S390_CC_ALWAYS, l2); + tcg_out_label(s, l1); + } + } + if (use_pair) { + TCGReg d1, d2; + + if (need_bswap) { + d1 = datalo, d2 = datahi; + insn = is_ld ? RXY_LRVG : RXY_STRVG; + } else { + d1 = datahi, d2 = datalo; + insn = is_ld ? RXY_LG : RXY_STG; + } + + if (h.base == d1 || h.index == d1) { + tcg_out_insn(s, RXY, LAY, TCG_TMP0, h.base, h.index, h.disp); + h.base = TCG_TMP0; + h.index = TCG_REG_NONE; + h.disp = 0; + } + tcg_out_insn_RXY(s, insn, d1, h.base, h.index, h.disp); + tcg_out_insn_RXY(s, insn, d2, h.base, h.index, h.disp + 8); + } + if (l2) { + tcg_out_label(s, l2); + } + + if (ldst) { + ldst->type = TCG_TYPE_I128; + ldst->datalo_reg = datalo; + ldst->datahi_reg = datahi; + ldst->raddr = tcg_splitwx_to_rx(s->code_ptr); + } +} + static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) { /* Reuse the zeroing that exists for goto_ptr. */ @@ -2222,6 +2309,12 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_qemu_st_i64: tcg_out_qemu_st(s, args[0], args[1], args[2], TCG_TYPE_I64); break; + case INDEX_op_qemu_ld_i128: + tcg_out_qemu_ldst_i128(s, args[0], args[1], args[2], args[3], true); + break; + case INDEX_op_qemu_st_i128: + tcg_out_qemu_ldst_i128(s, args[0], args[1], args[2], args[3], false); + break; case INDEX_op_ld16s_i64: tcg_out_mem(s, 0, RXY_LGH, args[0], args[1], TCG_REG_NONE, args[2]); @@ -3099,6 +3192,10 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_qemu_st_i64: case INDEX_op_qemu_st_i32: return C_O0_I2(r, r); + case INDEX_op_qemu_ld_i128: + return C_O2_I1(o, m, r); + case INDEX_op_qemu_st_i128: + return C_O0_I3(o, m, r); case INDEX_op_deposit_i32: case INDEX_op_deposit_i64: From patchwork Tue May 16 19:41:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682427 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp21114wrt; Tue, 16 May 2023 12:59:41 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5690W8+cxzKGH99f5l5a1BNsBy6fvc+DJokhNZ8mpLKSshZrz1zQb7wHMw7Met+Hoa6fZB X-Received: by 2002:a7b:c7d4:0:b0:3f4:2d22:536a with SMTP id z20-20020a7bc7d4000000b003f42d22536amr16496220wmk.19.1684267180752; Tue, 16 May 2023 12:59:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267180; cv=none; d=google.com; s=arc-20160816; b=jh3EwasW3P3wguyBR0k+7lN2l5VWwePoWFuyf6v+HpR1QBp9jk8YJl+NgbEt+S+Mr2 cweenOFP3OhJFWmbMh49Mwjz4TT8/WwSZwgGZ8XwSh/S9tHZP7yWbYJf74gUq+Y//5s0 OimPzfpChWpC3n63CeZbWc2P2yH3iwJLdGM/6b06Gj1Y9MjNXtVLUr2Vi5p09qthaXMg 9OPw1pyNqBPHFbQqZWXU/MJXUwM4pftr4EkL5M+dt5g90Ax/tt6X3p+W2UNM9hnahgJN WaV5ceGiOGyWemBeifM6+wLlOA0lsYXY63s4vOCpWL6SiPRbyrGLMa/Pahd1PavYNhsA a4pQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=GnOoWZ8n2BTaXrySEiTbt7oj9ue8e7GtkntGWJ2uxd8=; b=zWa5F5MD47AhxPasuBr9JxVbu15FI75uRKYoTWnGyAomCqFRkaLQAHNsLnuLfckLrr 0sKCkKoi4rdgY68knToD5E9Kx+jsjR9g4fWKGxzkGS/1Tb1gdZBavqtj2xC+RS7eW5ak tlvExtpa1b1P8QttbHR/sscd1xCwaozFQkeuyPfc9uQcVhQ5XdhkZQ/325UW+YiEA2+b KwEPMWKGvwN+ANkfhVnSr71bOLj3SpJgKdN7zEq4mPx5t5nKxVqgeDlutdfN+5z/vdUO L/ws6Ddv+P6Xc552OlKnGlTIwITPrV7iPE4r6xLCtM46+2EuYa2wz4TIh7LywPCS6T5R 2ByQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Fp+yE1Ir; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x7-20020a1c7c07000000b003f508f0d108si124982wmc.111.2023.05.16.12.59.40 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:59:40 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Fp+yE1Ir; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0eo-00035T-AF; Tue, 16 May 2023 15:48:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cE-0000Ha-IG for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:49 -0400 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0c6-0003jt-JL for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:44 -0400 Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-64ab2a37812so8129247b3a.1 for ; Tue, 16 May 2023 12:45:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266335; x=1686858335; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GnOoWZ8n2BTaXrySEiTbt7oj9ue8e7GtkntGWJ2uxd8=; b=Fp+yE1IrCg0TTxlGDk4Ehj9qkxp8MfqrQjKHCOiqCIGf5dI1NiGqHKMUsNscMQrnuQ rKzQK5TiE1EvKWmKDYctBPg50tYmUvAuNkKRbf7gQrh9u8jLn7kogPNUhMQvoClbGpDg vFacbCM+kEGTvclI6irEt6DCb4FVqHoeIi58jMJtyQ9yI7KjwJ7+kLo5ovPPvWie//JP jqmwPe5necIe1/VBHjEgQ5b3ObK4Z42OqSH47bEA8Elr0iSksAf7gTbpPDhfIwiBU2l/ 55af9yoMZ/Xl4QekQoYb+rdWDeY5oY8bqRiZUGhleOYyQwdItVqAHCoD/Knn+mWjvf2A QSxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266335; x=1686858335; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GnOoWZ8n2BTaXrySEiTbt7oj9ue8e7GtkntGWJ2uxd8=; b=OMcGhbWSJyk2RLi8nT2hAFxXZ/759NdTSOBLwCx/1VF0pfloVKbKzb10YAodtcSaLn BLDnCZrHgB6WNdN3Qk9YmaSLWq2D3deKslDl89KWIbjx8GwkZb0U0pxLr6q1AuOuGD66 lLgPFY1fY7JGfsKUkea7I5tjxknSSPgx7dq2D5Qf5pPtRAINTqZatHY7KdvAZ5oKwc3I xqOX7FMYGGOh4DcyKryKGQkCOpnJD+mSXbl7q7F+I7MY5h31jrPBl4udcn8QUEShlIOw Xi0fHktwlMk4YJn611AvEqNuSbiu6t22jHWON0iO1zScrZSvek6nHQBsgDuEXkdSm8La ES9A== X-Gm-Message-State: AC+VfDzPzfC+v9mtncPJIBRoI1/Nd2LtFLNny6JIF8ax7E5IWm7qLbmY /iQY8AM/snTlsTmZJjGqzoscrkZSpCnDWVjGUM8= X-Received: by 2002:a17:90a:bb05:b0:250:af6d:bd7b with SMTP id u5-20020a17090abb0500b00250af6dbd7bmr27409295pjr.24.1684266334832; Tue, 16 May 2023 12:45:34 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:34 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 53/80] tcg: Split out memory ops to tcg-op-ldst.c Date: Tue, 16 May 2023 12:41:18 -0700 Message-Id: <20230516194145.1749305-54-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::436; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x436.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- tcg/tcg-op-ldst.c | 1006 +++++++++++++++++++++++++++++++++++++++++++++ tcg/tcg-op.c | 974 ------------------------------------------- tcg/meson.build | 1 + 3 files changed, 1007 insertions(+), 974 deletions(-) create mode 100644 tcg/tcg-op-ldst.c diff --git a/tcg/tcg-op-ldst.c b/tcg/tcg-op-ldst.c new file mode 100644 index 0000000000..d8503d7da4 --- /dev/null +++ b/tcg/tcg-op-ldst.c @@ -0,0 +1,1006 @@ +/* + * Tiny Code Generator for QEMU + * + * Copyright (c) 2008 Fabrice Bellard + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +#include "qemu/osdep.h" +#include "exec/exec-all.h" +#include "tcg/tcg.h" +#include "tcg/tcg-temp-internal.h" +#include "tcg/tcg-op.h" +#include "tcg/tcg-mo.h" +#include "exec/plugin-gen.h" +#include "tcg-internal.h" + + +static inline MemOp tcg_canonicalize_memop(MemOp op, bool is64, bool st) +{ + /* Trigger the asserts within as early as possible. */ + unsigned a_bits = get_alignment_bits(op); + + /* Prefer MO_ALIGN+MO_XX over MO_ALIGN_XX+MO_XX */ + if (a_bits == (op & MO_SIZE)) { + op = (op & ~MO_AMASK) | MO_ALIGN; + } + + switch (op & MO_SIZE) { + case MO_8: + op &= ~MO_BSWAP; + break; + case MO_16: + break; + case MO_32: + if (!is64) { + op &= ~MO_SIGN; + } + break; + case MO_64: + if (is64) { + op &= ~MO_SIGN; + break; + } + /* fall through */ + default: + g_assert_not_reached(); + } + if (st) { + op &= ~MO_SIGN; + } + return op; +} + +static void gen_ldst_i32(TCGOpcode opc, TCGv_i32 val, TCGv addr, + MemOp memop, TCGArg idx) +{ + MemOpIdx oi = make_memop_idx(memop, idx); +#if TARGET_LONG_BITS == 32 + tcg_gen_op3i_i32(opc, val, addr, oi); +#else + if (TCG_TARGET_REG_BITS == 32) { + tcg_gen_op4i_i32(opc, val, TCGV_LOW(addr), TCGV_HIGH(addr), oi); + } else { + tcg_gen_op3(opc, tcgv_i32_arg(val), tcgv_i64_arg(addr), oi); + } +#endif +} + +static void gen_ldst_i64(TCGOpcode opc, TCGv_i64 val, TCGv addr, + MemOp memop, TCGArg idx) +{ + MemOpIdx oi = make_memop_idx(memop, idx); +#if TARGET_LONG_BITS == 32 + if (TCG_TARGET_REG_BITS == 32) { + tcg_gen_op4i_i32(opc, TCGV_LOW(val), TCGV_HIGH(val), addr, oi); + } else { + tcg_gen_op3(opc, tcgv_i64_arg(val), tcgv_i32_arg(addr), oi); + } +#else + if (TCG_TARGET_REG_BITS == 32) { + tcg_gen_op5i_i32(opc, TCGV_LOW(val), TCGV_HIGH(val), + TCGV_LOW(addr), TCGV_HIGH(addr), oi); + } else { + tcg_gen_op3i_i64(opc, val, addr, oi); + } +#endif +} + +static void tcg_gen_req_mo(TCGBar type) +{ +#ifdef TCG_GUEST_DEFAULT_MO + type &= TCG_GUEST_DEFAULT_MO; +#endif + type &= ~TCG_TARGET_DEFAULT_MO; + if (type) { + tcg_gen_mb(type | TCG_BAR_SC); + } +} + +static inline TCGv plugin_prep_mem_callbacks(TCGv vaddr) +{ +#ifdef CONFIG_PLUGIN + if (tcg_ctx->plugin_insn != NULL) { + /* Save a copy of the vaddr for use after a load. */ + TCGv temp = tcg_temp_new(); + tcg_gen_mov_tl(temp, vaddr); + return temp; + } +#endif + return vaddr; +} + +static void plugin_gen_mem_callbacks(TCGv vaddr, MemOpIdx oi, + enum qemu_plugin_mem_rw rw) +{ +#ifdef CONFIG_PLUGIN + if (tcg_ctx->plugin_insn != NULL) { + qemu_plugin_meminfo_t info = make_plugin_meminfo(oi, rw); + plugin_gen_empty_mem_callback(vaddr, info); + tcg_temp_free(vaddr); + } +#endif +} + +void tcg_gen_qemu_ld_i32(TCGv_i32 val, TCGv addr, TCGArg idx, MemOp memop) +{ + MemOp orig_memop; + MemOpIdx oi; + + tcg_gen_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); + memop = tcg_canonicalize_memop(memop, 0, 0); + oi = make_memop_idx(memop, idx); + + orig_memop = memop; + if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { + memop &= ~MO_BSWAP; + /* The bswap primitive benefits from zero-extended input. */ + if ((memop & MO_SSIZE) == MO_SW) { + memop &= ~MO_SIGN; + } + } + + addr = plugin_prep_mem_callbacks(addr); + gen_ldst_i32(INDEX_op_qemu_ld_i32, val, addr, memop, idx); + plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_R); + + if ((orig_memop ^ memop) & MO_BSWAP) { + switch (orig_memop & MO_SIZE) { + case MO_16: + tcg_gen_bswap16_i32(val, val, (orig_memop & MO_SIGN + ? TCG_BSWAP_IZ | TCG_BSWAP_OS + : TCG_BSWAP_IZ | TCG_BSWAP_OZ)); + break; + case MO_32: + tcg_gen_bswap32_i32(val, val); + break; + default: + g_assert_not_reached(); + } + } +} + +void tcg_gen_qemu_st_i32(TCGv_i32 val, TCGv addr, TCGArg idx, MemOp memop) +{ + TCGv_i32 swap = NULL; + MemOpIdx oi; + + tcg_gen_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST); + memop = tcg_canonicalize_memop(memop, 0, 1); + oi = make_memop_idx(memop, idx); + + if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { + swap = tcg_temp_ebb_new_i32(); + switch (memop & MO_SIZE) { + case MO_16: + tcg_gen_bswap16_i32(swap, val, 0); + break; + case MO_32: + tcg_gen_bswap32_i32(swap, val); + break; + default: + g_assert_not_reached(); + } + val = swap; + memop &= ~MO_BSWAP; + } + + addr = plugin_prep_mem_callbacks(addr); + if (TCG_TARGET_HAS_qemu_st8_i32 && (memop & MO_SIZE) == MO_8) { + gen_ldst_i32(INDEX_op_qemu_st8_i32, val, addr, memop, idx); + } else { + gen_ldst_i32(INDEX_op_qemu_st_i32, val, addr, memop, idx); + } + plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_W); + + if (swap) { + tcg_temp_free_i32(swap); + } +} + +void tcg_gen_qemu_ld_i64(TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop) +{ + MemOp orig_memop; + MemOpIdx oi; + + if (TCG_TARGET_REG_BITS == 32 && (memop & MO_SIZE) < MO_64) { + tcg_gen_qemu_ld_i32(TCGV_LOW(val), addr, idx, memop); + if (memop & MO_SIGN) { + tcg_gen_sari_i32(TCGV_HIGH(val), TCGV_LOW(val), 31); + } else { + tcg_gen_movi_i32(TCGV_HIGH(val), 0); + } + return; + } + + tcg_gen_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); + memop = tcg_canonicalize_memop(memop, 1, 0); + oi = make_memop_idx(memop, idx); + + orig_memop = memop; + if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { + memop &= ~MO_BSWAP; + /* The bswap primitive benefits from zero-extended input. */ + if ((memop & MO_SIGN) && (memop & MO_SIZE) < MO_64) { + memop &= ~MO_SIGN; + } + } + + addr = plugin_prep_mem_callbacks(addr); + gen_ldst_i64(INDEX_op_qemu_ld_i64, val, addr, memop, idx); + plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_R); + + if ((orig_memop ^ memop) & MO_BSWAP) { + int flags = (orig_memop & MO_SIGN + ? TCG_BSWAP_IZ | TCG_BSWAP_OS + : TCG_BSWAP_IZ | TCG_BSWAP_OZ); + switch (orig_memop & MO_SIZE) { + case MO_16: + tcg_gen_bswap16_i64(val, val, flags); + break; + case MO_32: + tcg_gen_bswap32_i64(val, val, flags); + break; + case MO_64: + tcg_gen_bswap64_i64(val, val); + break; + default: + g_assert_not_reached(); + } + } +} + +void tcg_gen_qemu_st_i64(TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop) +{ + TCGv_i64 swap = NULL; + MemOpIdx oi; + + if (TCG_TARGET_REG_BITS == 32 && (memop & MO_SIZE) < MO_64) { + tcg_gen_qemu_st_i32(TCGV_LOW(val), addr, idx, memop); + return; + } + + tcg_gen_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST); + memop = tcg_canonicalize_memop(memop, 1, 1); + oi = make_memop_idx(memop, idx); + + if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { + swap = tcg_temp_ebb_new_i64(); + switch (memop & MO_SIZE) { + case MO_16: + tcg_gen_bswap16_i64(swap, val, 0); + break; + case MO_32: + tcg_gen_bswap32_i64(swap, val, 0); + break; + case MO_64: + tcg_gen_bswap64_i64(swap, val); + break; + default: + g_assert_not_reached(); + } + val = swap; + memop &= ~MO_BSWAP; + } + + addr = plugin_prep_mem_callbacks(addr); + gen_ldst_i64(INDEX_op_qemu_st_i64, val, addr, memop, idx); + plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_W); + + if (swap) { + tcg_temp_free_i64(swap); + } +} + +/* + * Return true if @mop, without knowledge of the pointer alignment, + * does not require 16-byte atomicity, and it would be adventagous + * to avoid a call to a helper function. + */ +static bool use_two_i64_for_i128(MemOp mop) +{ +#ifdef CONFIG_SOFTMMU + /* Two softmmu tlb lookups is larger than one function call. */ + return false; +#else + /* + * For user-only, two 64-bit operations may well be smaller than a call. + * Determine if that would be legal for the requested atomicity. + */ + switch (mop & MO_ATOM_MASK) { + case MO_ATOM_NONE: + case MO_ATOM_IFALIGN_PAIR: + return true; + case MO_ATOM_IFALIGN: + case MO_ATOM_SUBALIGN: + case MO_ATOM_WITHIN16: + case MO_ATOM_WITHIN16_PAIR: + /* In a serialized context, no atomicity is required. */ + return !(tcg_ctx->gen_tb->cflags & CF_PARALLEL); + default: + g_assert_not_reached(); + } +#endif +} + +static void canonicalize_memop_i128_as_i64(MemOp ret[2], MemOp orig) +{ + MemOp mop_1 = orig, mop_2; + + tcg_debug_assert((orig & MO_SIZE) == MO_128); + tcg_debug_assert((orig & MO_SIGN) == 0); + + /* Reduce the size to 64-bit. */ + mop_1 = (mop_1 & ~MO_SIZE) | MO_64; + + /* Retain the alignment constraints of the original. */ + switch (orig & MO_AMASK) { + case MO_UNALN: + case MO_ALIGN_2: + case MO_ALIGN_4: + mop_2 = mop_1; + break; + case MO_ALIGN_8: + /* Prefer MO_ALIGN+MO_64 to MO_ALIGN_8+MO_64. */ + mop_1 = (mop_1 & ~MO_AMASK) | MO_ALIGN; + mop_2 = mop_1; + break; + case MO_ALIGN: + /* Second has 8-byte alignment; first has 16-byte alignment. */ + mop_2 = mop_1; + mop_1 = (mop_1 & ~MO_AMASK) | MO_ALIGN_16; + break; + case MO_ALIGN_16: + case MO_ALIGN_32: + case MO_ALIGN_64: + /* Second has 8-byte alignment; first retains original. */ + mop_2 = (mop_1 & ~MO_AMASK) | MO_ALIGN; + break; + default: + g_assert_not_reached(); + } + + /* Use a memory ordering implemented by the host. */ + if ((orig & MO_BSWAP) && !tcg_target_has_memory_bswap(mop_1)) { + mop_1 &= ~MO_BSWAP; + mop_2 &= ~MO_BSWAP; + } + + ret[0] = mop_1; + ret[1] = mop_2; +} + +#if TARGET_LONG_BITS == 64 +#define tcg_temp_ebb_new tcg_temp_ebb_new_i64 +#else +#define tcg_temp_ebb_new tcg_temp_ebb_new_i32 +#endif + +void tcg_gen_qemu_ld_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) +{ + const MemOpIdx oi = make_memop_idx(memop, idx); + + tcg_debug_assert((memop & MO_SIZE) == MO_128); + tcg_debug_assert((memop & MO_SIGN) == 0); + + tcg_gen_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); + addr = plugin_prep_mem_callbacks(addr); + + /* TODO: For now, force 32-bit hosts to use the helper. */ + if (TCG_TARGET_HAS_qemu_ldst_i128 && TCG_TARGET_REG_BITS == 64) { + TCGv_i64 lo, hi; + TCGArg addr_arg; + MemOpIdx adj_oi; + bool need_bswap = false; + + if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { + lo = TCGV128_HIGH(val); + hi = TCGV128_LOW(val); + adj_oi = make_memop_idx(memop & ~MO_BSWAP, idx); + need_bswap = true; + } else { + lo = TCGV128_LOW(val); + hi = TCGV128_HIGH(val); + adj_oi = oi; + } + +#if TARGET_LONG_BITS == 32 + addr_arg = tcgv_i32_arg(addr); +#else + addr_arg = tcgv_i64_arg(addr); +#endif + tcg_gen_op4ii_i64(INDEX_op_qemu_ld_i128, lo, hi, addr_arg, adj_oi); + + if (need_bswap) { + tcg_gen_bswap64_i64(lo, lo); + tcg_gen_bswap64_i64(hi, hi); + } + } else if (use_two_i64_for_i128(memop)) { + MemOp mop[2]; + TCGv addr_p8; + TCGv_i64 x, y; + + canonicalize_memop_i128_as_i64(mop, memop); + + /* + * Since there are no global TCGv_i128, there is no visible state + * changed if the second load faults. Load directly into the two + * subwords. + */ + if ((memop & MO_BSWAP) == MO_LE) { + x = TCGV128_LOW(val); + y = TCGV128_HIGH(val); + } else { + x = TCGV128_HIGH(val); + y = TCGV128_LOW(val); + } + + gen_ldst_i64(INDEX_op_qemu_ld_i64, x, addr, mop[0], idx); + + if ((mop[0] ^ memop) & MO_BSWAP) { + tcg_gen_bswap64_i64(x, x); + } + + addr_p8 = tcg_temp_ebb_new(); + tcg_gen_addi_tl(addr_p8, addr, 8); + gen_ldst_i64(INDEX_op_qemu_ld_i64, y, addr_p8, mop[1], idx); + tcg_temp_free(addr_p8); + + if ((mop[0] ^ memop) & MO_BSWAP) { + tcg_gen_bswap64_i64(y, y); + } + } else { + gen_helper_ld_i128(val, cpu_env, addr, tcg_constant_i32(oi)); + } + + plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_R); +} + +void tcg_gen_qemu_st_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) +{ + const MemOpIdx oi = make_memop_idx(memop, idx); + + tcg_debug_assert((memop & MO_SIZE) == MO_128); + tcg_debug_assert((memop & MO_SIGN) == 0); + + tcg_gen_req_mo(TCG_MO_ST_LD | TCG_MO_ST_ST); + addr = plugin_prep_mem_callbacks(addr); + + /* TODO: For now, force 32-bit hosts to use the helper. */ + + if (TCG_TARGET_HAS_qemu_ldst_i128 && TCG_TARGET_REG_BITS == 64) { + TCGv_i64 lo, hi; + TCGArg addr_arg; + MemOpIdx adj_oi; + bool need_bswap = false; + + if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { + lo = tcg_temp_new_i64(); + hi = tcg_temp_new_i64(); + tcg_gen_bswap64_i64(lo, TCGV128_HIGH(val)); + tcg_gen_bswap64_i64(hi, TCGV128_LOW(val)); + adj_oi = make_memop_idx(memop & ~MO_BSWAP, idx); + need_bswap = true; + } else { + lo = TCGV128_LOW(val); + hi = TCGV128_HIGH(val); + adj_oi = oi; + } + +#if TARGET_LONG_BITS == 32 + addr_arg = tcgv_i32_arg(addr); +#else + addr_arg = tcgv_i64_arg(addr); +#endif + tcg_gen_op4ii_i64(INDEX_op_qemu_st_i128, lo, hi, addr_arg, adj_oi); + + if (need_bswap) { + tcg_temp_free_i64(lo); + tcg_temp_free_i64(hi); + } + } else if (use_two_i64_for_i128(memop)) { + MemOp mop[2]; + TCGv addr_p8; + TCGv_i64 x, y; + + canonicalize_memop_i128_as_i64(mop, memop); + + if ((memop & MO_BSWAP) == MO_LE) { + x = TCGV128_LOW(val); + y = TCGV128_HIGH(val); + } else { + x = TCGV128_HIGH(val); + y = TCGV128_LOW(val); + } + + addr_p8 = tcg_temp_ebb_new(); + if ((mop[0] ^ memop) & MO_BSWAP) { + TCGv_i64 t = tcg_temp_ebb_new_i64(); + + tcg_gen_bswap64_i64(t, x); + gen_ldst_i64(INDEX_op_qemu_st_i64, t, addr, mop[0], idx); + tcg_gen_bswap64_i64(t, y); + tcg_gen_addi_tl(addr_p8, addr, 8); + gen_ldst_i64(INDEX_op_qemu_st_i64, t, addr_p8, mop[1], idx); + tcg_temp_free_i64(t); + } else { + gen_ldst_i64(INDEX_op_qemu_st_i64, x, addr, mop[0], idx); + tcg_gen_addi_tl(addr_p8, addr, 8); + gen_ldst_i64(INDEX_op_qemu_st_i64, y, addr_p8, mop[1], idx); + } + tcg_temp_free(addr_p8); + } else { + gen_helper_st_i128(cpu_env, addr, val, tcg_constant_i32(oi)); + } + + plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_W); +} + +static void tcg_gen_ext_i32(TCGv_i32 ret, TCGv_i32 val, MemOp opc) +{ + switch (opc & MO_SSIZE) { + case MO_SB: + tcg_gen_ext8s_i32(ret, val); + break; + case MO_UB: + tcg_gen_ext8u_i32(ret, val); + break; + case MO_SW: + tcg_gen_ext16s_i32(ret, val); + break; + case MO_UW: + tcg_gen_ext16u_i32(ret, val); + break; + default: + tcg_gen_mov_i32(ret, val); + break; + } +} + +static void tcg_gen_ext_i64(TCGv_i64 ret, TCGv_i64 val, MemOp opc) +{ + switch (opc & MO_SSIZE) { + case MO_SB: + tcg_gen_ext8s_i64(ret, val); + break; + case MO_UB: + tcg_gen_ext8u_i64(ret, val); + break; + case MO_SW: + tcg_gen_ext16s_i64(ret, val); + break; + case MO_UW: + tcg_gen_ext16u_i64(ret, val); + break; + case MO_SL: + tcg_gen_ext32s_i64(ret, val); + break; + case MO_UL: + tcg_gen_ext32u_i64(ret, val); + break; + default: + tcg_gen_mov_i64(ret, val); + break; + } +} + +typedef void (*gen_atomic_cx_i32)(TCGv_i32, TCGv_env, TCGv, + TCGv_i32, TCGv_i32, TCGv_i32); +typedef void (*gen_atomic_cx_i64)(TCGv_i64, TCGv_env, TCGv, + TCGv_i64, TCGv_i64, TCGv_i32); +typedef void (*gen_atomic_cx_i128)(TCGv_i128, TCGv_env, TCGv, + TCGv_i128, TCGv_i128, TCGv_i32); +typedef void (*gen_atomic_op_i32)(TCGv_i32, TCGv_env, TCGv, + TCGv_i32, TCGv_i32); +typedef void (*gen_atomic_op_i64)(TCGv_i64, TCGv_env, TCGv, + TCGv_i64, TCGv_i32); + +#ifdef CONFIG_ATOMIC64 +# define WITH_ATOMIC64(X) X, +#else +# define WITH_ATOMIC64(X) +#endif +#ifdef CONFIG_CMPXCHG128 +# define WITH_ATOMIC128(X) X, +#else +# define WITH_ATOMIC128(X) +#endif + +static void * const table_cmpxchg[(MO_SIZE | MO_BSWAP) + 1] = { + [MO_8] = gen_helper_atomic_cmpxchgb, + [MO_16 | MO_LE] = gen_helper_atomic_cmpxchgw_le, + [MO_16 | MO_BE] = gen_helper_atomic_cmpxchgw_be, + [MO_32 | MO_LE] = gen_helper_atomic_cmpxchgl_le, + [MO_32 | MO_BE] = gen_helper_atomic_cmpxchgl_be, + WITH_ATOMIC64([MO_64 | MO_LE] = gen_helper_atomic_cmpxchgq_le) + WITH_ATOMIC64([MO_64 | MO_BE] = gen_helper_atomic_cmpxchgq_be) + WITH_ATOMIC128([MO_128 | MO_LE] = gen_helper_atomic_cmpxchgo_le) + WITH_ATOMIC128([MO_128 | MO_BE] = gen_helper_atomic_cmpxchgo_be) +}; + +void tcg_gen_nonatomic_cmpxchg_i32(TCGv_i32 retv, TCGv addr, TCGv_i32 cmpv, + TCGv_i32 newv, TCGArg idx, MemOp memop) +{ + TCGv_i32 t1 = tcg_temp_ebb_new_i32(); + TCGv_i32 t2 = tcg_temp_ebb_new_i32(); + + tcg_gen_ext_i32(t2, cmpv, memop & MO_SIZE); + + tcg_gen_qemu_ld_i32(t1, addr, idx, memop & ~MO_SIGN); + tcg_gen_movcond_i32(TCG_COND_EQ, t2, t1, t2, newv, t1); + tcg_gen_qemu_st_i32(t2, addr, idx, memop); + tcg_temp_free_i32(t2); + + if (memop & MO_SIGN) { + tcg_gen_ext_i32(retv, t1, memop); + } else { + tcg_gen_mov_i32(retv, t1); + } + tcg_temp_free_i32(t1); +} + +void tcg_gen_atomic_cmpxchg_i32(TCGv_i32 retv, TCGv addr, TCGv_i32 cmpv, + TCGv_i32 newv, TCGArg idx, MemOp memop) +{ + gen_atomic_cx_i32 gen; + MemOpIdx oi; + + if (!(tcg_ctx->gen_tb->cflags & CF_PARALLEL)) { + tcg_gen_nonatomic_cmpxchg_i32(retv, addr, cmpv, newv, idx, memop); + return; + } + + memop = tcg_canonicalize_memop(memop, 0, 0); + gen = table_cmpxchg[memop & (MO_SIZE | MO_BSWAP)]; + tcg_debug_assert(gen != NULL); + + oi = make_memop_idx(memop & ~MO_SIGN, idx); + gen(retv, cpu_env, addr, cmpv, newv, tcg_constant_i32(oi)); + + if (memop & MO_SIGN) { + tcg_gen_ext_i32(retv, retv, memop); + } +} + +void tcg_gen_nonatomic_cmpxchg_i64(TCGv_i64 retv, TCGv addr, TCGv_i64 cmpv, + TCGv_i64 newv, TCGArg idx, MemOp memop) +{ + TCGv_i64 t1, t2; + + if (TCG_TARGET_REG_BITS == 32 && (memop & MO_SIZE) < MO_64) { + tcg_gen_nonatomic_cmpxchg_i32(TCGV_LOW(retv), addr, TCGV_LOW(cmpv), + TCGV_LOW(newv), idx, memop); + if (memop & MO_SIGN) { + tcg_gen_sari_i32(TCGV_HIGH(retv), TCGV_LOW(retv), 31); + } else { + tcg_gen_movi_i32(TCGV_HIGH(retv), 0); + } + return; + } + + t1 = tcg_temp_ebb_new_i64(); + t2 = tcg_temp_ebb_new_i64(); + + tcg_gen_ext_i64(t2, cmpv, memop & MO_SIZE); + + tcg_gen_qemu_ld_i64(t1, addr, idx, memop & ~MO_SIGN); + tcg_gen_movcond_i64(TCG_COND_EQ, t2, t1, t2, newv, t1); + tcg_gen_qemu_st_i64(t2, addr, idx, memop); + tcg_temp_free_i64(t2); + + if (memop & MO_SIGN) { + tcg_gen_ext_i64(retv, t1, memop); + } else { + tcg_gen_mov_i64(retv, t1); + } + tcg_temp_free_i64(t1); +} + +void tcg_gen_atomic_cmpxchg_i64(TCGv_i64 retv, TCGv addr, TCGv_i64 cmpv, + TCGv_i64 newv, TCGArg idx, MemOp memop) +{ + if (!(tcg_ctx->gen_tb->cflags & CF_PARALLEL)) { + tcg_gen_nonatomic_cmpxchg_i64(retv, addr, cmpv, newv, idx, memop); + return; + } + + if ((memop & MO_SIZE) == MO_64) { + gen_atomic_cx_i64 gen; + + memop = tcg_canonicalize_memop(memop, 1, 0); + gen = table_cmpxchg[memop & (MO_SIZE | MO_BSWAP)]; + if (gen) { + MemOpIdx oi = make_memop_idx(memop, idx); + gen(retv, cpu_env, addr, cmpv, newv, tcg_constant_i32(oi)); + return; + } + + gen_helper_exit_atomic(cpu_env); + + /* + * Produce a result for a well-formed opcode stream. This satisfies + * liveness for set before used, which happens before this dead code + * is removed. + */ + tcg_gen_movi_i64(retv, 0); + return; + } + + if (TCG_TARGET_REG_BITS == 32) { + tcg_gen_atomic_cmpxchg_i32(TCGV_LOW(retv), addr, TCGV_LOW(cmpv), + TCGV_LOW(newv), idx, memop); + if (memop & MO_SIGN) { + tcg_gen_sari_i32(TCGV_HIGH(retv), TCGV_LOW(retv), 31); + } else { + tcg_gen_movi_i32(TCGV_HIGH(retv), 0); + } + } else { + TCGv_i32 c32 = tcg_temp_ebb_new_i32(); + TCGv_i32 n32 = tcg_temp_ebb_new_i32(); + TCGv_i32 r32 = tcg_temp_ebb_new_i32(); + + tcg_gen_extrl_i64_i32(c32, cmpv); + tcg_gen_extrl_i64_i32(n32, newv); + tcg_gen_atomic_cmpxchg_i32(r32, addr, c32, n32, idx, memop & ~MO_SIGN); + tcg_temp_free_i32(c32); + tcg_temp_free_i32(n32); + + tcg_gen_extu_i32_i64(retv, r32); + tcg_temp_free_i32(r32); + + if (memop & MO_SIGN) { + tcg_gen_ext_i64(retv, retv, memop); + } + } +} + +void tcg_gen_nonatomic_cmpxchg_i128(TCGv_i128 retv, TCGv addr, TCGv_i128 cmpv, + TCGv_i128 newv, TCGArg idx, MemOp memop) +{ + if (TCG_TARGET_REG_BITS == 32) { + /* Inline expansion below is simply too large for 32-bit hosts. */ + gen_atomic_cx_i128 gen = ((memop & MO_BSWAP) == MO_LE + ? gen_helper_nonatomic_cmpxchgo_le + : gen_helper_nonatomic_cmpxchgo_be); + MemOpIdx oi = make_memop_idx(memop, idx); + + tcg_debug_assert((memop & MO_SIZE) == MO_128); + tcg_debug_assert((memop & MO_SIGN) == 0); + + gen(retv, cpu_env, addr, cmpv, newv, tcg_constant_i32(oi)); + } else { + TCGv_i128 oldv = tcg_temp_ebb_new_i128(); + TCGv_i128 tmpv = tcg_temp_ebb_new_i128(); + TCGv_i64 t0 = tcg_temp_ebb_new_i64(); + TCGv_i64 t1 = tcg_temp_ebb_new_i64(); + TCGv_i64 z = tcg_constant_i64(0); + + tcg_gen_qemu_ld_i128(oldv, addr, idx, memop); + + /* Compare i128 */ + tcg_gen_xor_i64(t0, TCGV128_LOW(oldv), TCGV128_LOW(cmpv)); + tcg_gen_xor_i64(t1, TCGV128_HIGH(oldv), TCGV128_HIGH(cmpv)); + tcg_gen_or_i64(t0, t0, t1); + + /* tmpv = equal ? newv : oldv */ + tcg_gen_movcond_i64(TCG_COND_EQ, TCGV128_LOW(tmpv), t0, z, + TCGV128_LOW(newv), TCGV128_LOW(oldv)); + tcg_gen_movcond_i64(TCG_COND_EQ, TCGV128_HIGH(tmpv), t0, z, + TCGV128_HIGH(newv), TCGV128_HIGH(oldv)); + + /* Unconditional writeback. */ + tcg_gen_qemu_st_i128(tmpv, addr, idx, memop); + tcg_gen_mov_i128(retv, oldv); + + tcg_temp_free_i64(t0); + tcg_temp_free_i64(t1); + tcg_temp_free_i128(tmpv); + tcg_temp_free_i128(oldv); + } +} + +void tcg_gen_atomic_cmpxchg_i128(TCGv_i128 retv, TCGv addr, TCGv_i128 cmpv, + TCGv_i128 newv, TCGArg idx, MemOp memop) +{ + gen_atomic_cx_i128 gen; + + if (!(tcg_ctx->gen_tb->cflags & CF_PARALLEL)) { + tcg_gen_nonatomic_cmpxchg_i128(retv, addr, cmpv, newv, idx, memop); + return; + } + + tcg_debug_assert((memop & MO_SIZE) == MO_128); + tcg_debug_assert((memop & MO_SIGN) == 0); + gen = table_cmpxchg[memop & (MO_SIZE | MO_BSWAP)]; + + if (gen) { + MemOpIdx oi = make_memop_idx(memop, idx); + gen(retv, cpu_env, addr, cmpv, newv, tcg_constant_i32(oi)); + return; + } + + gen_helper_exit_atomic(cpu_env); + + /* + * Produce a result for a well-formed opcode stream. This satisfies + * liveness for set before used, which happens before this dead code + * is removed. + */ + tcg_gen_movi_i64(TCGV128_LOW(retv), 0); + tcg_gen_movi_i64(TCGV128_HIGH(retv), 0); +} + +static void do_nonatomic_op_i32(TCGv_i32 ret, TCGv addr, TCGv_i32 val, + TCGArg idx, MemOp memop, bool new_val, + void (*gen)(TCGv_i32, TCGv_i32, TCGv_i32)) +{ + TCGv_i32 t1 = tcg_temp_ebb_new_i32(); + TCGv_i32 t2 = tcg_temp_ebb_new_i32(); + + memop = tcg_canonicalize_memop(memop, 0, 0); + + tcg_gen_qemu_ld_i32(t1, addr, idx, memop); + tcg_gen_ext_i32(t2, val, memop); + gen(t2, t1, t2); + tcg_gen_qemu_st_i32(t2, addr, idx, memop); + + tcg_gen_ext_i32(ret, (new_val ? t2 : t1), memop); + tcg_temp_free_i32(t1); + tcg_temp_free_i32(t2); +} + +static void do_atomic_op_i32(TCGv_i32 ret, TCGv addr, TCGv_i32 val, + TCGArg idx, MemOp memop, void * const table[]) +{ + gen_atomic_op_i32 gen; + MemOpIdx oi; + + memop = tcg_canonicalize_memop(memop, 0, 0); + + gen = table[memop & (MO_SIZE | MO_BSWAP)]; + tcg_debug_assert(gen != NULL); + + oi = make_memop_idx(memop & ~MO_SIGN, idx); + gen(ret, cpu_env, addr, val, tcg_constant_i32(oi)); + + if (memop & MO_SIGN) { + tcg_gen_ext_i32(ret, ret, memop); + } +} + +static void do_nonatomic_op_i64(TCGv_i64 ret, TCGv addr, TCGv_i64 val, + TCGArg idx, MemOp memop, bool new_val, + void (*gen)(TCGv_i64, TCGv_i64, TCGv_i64)) +{ + TCGv_i64 t1 = tcg_temp_ebb_new_i64(); + TCGv_i64 t2 = tcg_temp_ebb_new_i64(); + + memop = tcg_canonicalize_memop(memop, 1, 0); + + tcg_gen_qemu_ld_i64(t1, addr, idx, memop); + tcg_gen_ext_i64(t2, val, memop); + gen(t2, t1, t2); + tcg_gen_qemu_st_i64(t2, addr, idx, memop); + + tcg_gen_ext_i64(ret, (new_val ? t2 : t1), memop); + tcg_temp_free_i64(t1); + tcg_temp_free_i64(t2); +} + +static void do_atomic_op_i64(TCGv_i64 ret, TCGv addr, TCGv_i64 val, + TCGArg idx, MemOp memop, void * const table[]) +{ + memop = tcg_canonicalize_memop(memop, 1, 0); + + if ((memop & MO_SIZE) == MO_64) { +#ifdef CONFIG_ATOMIC64 + gen_atomic_op_i64 gen; + MemOpIdx oi; + + gen = table[memop & (MO_SIZE | MO_BSWAP)]; + tcg_debug_assert(gen != NULL); + + oi = make_memop_idx(memop & ~MO_SIGN, idx); + gen(ret, cpu_env, addr, val, tcg_constant_i32(oi)); +#else + gen_helper_exit_atomic(cpu_env); + /* Produce a result, so that we have a well-formed opcode stream + with respect to uses of the result in the (dead) code following. */ + tcg_gen_movi_i64(ret, 0); +#endif /* CONFIG_ATOMIC64 */ + } else { + TCGv_i32 v32 = tcg_temp_ebb_new_i32(); + TCGv_i32 r32 = tcg_temp_ebb_new_i32(); + + tcg_gen_extrl_i64_i32(v32, val); + do_atomic_op_i32(r32, addr, v32, idx, memop & ~MO_SIGN, table); + tcg_temp_free_i32(v32); + + tcg_gen_extu_i32_i64(ret, r32); + tcg_temp_free_i32(r32); + + if (memop & MO_SIGN) { + tcg_gen_ext_i64(ret, ret, memop); + } + } +} + +#define GEN_ATOMIC_HELPER(NAME, OP, NEW) \ +static void * const table_##NAME[(MO_SIZE | MO_BSWAP) + 1] = { \ + [MO_8] = gen_helper_atomic_##NAME##b, \ + [MO_16 | MO_LE] = gen_helper_atomic_##NAME##w_le, \ + [MO_16 | MO_BE] = gen_helper_atomic_##NAME##w_be, \ + [MO_32 | MO_LE] = gen_helper_atomic_##NAME##l_le, \ + [MO_32 | MO_BE] = gen_helper_atomic_##NAME##l_be, \ + WITH_ATOMIC64([MO_64 | MO_LE] = gen_helper_atomic_##NAME##q_le) \ + WITH_ATOMIC64([MO_64 | MO_BE] = gen_helper_atomic_##NAME##q_be) \ +}; \ +void tcg_gen_atomic_##NAME##_i32 \ + (TCGv_i32 ret, TCGv addr, TCGv_i32 val, TCGArg idx, MemOp memop) \ +{ \ + if (tcg_ctx->gen_tb->cflags & CF_PARALLEL) { \ + do_atomic_op_i32(ret, addr, val, idx, memop, table_##NAME); \ + } else { \ + do_nonatomic_op_i32(ret, addr, val, idx, memop, NEW, \ + tcg_gen_##OP##_i32); \ + } \ +} \ +void tcg_gen_atomic_##NAME##_i64 \ + (TCGv_i64 ret, TCGv addr, TCGv_i64 val, TCGArg idx, MemOp memop) \ +{ \ + if (tcg_ctx->gen_tb->cflags & CF_PARALLEL) { \ + do_atomic_op_i64(ret, addr, val, idx, memop, table_##NAME); \ + } else { \ + do_nonatomic_op_i64(ret, addr, val, idx, memop, NEW, \ + tcg_gen_##OP##_i64); \ + } \ +} + +GEN_ATOMIC_HELPER(fetch_add, add, 0) +GEN_ATOMIC_HELPER(fetch_and, and, 0) +GEN_ATOMIC_HELPER(fetch_or, or, 0) +GEN_ATOMIC_HELPER(fetch_xor, xor, 0) +GEN_ATOMIC_HELPER(fetch_smin, smin, 0) +GEN_ATOMIC_HELPER(fetch_umin, umin, 0) +GEN_ATOMIC_HELPER(fetch_smax, smax, 0) +GEN_ATOMIC_HELPER(fetch_umax, umax, 0) + +GEN_ATOMIC_HELPER(add_fetch, add, 1) +GEN_ATOMIC_HELPER(and_fetch, and, 1) +GEN_ATOMIC_HELPER(or_fetch, or, 1) +GEN_ATOMIC_HELPER(xor_fetch, xor, 1) +GEN_ATOMIC_HELPER(smin_fetch, smin, 1) +GEN_ATOMIC_HELPER(umin_fetch, umin, 1) +GEN_ATOMIC_HELPER(smax_fetch, smax, 1) +GEN_ATOMIC_HELPER(umax_fetch, umax, 1) + +static void tcg_gen_mov2_i32(TCGv_i32 r, TCGv_i32 a, TCGv_i32 b) +{ + tcg_gen_mov_i32(r, b); +} + +static void tcg_gen_mov2_i64(TCGv_i64 r, TCGv_i64 a, TCGv_i64 b) +{ + tcg_gen_mov_i64(r, b); +} + +GEN_ATOMIC_HELPER(xchg, mov2, 0) + +#undef GEN_ATOMIC_HELPER diff --git a/tcg/tcg-op.c b/tcg/tcg-op.c index c419228cc4..edbd1c61d7 100644 --- a/tcg/tcg-op.c +++ b/tcg/tcg-op.c @@ -27,7 +27,6 @@ #include "tcg/tcg.h" #include "tcg/tcg-temp-internal.h" #include "tcg/tcg-op.h" -#include "tcg/tcg-mo.h" #include "exec/plugin-gen.h" #include "tcg-internal.h" @@ -2841,976 +2840,3 @@ void tcg_gen_lookup_and_goto_ptr(void) tcg_gen_op1i(INDEX_op_goto_ptr, tcgv_ptr_arg(ptr)); tcg_temp_free_ptr(ptr); } - -static inline MemOp tcg_canonicalize_memop(MemOp op, bool is64, bool st) -{ - /* Trigger the asserts within as early as possible. */ - unsigned a_bits = get_alignment_bits(op); - - /* Prefer MO_ALIGN+MO_XX over MO_ALIGN_XX+MO_XX */ - if (a_bits == (op & MO_SIZE)) { - op = (op & ~MO_AMASK) | MO_ALIGN; - } - - switch (op & MO_SIZE) { - case MO_8: - op &= ~MO_BSWAP; - break; - case MO_16: - break; - case MO_32: - if (!is64) { - op &= ~MO_SIGN; - } - break; - case MO_64: - if (is64) { - op &= ~MO_SIGN; - break; - } - /* fall through */ - default: - g_assert_not_reached(); - } - if (st) { - op &= ~MO_SIGN; - } - return op; -} - -static void gen_ldst_i32(TCGOpcode opc, TCGv_i32 val, TCGv addr, - MemOp memop, TCGArg idx) -{ - MemOpIdx oi = make_memop_idx(memop, idx); -#if TARGET_LONG_BITS == 32 - tcg_gen_op3i_i32(opc, val, addr, oi); -#else - if (TCG_TARGET_REG_BITS == 32) { - tcg_gen_op4i_i32(opc, val, TCGV_LOW(addr), TCGV_HIGH(addr), oi); - } else { - tcg_gen_op3(opc, tcgv_i32_arg(val), tcgv_i64_arg(addr), oi); - } -#endif -} - -static void gen_ldst_i64(TCGOpcode opc, TCGv_i64 val, TCGv addr, - MemOp memop, TCGArg idx) -{ - MemOpIdx oi = make_memop_idx(memop, idx); -#if TARGET_LONG_BITS == 32 - if (TCG_TARGET_REG_BITS == 32) { - tcg_gen_op4i_i32(opc, TCGV_LOW(val), TCGV_HIGH(val), addr, oi); - } else { - tcg_gen_op3(opc, tcgv_i64_arg(val), tcgv_i32_arg(addr), oi); - } -#else - if (TCG_TARGET_REG_BITS == 32) { - tcg_gen_op5i_i32(opc, TCGV_LOW(val), TCGV_HIGH(val), - TCGV_LOW(addr), TCGV_HIGH(addr), oi); - } else { - tcg_gen_op3i_i64(opc, val, addr, oi); - } -#endif -} - -static void tcg_gen_req_mo(TCGBar type) -{ -#ifdef TCG_GUEST_DEFAULT_MO - type &= TCG_GUEST_DEFAULT_MO; -#endif - type &= ~TCG_TARGET_DEFAULT_MO; - if (type) { - tcg_gen_mb(type | TCG_BAR_SC); - } -} - -static inline TCGv plugin_prep_mem_callbacks(TCGv vaddr) -{ -#ifdef CONFIG_PLUGIN - if (tcg_ctx->plugin_insn != NULL) { - /* Save a copy of the vaddr for use after a load. */ - TCGv temp = tcg_temp_new(); - tcg_gen_mov_tl(temp, vaddr); - return temp; - } -#endif - return vaddr; -} - -static void plugin_gen_mem_callbacks(TCGv vaddr, MemOpIdx oi, - enum qemu_plugin_mem_rw rw) -{ -#ifdef CONFIG_PLUGIN - if (tcg_ctx->plugin_insn != NULL) { - qemu_plugin_meminfo_t info = make_plugin_meminfo(oi, rw); - plugin_gen_empty_mem_callback(vaddr, info); - tcg_temp_free(vaddr); - } -#endif -} - -void tcg_gen_qemu_ld_i32(TCGv_i32 val, TCGv addr, TCGArg idx, MemOp memop) -{ - MemOp orig_memop; - MemOpIdx oi; - - tcg_gen_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); - memop = tcg_canonicalize_memop(memop, 0, 0); - oi = make_memop_idx(memop, idx); - - orig_memop = memop; - if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { - memop &= ~MO_BSWAP; - /* The bswap primitive benefits from zero-extended input. */ - if ((memop & MO_SSIZE) == MO_SW) { - memop &= ~MO_SIGN; - } - } - - addr = plugin_prep_mem_callbacks(addr); - gen_ldst_i32(INDEX_op_qemu_ld_i32, val, addr, memop, idx); - plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_R); - - if ((orig_memop ^ memop) & MO_BSWAP) { - switch (orig_memop & MO_SIZE) { - case MO_16: - tcg_gen_bswap16_i32(val, val, (orig_memop & MO_SIGN - ? TCG_BSWAP_IZ | TCG_BSWAP_OS - : TCG_BSWAP_IZ | TCG_BSWAP_OZ)); - break; - case MO_32: - tcg_gen_bswap32_i32(val, val); - break; - default: - g_assert_not_reached(); - } - } -} - -void tcg_gen_qemu_st_i32(TCGv_i32 val, TCGv addr, TCGArg idx, MemOp memop) -{ - TCGv_i32 swap = NULL; - MemOpIdx oi; - - tcg_gen_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST); - memop = tcg_canonicalize_memop(memop, 0, 1); - oi = make_memop_idx(memop, idx); - - if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { - swap = tcg_temp_ebb_new_i32(); - switch (memop & MO_SIZE) { - case MO_16: - tcg_gen_bswap16_i32(swap, val, 0); - break; - case MO_32: - tcg_gen_bswap32_i32(swap, val); - break; - default: - g_assert_not_reached(); - } - val = swap; - memop &= ~MO_BSWAP; - } - - addr = plugin_prep_mem_callbacks(addr); - if (TCG_TARGET_HAS_qemu_st8_i32 && (memop & MO_SIZE) == MO_8) { - gen_ldst_i32(INDEX_op_qemu_st8_i32, val, addr, memop, idx); - } else { - gen_ldst_i32(INDEX_op_qemu_st_i32, val, addr, memop, idx); - } - plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_W); - - if (swap) { - tcg_temp_free_i32(swap); - } -} - -void tcg_gen_qemu_ld_i64(TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop) -{ - MemOp orig_memop; - MemOpIdx oi; - - if (TCG_TARGET_REG_BITS == 32 && (memop & MO_SIZE) < MO_64) { - tcg_gen_qemu_ld_i32(TCGV_LOW(val), addr, idx, memop); - if (memop & MO_SIGN) { - tcg_gen_sari_i32(TCGV_HIGH(val), TCGV_LOW(val), 31); - } else { - tcg_gen_movi_i32(TCGV_HIGH(val), 0); - } - return; - } - - tcg_gen_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); - memop = tcg_canonicalize_memop(memop, 1, 0); - oi = make_memop_idx(memop, idx); - - orig_memop = memop; - if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { - memop &= ~MO_BSWAP; - /* The bswap primitive benefits from zero-extended input. */ - if ((memop & MO_SIGN) && (memop & MO_SIZE) < MO_64) { - memop &= ~MO_SIGN; - } - } - - addr = plugin_prep_mem_callbacks(addr); - gen_ldst_i64(INDEX_op_qemu_ld_i64, val, addr, memop, idx); - plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_R); - - if ((orig_memop ^ memop) & MO_BSWAP) { - int flags = (orig_memop & MO_SIGN - ? TCG_BSWAP_IZ | TCG_BSWAP_OS - : TCG_BSWAP_IZ | TCG_BSWAP_OZ); - switch (orig_memop & MO_SIZE) { - case MO_16: - tcg_gen_bswap16_i64(val, val, flags); - break; - case MO_32: - tcg_gen_bswap32_i64(val, val, flags); - break; - case MO_64: - tcg_gen_bswap64_i64(val, val); - break; - default: - g_assert_not_reached(); - } - } -} - -void tcg_gen_qemu_st_i64(TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop) -{ - TCGv_i64 swap = NULL; - MemOpIdx oi; - - if (TCG_TARGET_REG_BITS == 32 && (memop & MO_SIZE) < MO_64) { - tcg_gen_qemu_st_i32(TCGV_LOW(val), addr, idx, memop); - return; - } - - tcg_gen_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST); - memop = tcg_canonicalize_memop(memop, 1, 1); - oi = make_memop_idx(memop, idx); - - if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { - swap = tcg_temp_ebb_new_i64(); - switch (memop & MO_SIZE) { - case MO_16: - tcg_gen_bswap16_i64(swap, val, 0); - break; - case MO_32: - tcg_gen_bswap32_i64(swap, val, 0); - break; - case MO_64: - tcg_gen_bswap64_i64(swap, val); - break; - default: - g_assert_not_reached(); - } - val = swap; - memop &= ~MO_BSWAP; - } - - addr = plugin_prep_mem_callbacks(addr); - gen_ldst_i64(INDEX_op_qemu_st_i64, val, addr, memop, idx); - plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_W); - - if (swap) { - tcg_temp_free_i64(swap); - } -} - -/* - * Return true if @mop, without knowledge of the pointer alignment, - * does not require 16-byte atomicity, and it would be adventagous - * to avoid a call to a helper function. - */ -static bool use_two_i64_for_i128(MemOp mop) -{ -#ifdef CONFIG_SOFTMMU - /* Two softmmu tlb lookups is larger than one function call. */ - return false; -#else - /* - * For user-only, two 64-bit operations may well be smaller than a call. - * Determine if that would be legal for the requested atomicity. - */ - switch (mop & MO_ATOM_MASK) { - case MO_ATOM_NONE: - case MO_ATOM_IFALIGN_PAIR: - return true; - case MO_ATOM_IFALIGN: - case MO_ATOM_SUBALIGN: - case MO_ATOM_WITHIN16: - case MO_ATOM_WITHIN16_PAIR: - /* In a serialized context, no atomicity is required. */ - return !(tcg_ctx->gen_tb->cflags & CF_PARALLEL); - default: - g_assert_not_reached(); - } -#endif -} - -static void canonicalize_memop_i128_as_i64(MemOp ret[2], MemOp orig) -{ - MemOp mop_1 = orig, mop_2; - - tcg_debug_assert((orig & MO_SIZE) == MO_128); - tcg_debug_assert((orig & MO_SIGN) == 0); - - /* Reduce the size to 64-bit. */ - mop_1 = (mop_1 & ~MO_SIZE) | MO_64; - - /* Retain the alignment constraints of the original. */ - switch (orig & MO_AMASK) { - case MO_UNALN: - case MO_ALIGN_2: - case MO_ALIGN_4: - mop_2 = mop_1; - break; - case MO_ALIGN_8: - /* Prefer MO_ALIGN+MO_64 to MO_ALIGN_8+MO_64. */ - mop_1 = (mop_1 & ~MO_AMASK) | MO_ALIGN; - mop_2 = mop_1; - break; - case MO_ALIGN: - /* Second has 8-byte alignment; first has 16-byte alignment. */ - mop_2 = mop_1; - mop_1 = (mop_1 & ~MO_AMASK) | MO_ALIGN_16; - break; - case MO_ALIGN_16: - case MO_ALIGN_32: - case MO_ALIGN_64: - /* Second has 8-byte alignment; first retains original. */ - mop_2 = (mop_1 & ~MO_AMASK) | MO_ALIGN; - break; - default: - g_assert_not_reached(); - } - - /* Use a memory ordering implemented by the host. */ - if ((orig & MO_BSWAP) && !tcg_target_has_memory_bswap(mop_1)) { - mop_1 &= ~MO_BSWAP; - mop_2 &= ~MO_BSWAP; - } - - ret[0] = mop_1; - ret[1] = mop_2; -} - -#if TARGET_LONG_BITS == 64 -#define tcg_temp_ebb_new tcg_temp_ebb_new_i64 -#else -#define tcg_temp_ebb_new tcg_temp_ebb_new_i32 -#endif - -void tcg_gen_qemu_ld_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) -{ - const MemOpIdx oi = make_memop_idx(memop, idx); - - tcg_debug_assert((memop & MO_SIZE) == MO_128); - tcg_debug_assert((memop & MO_SIGN) == 0); - - tcg_gen_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); - addr = plugin_prep_mem_callbacks(addr); - - /* TODO: For now, force 32-bit hosts to use the helper. */ - if (TCG_TARGET_HAS_qemu_ldst_i128 && TCG_TARGET_REG_BITS == 64) { - TCGv_i64 lo, hi; - TCGArg addr_arg; - MemOpIdx adj_oi; - bool need_bswap = false; - - if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { - lo = TCGV128_HIGH(val); - hi = TCGV128_LOW(val); - adj_oi = make_memop_idx(memop & ~MO_BSWAP, idx); - need_bswap = true; - } else { - lo = TCGV128_LOW(val); - hi = TCGV128_HIGH(val); - adj_oi = oi; - } - -#if TARGET_LONG_BITS == 32 - addr_arg = tcgv_i32_arg(addr); -#else - addr_arg = tcgv_i64_arg(addr); -#endif - tcg_gen_op4ii_i64(INDEX_op_qemu_ld_i128, lo, hi, addr_arg, adj_oi); - - if (need_bswap) { - tcg_gen_bswap64_i64(lo, lo); - tcg_gen_bswap64_i64(hi, hi); - } - } else if (use_two_i64_for_i128(memop)) { - MemOp mop[2]; - TCGv addr_p8; - TCGv_i64 x, y; - - canonicalize_memop_i128_as_i64(mop, memop); - - /* - * Since there are no global TCGv_i128, there is no visible state - * changed if the second load faults. Load directly into the two - * subwords. - */ - if ((memop & MO_BSWAP) == MO_LE) { - x = TCGV128_LOW(val); - y = TCGV128_HIGH(val); - } else { - x = TCGV128_HIGH(val); - y = TCGV128_LOW(val); - } - - gen_ldst_i64(INDEX_op_qemu_ld_i64, x, addr, mop[0], idx); - - if ((mop[0] ^ memop) & MO_BSWAP) { - tcg_gen_bswap64_i64(x, x); - } - - addr_p8 = tcg_temp_ebb_new(); - tcg_gen_addi_tl(addr_p8, addr, 8); - gen_ldst_i64(INDEX_op_qemu_ld_i64, y, addr_p8, mop[1], idx); - tcg_temp_free(addr_p8); - - if ((mop[0] ^ memop) & MO_BSWAP) { - tcg_gen_bswap64_i64(y, y); - } - } else { - gen_helper_ld_i128(val, cpu_env, addr, tcg_constant_i32(oi)); - } - - plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_R); -} - -void tcg_gen_qemu_st_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) -{ - const MemOpIdx oi = make_memop_idx(memop, idx); - - tcg_debug_assert((memop & MO_SIZE) == MO_128); - tcg_debug_assert((memop & MO_SIGN) == 0); - - tcg_gen_req_mo(TCG_MO_ST_LD | TCG_MO_ST_ST); - addr = plugin_prep_mem_callbacks(addr); - - /* TODO: For now, force 32-bit hosts to use the helper. */ - - if (TCG_TARGET_HAS_qemu_ldst_i128 && TCG_TARGET_REG_BITS == 64) { - TCGv_i64 lo, hi; - TCGArg addr_arg; - MemOpIdx adj_oi; - bool need_bswap = false; - - if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { - lo = tcg_temp_new_i64(); - hi = tcg_temp_new_i64(); - tcg_gen_bswap64_i64(lo, TCGV128_HIGH(val)); - tcg_gen_bswap64_i64(hi, TCGV128_LOW(val)); - adj_oi = make_memop_idx(memop & ~MO_BSWAP, idx); - need_bswap = true; - } else { - lo = TCGV128_LOW(val); - hi = TCGV128_HIGH(val); - adj_oi = oi; - } - -#if TARGET_LONG_BITS == 32 - addr_arg = tcgv_i32_arg(addr); -#else - addr_arg = tcgv_i64_arg(addr); -#endif - tcg_gen_op4ii_i64(INDEX_op_qemu_st_i128, lo, hi, addr_arg, adj_oi); - - if (need_bswap) { - tcg_temp_free_i64(lo); - tcg_temp_free_i64(hi); - } - } else if (use_two_i64_for_i128(memop)) { - MemOp mop[2]; - TCGv addr_p8; - TCGv_i64 x, y; - - canonicalize_memop_i128_as_i64(mop, memop); - - if ((memop & MO_BSWAP) == MO_LE) { - x = TCGV128_LOW(val); - y = TCGV128_HIGH(val); - } else { - x = TCGV128_HIGH(val); - y = TCGV128_LOW(val); - } - - addr_p8 = tcg_temp_ebb_new(); - if ((mop[0] ^ memop) & MO_BSWAP) { - TCGv_i64 t = tcg_temp_ebb_new_i64(); - - tcg_gen_bswap64_i64(t, x); - gen_ldst_i64(INDEX_op_qemu_st_i64, t, addr, mop[0], idx); - tcg_gen_bswap64_i64(t, y); - tcg_gen_addi_tl(addr_p8, addr, 8); - gen_ldst_i64(INDEX_op_qemu_st_i64, t, addr_p8, mop[1], idx); - tcg_temp_free_i64(t); - } else { - gen_ldst_i64(INDEX_op_qemu_st_i64, x, addr, mop[0], idx); - tcg_gen_addi_tl(addr_p8, addr, 8); - gen_ldst_i64(INDEX_op_qemu_st_i64, y, addr_p8, mop[1], idx); - } - tcg_temp_free(addr_p8); - } else { - gen_helper_st_i128(cpu_env, addr, val, tcg_constant_i32(oi)); - } - - plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_W); -} - -static void tcg_gen_ext_i32(TCGv_i32 ret, TCGv_i32 val, MemOp opc) -{ - switch (opc & MO_SSIZE) { - case MO_SB: - tcg_gen_ext8s_i32(ret, val); - break; - case MO_UB: - tcg_gen_ext8u_i32(ret, val); - break; - case MO_SW: - tcg_gen_ext16s_i32(ret, val); - break; - case MO_UW: - tcg_gen_ext16u_i32(ret, val); - break; - default: - tcg_gen_mov_i32(ret, val); - break; - } -} - -static void tcg_gen_ext_i64(TCGv_i64 ret, TCGv_i64 val, MemOp opc) -{ - switch (opc & MO_SSIZE) { - case MO_SB: - tcg_gen_ext8s_i64(ret, val); - break; - case MO_UB: - tcg_gen_ext8u_i64(ret, val); - break; - case MO_SW: - tcg_gen_ext16s_i64(ret, val); - break; - case MO_UW: - tcg_gen_ext16u_i64(ret, val); - break; - case MO_SL: - tcg_gen_ext32s_i64(ret, val); - break; - case MO_UL: - tcg_gen_ext32u_i64(ret, val); - break; - default: - tcg_gen_mov_i64(ret, val); - break; - } -} - -typedef void (*gen_atomic_cx_i32)(TCGv_i32, TCGv_env, TCGv, - TCGv_i32, TCGv_i32, TCGv_i32); -typedef void (*gen_atomic_cx_i64)(TCGv_i64, TCGv_env, TCGv, - TCGv_i64, TCGv_i64, TCGv_i32); -typedef void (*gen_atomic_cx_i128)(TCGv_i128, TCGv_env, TCGv, - TCGv_i128, TCGv_i128, TCGv_i32); -typedef void (*gen_atomic_op_i32)(TCGv_i32, TCGv_env, TCGv, - TCGv_i32, TCGv_i32); -typedef void (*gen_atomic_op_i64)(TCGv_i64, TCGv_env, TCGv, - TCGv_i64, TCGv_i32); - -#ifdef CONFIG_ATOMIC64 -# define WITH_ATOMIC64(X) X, -#else -# define WITH_ATOMIC64(X) -#endif -#ifdef CONFIG_CMPXCHG128 -# define WITH_ATOMIC128(X) X, -#else -# define WITH_ATOMIC128(X) -#endif - -static void * const table_cmpxchg[(MO_SIZE | MO_BSWAP) + 1] = { - [MO_8] = gen_helper_atomic_cmpxchgb, - [MO_16 | MO_LE] = gen_helper_atomic_cmpxchgw_le, - [MO_16 | MO_BE] = gen_helper_atomic_cmpxchgw_be, - [MO_32 | MO_LE] = gen_helper_atomic_cmpxchgl_le, - [MO_32 | MO_BE] = gen_helper_atomic_cmpxchgl_be, - WITH_ATOMIC64([MO_64 | MO_LE] = gen_helper_atomic_cmpxchgq_le) - WITH_ATOMIC64([MO_64 | MO_BE] = gen_helper_atomic_cmpxchgq_be) - WITH_ATOMIC128([MO_128 | MO_LE] = gen_helper_atomic_cmpxchgo_le) - WITH_ATOMIC128([MO_128 | MO_BE] = gen_helper_atomic_cmpxchgo_be) -}; - -void tcg_gen_nonatomic_cmpxchg_i32(TCGv_i32 retv, TCGv addr, TCGv_i32 cmpv, - TCGv_i32 newv, TCGArg idx, MemOp memop) -{ - TCGv_i32 t1 = tcg_temp_ebb_new_i32(); - TCGv_i32 t2 = tcg_temp_ebb_new_i32(); - - tcg_gen_ext_i32(t2, cmpv, memop & MO_SIZE); - - tcg_gen_qemu_ld_i32(t1, addr, idx, memop & ~MO_SIGN); - tcg_gen_movcond_i32(TCG_COND_EQ, t2, t1, t2, newv, t1); - tcg_gen_qemu_st_i32(t2, addr, idx, memop); - tcg_temp_free_i32(t2); - - if (memop & MO_SIGN) { - tcg_gen_ext_i32(retv, t1, memop); - } else { - tcg_gen_mov_i32(retv, t1); - } - tcg_temp_free_i32(t1); -} - -void tcg_gen_atomic_cmpxchg_i32(TCGv_i32 retv, TCGv addr, TCGv_i32 cmpv, - TCGv_i32 newv, TCGArg idx, MemOp memop) -{ - gen_atomic_cx_i32 gen; - MemOpIdx oi; - - if (!(tcg_ctx->gen_tb->cflags & CF_PARALLEL)) { - tcg_gen_nonatomic_cmpxchg_i32(retv, addr, cmpv, newv, idx, memop); - return; - } - - memop = tcg_canonicalize_memop(memop, 0, 0); - gen = table_cmpxchg[memop & (MO_SIZE | MO_BSWAP)]; - tcg_debug_assert(gen != NULL); - - oi = make_memop_idx(memop & ~MO_SIGN, idx); - gen(retv, cpu_env, addr, cmpv, newv, tcg_constant_i32(oi)); - - if (memop & MO_SIGN) { - tcg_gen_ext_i32(retv, retv, memop); - } -} - -void tcg_gen_nonatomic_cmpxchg_i64(TCGv_i64 retv, TCGv addr, TCGv_i64 cmpv, - TCGv_i64 newv, TCGArg idx, MemOp memop) -{ - TCGv_i64 t1, t2; - - if (TCG_TARGET_REG_BITS == 32 && (memop & MO_SIZE) < MO_64) { - tcg_gen_nonatomic_cmpxchg_i32(TCGV_LOW(retv), addr, TCGV_LOW(cmpv), - TCGV_LOW(newv), idx, memop); - if (memop & MO_SIGN) { - tcg_gen_sari_i32(TCGV_HIGH(retv), TCGV_LOW(retv), 31); - } else { - tcg_gen_movi_i32(TCGV_HIGH(retv), 0); - } - return; - } - - t1 = tcg_temp_ebb_new_i64(); - t2 = tcg_temp_ebb_new_i64(); - - tcg_gen_ext_i64(t2, cmpv, memop & MO_SIZE); - - tcg_gen_qemu_ld_i64(t1, addr, idx, memop & ~MO_SIGN); - tcg_gen_movcond_i64(TCG_COND_EQ, t2, t1, t2, newv, t1); - tcg_gen_qemu_st_i64(t2, addr, idx, memop); - tcg_temp_free_i64(t2); - - if (memop & MO_SIGN) { - tcg_gen_ext_i64(retv, t1, memop); - } else { - tcg_gen_mov_i64(retv, t1); - } - tcg_temp_free_i64(t1); -} - -void tcg_gen_atomic_cmpxchg_i64(TCGv_i64 retv, TCGv addr, TCGv_i64 cmpv, - TCGv_i64 newv, TCGArg idx, MemOp memop) -{ - if (!(tcg_ctx->gen_tb->cflags & CF_PARALLEL)) { - tcg_gen_nonatomic_cmpxchg_i64(retv, addr, cmpv, newv, idx, memop); - return; - } - - if ((memop & MO_SIZE) == MO_64) { - gen_atomic_cx_i64 gen; - - memop = tcg_canonicalize_memop(memop, 1, 0); - gen = table_cmpxchg[memop & (MO_SIZE | MO_BSWAP)]; - if (gen) { - MemOpIdx oi = make_memop_idx(memop, idx); - gen(retv, cpu_env, addr, cmpv, newv, tcg_constant_i32(oi)); - return; - } - - gen_helper_exit_atomic(cpu_env); - - /* - * Produce a result for a well-formed opcode stream. This satisfies - * liveness for set before used, which happens before this dead code - * is removed. - */ - tcg_gen_movi_i64(retv, 0); - return; - } - - if (TCG_TARGET_REG_BITS == 32) { - tcg_gen_atomic_cmpxchg_i32(TCGV_LOW(retv), addr, TCGV_LOW(cmpv), - TCGV_LOW(newv), idx, memop); - if (memop & MO_SIGN) { - tcg_gen_sari_i32(TCGV_HIGH(retv), TCGV_LOW(retv), 31); - } else { - tcg_gen_movi_i32(TCGV_HIGH(retv), 0); - } - } else { - TCGv_i32 c32 = tcg_temp_ebb_new_i32(); - TCGv_i32 n32 = tcg_temp_ebb_new_i32(); - TCGv_i32 r32 = tcg_temp_ebb_new_i32(); - - tcg_gen_extrl_i64_i32(c32, cmpv); - tcg_gen_extrl_i64_i32(n32, newv); - tcg_gen_atomic_cmpxchg_i32(r32, addr, c32, n32, idx, memop & ~MO_SIGN); - tcg_temp_free_i32(c32); - tcg_temp_free_i32(n32); - - tcg_gen_extu_i32_i64(retv, r32); - tcg_temp_free_i32(r32); - - if (memop & MO_SIGN) { - tcg_gen_ext_i64(retv, retv, memop); - } - } -} - -void tcg_gen_nonatomic_cmpxchg_i128(TCGv_i128 retv, TCGv addr, TCGv_i128 cmpv, - TCGv_i128 newv, TCGArg idx, MemOp memop) -{ - if (TCG_TARGET_REG_BITS == 32) { - /* Inline expansion below is simply too large for 32-bit hosts. */ - gen_atomic_cx_i128 gen = ((memop & MO_BSWAP) == MO_LE - ? gen_helper_nonatomic_cmpxchgo_le - : gen_helper_nonatomic_cmpxchgo_be); - MemOpIdx oi = make_memop_idx(memop, idx); - - tcg_debug_assert((memop & MO_SIZE) == MO_128); - tcg_debug_assert((memop & MO_SIGN) == 0); - - gen(retv, cpu_env, addr, cmpv, newv, tcg_constant_i32(oi)); - } else { - TCGv_i128 oldv = tcg_temp_ebb_new_i128(); - TCGv_i128 tmpv = tcg_temp_ebb_new_i128(); - TCGv_i64 t0 = tcg_temp_ebb_new_i64(); - TCGv_i64 t1 = tcg_temp_ebb_new_i64(); - TCGv_i64 z = tcg_constant_i64(0); - - tcg_gen_qemu_ld_i128(oldv, addr, idx, memop); - - /* Compare i128 */ - tcg_gen_xor_i64(t0, TCGV128_LOW(oldv), TCGV128_LOW(cmpv)); - tcg_gen_xor_i64(t1, TCGV128_HIGH(oldv), TCGV128_HIGH(cmpv)); - tcg_gen_or_i64(t0, t0, t1); - - /* tmpv = equal ? newv : oldv */ - tcg_gen_movcond_i64(TCG_COND_EQ, TCGV128_LOW(tmpv), t0, z, - TCGV128_LOW(newv), TCGV128_LOW(oldv)); - tcg_gen_movcond_i64(TCG_COND_EQ, TCGV128_HIGH(tmpv), t0, z, - TCGV128_HIGH(newv), TCGV128_HIGH(oldv)); - - /* Unconditional writeback. */ - tcg_gen_qemu_st_i128(tmpv, addr, idx, memop); - tcg_gen_mov_i128(retv, oldv); - - tcg_temp_free_i64(t0); - tcg_temp_free_i64(t1); - tcg_temp_free_i128(tmpv); - tcg_temp_free_i128(oldv); - } -} - -void tcg_gen_atomic_cmpxchg_i128(TCGv_i128 retv, TCGv addr, TCGv_i128 cmpv, - TCGv_i128 newv, TCGArg idx, MemOp memop) -{ - gen_atomic_cx_i128 gen; - - if (!(tcg_ctx->gen_tb->cflags & CF_PARALLEL)) { - tcg_gen_nonatomic_cmpxchg_i128(retv, addr, cmpv, newv, idx, memop); - return; - } - - tcg_debug_assert((memop & MO_SIZE) == MO_128); - tcg_debug_assert((memop & MO_SIGN) == 0); - gen = table_cmpxchg[memop & (MO_SIZE | MO_BSWAP)]; - - if (gen) { - MemOpIdx oi = make_memop_idx(memop, idx); - gen(retv, cpu_env, addr, cmpv, newv, tcg_constant_i32(oi)); - return; - } - - gen_helper_exit_atomic(cpu_env); - - /* - * Produce a result for a well-formed opcode stream. This satisfies - * liveness for set before used, which happens before this dead code - * is removed. - */ - tcg_gen_movi_i64(TCGV128_LOW(retv), 0); - tcg_gen_movi_i64(TCGV128_HIGH(retv), 0); -} - -static void do_nonatomic_op_i32(TCGv_i32 ret, TCGv addr, TCGv_i32 val, - TCGArg idx, MemOp memop, bool new_val, - void (*gen)(TCGv_i32, TCGv_i32, TCGv_i32)) -{ - TCGv_i32 t1 = tcg_temp_ebb_new_i32(); - TCGv_i32 t2 = tcg_temp_ebb_new_i32(); - - memop = tcg_canonicalize_memop(memop, 0, 0); - - tcg_gen_qemu_ld_i32(t1, addr, idx, memop); - tcg_gen_ext_i32(t2, val, memop); - gen(t2, t1, t2); - tcg_gen_qemu_st_i32(t2, addr, idx, memop); - - tcg_gen_ext_i32(ret, (new_val ? t2 : t1), memop); - tcg_temp_free_i32(t1); - tcg_temp_free_i32(t2); -} - -static void do_atomic_op_i32(TCGv_i32 ret, TCGv addr, TCGv_i32 val, - TCGArg idx, MemOp memop, void * const table[]) -{ - gen_atomic_op_i32 gen; - MemOpIdx oi; - - memop = tcg_canonicalize_memop(memop, 0, 0); - - gen = table[memop & (MO_SIZE | MO_BSWAP)]; - tcg_debug_assert(gen != NULL); - - oi = make_memop_idx(memop & ~MO_SIGN, idx); - gen(ret, cpu_env, addr, val, tcg_constant_i32(oi)); - - if (memop & MO_SIGN) { - tcg_gen_ext_i32(ret, ret, memop); - } -} - -static void do_nonatomic_op_i64(TCGv_i64 ret, TCGv addr, TCGv_i64 val, - TCGArg idx, MemOp memop, bool new_val, - void (*gen)(TCGv_i64, TCGv_i64, TCGv_i64)) -{ - TCGv_i64 t1 = tcg_temp_ebb_new_i64(); - TCGv_i64 t2 = tcg_temp_ebb_new_i64(); - - memop = tcg_canonicalize_memop(memop, 1, 0); - - tcg_gen_qemu_ld_i64(t1, addr, idx, memop); - tcg_gen_ext_i64(t2, val, memop); - gen(t2, t1, t2); - tcg_gen_qemu_st_i64(t2, addr, idx, memop); - - tcg_gen_ext_i64(ret, (new_val ? t2 : t1), memop); - tcg_temp_free_i64(t1); - tcg_temp_free_i64(t2); -} - -static void do_atomic_op_i64(TCGv_i64 ret, TCGv addr, TCGv_i64 val, - TCGArg idx, MemOp memop, void * const table[]) -{ - memop = tcg_canonicalize_memop(memop, 1, 0); - - if ((memop & MO_SIZE) == MO_64) { -#ifdef CONFIG_ATOMIC64 - gen_atomic_op_i64 gen; - MemOpIdx oi; - - gen = table[memop & (MO_SIZE | MO_BSWAP)]; - tcg_debug_assert(gen != NULL); - - oi = make_memop_idx(memop & ~MO_SIGN, idx); - gen(ret, cpu_env, addr, val, tcg_constant_i32(oi)); -#else - gen_helper_exit_atomic(cpu_env); - /* Produce a result, so that we have a well-formed opcode stream - with respect to uses of the result in the (dead) code following. */ - tcg_gen_movi_i64(ret, 0); -#endif /* CONFIG_ATOMIC64 */ - } else { - TCGv_i32 v32 = tcg_temp_ebb_new_i32(); - TCGv_i32 r32 = tcg_temp_ebb_new_i32(); - - tcg_gen_extrl_i64_i32(v32, val); - do_atomic_op_i32(r32, addr, v32, idx, memop & ~MO_SIGN, table); - tcg_temp_free_i32(v32); - - tcg_gen_extu_i32_i64(ret, r32); - tcg_temp_free_i32(r32); - - if (memop & MO_SIGN) { - tcg_gen_ext_i64(ret, ret, memop); - } - } -} - -#define GEN_ATOMIC_HELPER(NAME, OP, NEW) \ -static void * const table_##NAME[(MO_SIZE | MO_BSWAP) + 1] = { \ - [MO_8] = gen_helper_atomic_##NAME##b, \ - [MO_16 | MO_LE] = gen_helper_atomic_##NAME##w_le, \ - [MO_16 | MO_BE] = gen_helper_atomic_##NAME##w_be, \ - [MO_32 | MO_LE] = gen_helper_atomic_##NAME##l_le, \ - [MO_32 | MO_BE] = gen_helper_atomic_##NAME##l_be, \ - WITH_ATOMIC64([MO_64 | MO_LE] = gen_helper_atomic_##NAME##q_le) \ - WITH_ATOMIC64([MO_64 | MO_BE] = gen_helper_atomic_##NAME##q_be) \ -}; \ -void tcg_gen_atomic_##NAME##_i32 \ - (TCGv_i32 ret, TCGv addr, TCGv_i32 val, TCGArg idx, MemOp memop) \ -{ \ - if (tcg_ctx->gen_tb->cflags & CF_PARALLEL) { \ - do_atomic_op_i32(ret, addr, val, idx, memop, table_##NAME); \ - } else { \ - do_nonatomic_op_i32(ret, addr, val, idx, memop, NEW, \ - tcg_gen_##OP##_i32); \ - } \ -} \ -void tcg_gen_atomic_##NAME##_i64 \ - (TCGv_i64 ret, TCGv addr, TCGv_i64 val, TCGArg idx, MemOp memop) \ -{ \ - if (tcg_ctx->gen_tb->cflags & CF_PARALLEL) { \ - do_atomic_op_i64(ret, addr, val, idx, memop, table_##NAME); \ - } else { \ - do_nonatomic_op_i64(ret, addr, val, idx, memop, NEW, \ - tcg_gen_##OP##_i64); \ - } \ -} - -GEN_ATOMIC_HELPER(fetch_add, add, 0) -GEN_ATOMIC_HELPER(fetch_and, and, 0) -GEN_ATOMIC_HELPER(fetch_or, or, 0) -GEN_ATOMIC_HELPER(fetch_xor, xor, 0) -GEN_ATOMIC_HELPER(fetch_smin, smin, 0) -GEN_ATOMIC_HELPER(fetch_umin, umin, 0) -GEN_ATOMIC_HELPER(fetch_smax, smax, 0) -GEN_ATOMIC_HELPER(fetch_umax, umax, 0) - -GEN_ATOMIC_HELPER(add_fetch, add, 1) -GEN_ATOMIC_HELPER(and_fetch, and, 1) -GEN_ATOMIC_HELPER(or_fetch, or, 1) -GEN_ATOMIC_HELPER(xor_fetch, xor, 1) -GEN_ATOMIC_HELPER(smin_fetch, smin, 1) -GEN_ATOMIC_HELPER(umin_fetch, umin, 1) -GEN_ATOMIC_HELPER(smax_fetch, smax, 1) -GEN_ATOMIC_HELPER(umax_fetch, umax, 1) - -static void tcg_gen_mov2_i32(TCGv_i32 r, TCGv_i32 a, TCGv_i32 b) -{ - tcg_gen_mov_i32(r, b); -} - -static void tcg_gen_mov2_i64(TCGv_i64 r, TCGv_i64 a, TCGv_i64 b) -{ - tcg_gen_mov_i64(r, b); -} - -GEN_ATOMIC_HELPER(xchg, mov2, 0) - -#undef GEN_ATOMIC_HELPER diff --git a/tcg/meson.build b/tcg/meson.build index c4c63b19d4..f56c465f4d 100644 --- a/tcg/meson.build +++ b/tcg/meson.build @@ -6,6 +6,7 @@ tcg_ss.add(files( 'tcg.c', 'tcg-common.c', 'tcg-op.c', + 'tcg-op-ldst.c', 'tcg-op-gvec.c', 'tcg-op-vec.c', )) From patchwork Tue May 16 19:41:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682388 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp18789wrt; Tue, 16 May 2023 12:51:27 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5/Ze3HQOvLMjm3u+fCgX8E7JVs1gV54iBLQzQhNJmOatD5/sww38hj3kEoyvjB3+vNg/Sr X-Received: by 2002:a05:6214:19ef:b0:61b:79ab:7129 with SMTP id q15-20020a05621419ef00b0061b79ab7129mr61684203qvc.37.1684266687059; Tue, 16 May 2023 12:51:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266687; cv=none; d=google.com; s=arc-20160816; b=b5E8nRZA0+Tq4WYW8FtQhUZE2Myhx0foy5BQ6Ezr/uL+PXZJDKtAMN3XlGUnX+zkHv sNJLGf8iFCrRAHXwNjR/LCsi2ai3TDu1qbUdSswExDzKsrix3+9FTp4K3RvVKMqBue0+ FG1XDQRRXL7efFUydRbrSrxzZFnYUbg0ei88IEx23840dSUd4kREH5IC63XHTSVI4CO1 Ex8bCJOYvkNYDzcbLivVNXsqcqQ4tRe2BqVWfzYKq2fHxVpJ9OxmbNa8WIJ0hRKSn1zO 7UuKcl8hEFmpQEBFekqe+3yhzFxU9fto2id8hKO7pwXunQo8KjIrwsNBfoH65gXtrL/s YJwQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=gWhgnFD1Hnh8xksYaxZ0Bm0a3qHlsqVm0XjZ42LcIb0=; b=yTidDAdQuy115zCrTCtLVHYpP0GmtvzdsR/Iuz75pAUEu4Ot99QPCWO4m7+mdkHcFI RAw65/JRTxI4YPYmmRExUrx4vGqbl4lb12PbSUcDjYBq6HMFOu2Ps+DR92cJrrdYjJPC bNYl9nTxz0pPqakjwquth/LmyJv05YrDm6vMI9OchTa2LgxONSRPD9MvcOD1Q5RQ7+09 oEMdcmTyB5Vz7ILY546wyQPZIuPbYsUMGk4f6/H1lKhjG0pnO3ATAGZUNZ2jyZ+izQDo /H/YEH2izUBQKlVddXcueeG8lLLmv5/mhkdazGE+capC6xx63oNdWpSJbl2o4HyOri9p vtwg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=t6icjcnf; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id kl30-20020a056214519e00b005f24d617fb3si11859801qvb.370.2023.05.16.12.51.26 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:51:27 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=t6icjcnf; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0f9-0004Nf-HW; Tue, 16 May 2023 15:48:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cC-0000FA-Ou for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:46 -0400 Received: from mail-pj1-x1030.google.com ([2607:f8b0:4864:20::1030]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0c6-0003jw-JO for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:42 -0400 Received: by mail-pj1-x1030.google.com with SMTP id 98e67ed59e1d1-2533b600d35so58152a91.1 for ; Tue, 16 May 2023 12:45:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266335; x=1686858335; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gWhgnFD1Hnh8xksYaxZ0Bm0a3qHlsqVm0XjZ42LcIb0=; b=t6icjcnf1uwM6EBpSZlNvy46PBqxer/SMqu9PjnhQNeja6Q6vJ+At1jQFCDXiWCWLT XrTr0yV18crxmhW4Zuca6mApZjwfh9ny/LJqsSmqCIOXjM/JFGF/UmPpAOnBKdTl13MW 3wMNNnjhBAItjCFQ0Ss7fa4MM1Ka/o7n/sfdQ6+n799fGJznGfD+222VNfQYMrt6rXmM sed+WaTGJhkLcpWv6gQNGJouXUTOQxPEkPxVk7DCKKtR65mzhG/VL/mV4W4UbAw8EZ+M 2zXjuYPpUf9UFIL3G9ZICN41rKpr/5AJwdFpUUWsrsaQ+rHwxzRLape+blPeDEguMyaA nzsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266335; x=1686858335; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gWhgnFD1Hnh8xksYaxZ0Bm0a3qHlsqVm0XjZ42LcIb0=; b=KaNa/ITexZOBVRqtfrApwx7h4E2JhN3N6woA9La8fXUpEXvyofD3IZCQJ/lYN58o1A CLgWaklsdK0moi5a1vDGtgJc7YPib+PCFSK5J4TGq2Sb1c8aq8FrtjmHtZ54hp+LwOPm vzLkS+6esh/g63pmw1jfzvouN8PAcPuqpEpAOD3IpLI65GhTMS01PgMQ4HLUcmmtALTu 4wiQUpv3Mv4JZeMIfVX8Zvau/lAtS0XTCYfipZe9nLwUyzAYPgHtqLUv5ty1Phugr0BG 8EFHvO2xdtzY62n5qLIH6jrdAot4MRAC2bezPZhUTrN3lVqPEblUn+ZYeITxKE9Sr1rh lIdg== X-Gm-Message-State: AC+VfDxx3MYFD2pRV0WX/lBWd5WSGZbOHRr6+e7w/lrr3qE0A5QDSQXr KEl08U7VN+yo8k51vZJ0lSOV0cVXiDU2fA0/rV0= X-Received: by 2002:a17:90b:2250:b0:24d:e123:1eb2 with SMTP id hk16-20020a17090b225000b0024de1231eb2mr39061365pjb.37.1684266335696; Tue, 16 May 2023 12:45:35 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:35 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Anton Johansson , =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 54/80] tcg: Widen gen_insn_data to uint64_t Date: Tue, 16 May 2023 12:41:19 -0700 Message-Id: <20230516194145.1749305-55-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1030; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1030.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org We already pass uint64_t to restore_state_to_opc; this changes all of the other uses from insn_start through the encoding to decoding. Reviewed-by: Anton Johansson Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- include/tcg/tcg-op.h | 39 +++++++++------------------------------ include/tcg/tcg-opc.h | 2 +- include/tcg/tcg.h | 30 +++++++++++++++--------------- accel/tcg/translate-all.c | 28 ++++++++++++++++------------ tcg/tcg.c | 18 ++++-------------- 5 files changed, 45 insertions(+), 72 deletions(-) diff --git a/include/tcg/tcg-op.h b/include/tcg/tcg-op.h index 4401fa493c..de3b70aa84 100644 --- a/include/tcg/tcg-op.h +++ b/include/tcg/tcg-op.h @@ -723,48 +723,27 @@ static inline void tcg_gen_concat32_i64(TCGv_i64 ret, TCGv_i64 lo, TCGv_i64 hi) #endif #if TARGET_INSN_START_WORDS == 1 -# if TARGET_LONG_BITS <= TCG_TARGET_REG_BITS static inline void tcg_gen_insn_start(target_ulong pc) { - tcg_gen_op1(INDEX_op_insn_start, pc); + TCGOp *op = tcg_emit_op(INDEX_op_insn_start, 64 / TCG_TARGET_REG_BITS); + tcg_set_insn_start_param(op, 0, pc); } -# else -static inline void tcg_gen_insn_start(target_ulong pc) -{ - tcg_gen_op2(INDEX_op_insn_start, (uint32_t)pc, (uint32_t)(pc >> 32)); -} -# endif #elif TARGET_INSN_START_WORDS == 2 -# if TARGET_LONG_BITS <= TCG_TARGET_REG_BITS static inline void tcg_gen_insn_start(target_ulong pc, target_ulong a1) { - tcg_gen_op2(INDEX_op_insn_start, pc, a1); + TCGOp *op = tcg_emit_op(INDEX_op_insn_start, 2 * 64 / TCG_TARGET_REG_BITS); + tcg_set_insn_start_param(op, 0, pc); + tcg_set_insn_start_param(op, 1, a1); } -# else -static inline void tcg_gen_insn_start(target_ulong pc, target_ulong a1) -{ - tcg_gen_op4(INDEX_op_insn_start, - (uint32_t)pc, (uint32_t)(pc >> 32), - (uint32_t)a1, (uint32_t)(a1 >> 32)); -} -# endif #elif TARGET_INSN_START_WORDS == 3 -# if TARGET_LONG_BITS <= TCG_TARGET_REG_BITS static inline void tcg_gen_insn_start(target_ulong pc, target_ulong a1, target_ulong a2) { - tcg_gen_op3(INDEX_op_insn_start, pc, a1, a2); + TCGOp *op = tcg_emit_op(INDEX_op_insn_start, 3 * 64 / TCG_TARGET_REG_BITS); + tcg_set_insn_start_param(op, 0, pc); + tcg_set_insn_start_param(op, 1, a1); + tcg_set_insn_start_param(op, 2, a2); } -# else -static inline void tcg_gen_insn_start(target_ulong pc, target_ulong a1, - target_ulong a2) -{ - tcg_gen_op6(INDEX_op_insn_start, - (uint32_t)pc, (uint32_t)(pc >> 32), - (uint32_t)a1, (uint32_t)(a1 >> 32), - (uint32_t)a2, (uint32_t)(a2 >> 32)); -} -# endif #else # error "Unhandled number of operands to insn_start" #endif diff --git a/include/tcg/tcg-opc.h b/include/tcg/tcg-opc.h index 94cf7c5d6a..29216366d2 100644 --- a/include/tcg/tcg-opc.h +++ b/include/tcg/tcg-opc.h @@ -190,7 +190,7 @@ DEF(mulsh_i64, 1, 2, 0, IMPL64 | IMPL(TCG_TARGET_HAS_mulsh_i64)) #define DATA64_ARGS (TCG_TARGET_REG_BITS == 64 ? 1 : 2) /* QEMU specific */ -DEF(insn_start, 0, 0, TLADDR_ARGS * TARGET_INSN_START_WORDS, +DEF(insn_start, 0, 0, DATA64_ARGS * TARGET_INSN_START_WORDS, TCG_OPF_NOT_PRESENT) DEF(exit_tb, 0, 0, 1, TCG_OPF_BB_EXIT | TCG_OPF_BB_END) DEF(goto_tb, 0, 0, 1, TCG_OPF_BB_EXIT | TCG_OPF_BB_END) diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h index b19e167e1d..f40de4177d 100644 --- a/include/tcg/tcg.h +++ b/include/tcg/tcg.h @@ -629,7 +629,7 @@ struct TCGContext { TCGTemp *reg_to_temp[TCG_TARGET_NB_REGS]; uint16_t gen_insn_end_off[TCG_MAX_INSNS]; - target_ulong gen_insn_data[TCG_MAX_INSNS][TARGET_INSN_START_WORDS]; + uint64_t gen_insn_data[TCG_MAX_INSNS][TARGET_INSN_START_WORDS]; /* Exit to translator on overflow. */ sigjmp_buf jmp_trans; @@ -771,24 +771,24 @@ static inline void tcg_set_insn_param(TCGOp *op, int arg, TCGArg v) op->args[arg] = v; } -static inline target_ulong tcg_get_insn_start_param(TCGOp *op, int arg) +static inline uint64_t tcg_get_insn_start_param(TCGOp *op, int arg) { -#if TARGET_LONG_BITS <= TCG_TARGET_REG_BITS - return tcg_get_insn_param(op, arg); -#else - return tcg_get_insn_param(op, arg * 2) | - ((uint64_t)tcg_get_insn_param(op, arg * 2 + 1) << 32); -#endif + if (TCG_TARGET_REG_BITS == 64) { + return tcg_get_insn_param(op, arg); + } else { + return deposit64(tcg_get_insn_param(op, arg * 2), 32, 32, + tcg_get_insn_param(op, arg * 2 + 1)); + } } -static inline void tcg_set_insn_start_param(TCGOp *op, int arg, target_ulong v) +static inline void tcg_set_insn_start_param(TCGOp *op, int arg, uint64_t v) { -#if TARGET_LONG_BITS <= TCG_TARGET_REG_BITS - tcg_set_insn_param(op, arg, v); -#else - tcg_set_insn_param(op, arg * 2, v); - tcg_set_insn_param(op, arg * 2 + 1, v >> 32); -#endif + if (TCG_TARGET_REG_BITS == 64) { + tcg_set_insn_param(op, arg, v); + } else { + tcg_set_insn_param(op, arg * 2, v); + tcg_set_insn_param(op, arg * 2 + 1, v >> 32); + } } /* The last op that was emitted. */ diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c index 5b13281119..7b7d9a5fff 100644 --- a/accel/tcg/translate-all.c +++ b/accel/tcg/translate-all.c @@ -72,9 +72,11 @@ QEMU_BUILD_BUG_ON(CPU_TRACE_DSTATE_MAX_EVENTS > TBContext tb_ctx; -/* Encode VAL as a signed leb128 sequence at P. - Return P incremented past the encoded value. */ -static uint8_t *encode_sleb128(uint8_t *p, target_long val) +/* + * Encode VAL as a signed leb128 sequence at P. + * Return P incremented past the encoded value. + */ +static uint8_t *encode_sleb128(uint8_t *p, int64_t val) { int more, byte; @@ -92,21 +94,23 @@ static uint8_t *encode_sleb128(uint8_t *p, target_long val) return p; } -/* Decode a signed leb128 sequence at *PP; increment *PP past the - decoded value. Return the decoded value. */ -static target_long decode_sleb128(const uint8_t **pp) +/* + * Decode a signed leb128 sequence at *PP; increment *PP past the + * decoded value. Return the decoded value. + */ +static int64_t decode_sleb128(const uint8_t **pp) { const uint8_t *p = *pp; - target_long val = 0; + int64_t val = 0; int byte, shift = 0; do { byte = *p++; - val |= (target_ulong)(byte & 0x7f) << shift; + val |= (int64_t)(byte & 0x7f) << shift; shift += 7; } while (byte & 0x80); if (shift < TARGET_LONG_BITS && (byte & 0x40)) { - val |= -(target_ulong)1 << shift; + val |= -(int64_t)1 << shift; } *pp = p; @@ -132,7 +136,7 @@ static int encode_search(TranslationBlock *tb, uint8_t *block) int i, j, n; for (i = 0, n = tb->icount; i < n; ++i) { - target_ulong prev; + uint64_t prev; for (j = 0; j < TARGET_INSN_START_WORDS; ++j) { if (i == 0) { @@ -444,7 +448,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu, /* Dump header and the first instruction */ fprintf(logfile, "OUT: [size=%d]\n", gen_code_size); fprintf(logfile, - " -- guest addr 0x" TARGET_FMT_lx " + tb prologue\n", + " -- guest addr 0x%016" PRIx64 " + tb prologue\n", tcg_ctx->gen_insn_data[insn][0]); chunk_start = tcg_ctx->gen_insn_end_off[insn]; disas(logfile, tb->tc.ptr, chunk_start); @@ -457,7 +461,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu, while (insn < tb->icount) { size_t chunk_end = tcg_ctx->gen_insn_end_off[insn]; if (chunk_end > chunk_start) { - fprintf(logfile, " -- guest addr 0x" TARGET_FMT_lx "\n", + fprintf(logfile, " -- guest addr 0x%016" PRIx64 "\n", tcg_ctx->gen_insn_data[insn][0]); disas(logfile, tb->tc.ptr + chunk_start, chunk_end - chunk_start); diff --git a/tcg/tcg.c b/tcg/tcg.c index 878e780cfa..da34e5ab22 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -2382,13 +2382,8 @@ static void tcg_dump_ops(TCGContext *s, FILE *f, bool have_prefs) col += ne_fprintf(f, "\n ----"); for (i = 0; i < TARGET_INSN_START_WORDS; ++i) { - target_ulong a; -#if TARGET_LONG_BITS > TCG_TARGET_REG_BITS - a = deposit64(op->args[i * 2], 32, 32, op->args[i * 2 + 1]); -#else - a = op->args[i]; -#endif - col += ne_fprintf(f, " " TARGET_FMT_lx, a); + col += ne_fprintf(f, " %016" PRIx64, + tcg_get_insn_start_param(op, i)); } } else if (c == INDEX_op_call) { const TCGHelperInfo *info = tcg_call_info(op); @@ -6079,13 +6074,8 @@ int tcg_gen_code(TCGContext *s, TranslationBlock *tb, target_ulong pc_start) } num_insns++; for (i = 0; i < TARGET_INSN_START_WORDS; ++i) { - target_ulong a; -#if TARGET_LONG_BITS > TCG_TARGET_REG_BITS - a = deposit64(op->args[i * 2], 32, 32, op->args[i * 2 + 1]); -#else - a = op->args[i]; -#endif - s->gen_insn_data[num_insns][i] = a; + s->gen_insn_data[num_insns][i] = + tcg_get_insn_start_param(op, i); } break; case INDEX_op_discard: From patchwork Tue May 16 19:41:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682402 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp20026wrt; Tue, 16 May 2023 12:55:42 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6GAyHJniBPynG22vKwoxpsBlF+2qMKDsipU2xJ2WtjAcFXBiN7sDip83VxqSTgyeIFRL1K X-Received: by 2002:a05:6214:2b06:b0:5f1:606b:a9ca with SMTP id jx6-20020a0562142b0600b005f1606ba9camr67275753qvb.37.1684266941822; Tue, 16 May 2023 12:55:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266941; cv=none; d=google.com; s=arc-20160816; b=HKxrtRgnnowHYMjuv0rZDdww3a3gxP99LhYHxHvAmGMnKXXsWpMoM3ngtu2LjZZSyx PPa/YB6kLdHNRhRQcCHIMYWuD3x0WlyfAzSxeIeE48lA9ICZx6mTAigQLd3WPPiU5A6G Zj6+tnQySWuayDQEJZ9U3er0bWZLWk0lrh/Qd94CNFxiPUIQWPiRg/6stWG98yVnIMlt J1coPGXT+KX2/vte8NZ4mkkrZe8VrxxuBNRaO9Y3gz/UhPMI65mhWj7AtWTz9YmcxeCC 6mm7qbPzxNgVjtOXgQHcF7OfGIaXbUC9jzJsJt9dw4g2juNjOmlOk0tAYfiFQF4GqbFh uV/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=x76Mg4C5wSL1+atXcZvGeh9IH9ky2HxmUhpgDLdcbeQ=; b=OJyLOvGQ6Hfu5npSh+C8Jd9yLoo8YWKbsgWLr+BHob1zZ+XeDYsm9Im7l1BC26Zo7G fBatIQco3BlQYsZ1cgDDXnUfTCyoaFkn7jVzq1U7sXKxmnRVLv8ebyTO92WqmsYBlXsv Sl3LYRt1Uezqqq97O90Oq9xbcCY4NN2iiJUM+eQ5UuSN0eTtBv4mkxh4UlZ8GPPkCYVt kRSfMYf6Jxo8vg3ZH2pjHU8Xjh7MS5+J9h8rHfPfezfXvwylLzveCPDymluMjQsLq0TU NrZr6bUjjuehlN4vSvOAW4ckrrksOGZP/Db35orfaDdOvpTgCR2zgu3fDRMm4hmy3QpZ UqoQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WJ3d9Xms; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f11-20020a0562141d2b00b005e63b4566c5si12288413qvd.81.2023.05.16.12.55.41 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:55:41 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WJ3d9Xms; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0cY-0000qG-PH; Tue, 16 May 2023 15:46:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cC-0000F7-Nd for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:46 -0400 Received: from mail-pj1-x102b.google.com ([2607:f8b0:4864:20::102b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0c6-0003hm-Jq for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:42 -0400 Received: by mail-pj1-x102b.google.com with SMTP id 98e67ed59e1d1-24decf5cc03so54291a91.0 for ; Tue, 16 May 2023 12:45:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266336; x=1686858336; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=x76Mg4C5wSL1+atXcZvGeh9IH9ky2HxmUhpgDLdcbeQ=; b=WJ3d9XmsBt3SvUaJVMpcs4QM8Gc/hfgfIRhz9O3B9fmMSuLW6DkFbHR1S9k/+WQC8B ICDc2icNqTDIOdmAwO4bxAs6qZRhIFJJLk9HRNB20aX+1i4n27bbMqZa/+r62968AhbU 0Y23TJnLRbtFoUWNgnuwn8ca0djGIs+IO1XbDYGHwwpsouZBjCedWk9Q5Kln46c6F9YN vSysJPGkDfVAbghp5rsJHH4Lsjc+SFgJHYtyLEB1sUqQ1BMKITwjHT1Ae8zdm8Lt4boC xI3wJ/pO3hu6vQrLoaV5taTj8qhiZMm4q3wTnZH/oAPxq/dihsoqpEMYmVEcRhDyAuBC 6wOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266336; x=1686858336; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=x76Mg4C5wSL1+atXcZvGeh9IH9ky2HxmUhpgDLdcbeQ=; b=C8mvpM3mDFC5pmcHfUIATAb3McZhJn/3vZyDXlKLzXFF7iyjH529Mxj4/d1J/3ObMK IAcLh8gJkFy77A0YhO3qo6lHAp4ertlZK9Ftsas42D5uRQo8imzFLBK39At726TMfyUu s8+kA7EkzhcJ83Dx9GWm+kHrAkdctPJuEQlF5pMB+L4mOGl6u8r6mvvKMtQu/DIRSlH8 ErlVjtpg3wnZA5aaS21LdAcZkerSqIR/TmqNqWlnvgEfgWhiB2pZhCdzp+znOcG3iGsv k5vVQzz2xOlChsyrxACWvIBuJdpDiS6TcuKGxfMylrKsF6ihEHMQz6lovEfnQ4x1u2An SvGA== X-Gm-Message-State: AC+VfDyHD8ipDcyPkSOboMqFnAPIDpLXRSC84cm7bJYtajVJSAq54WOF X5G67JwedSx5KTXUqHImS9j4hgCfuOaBtyQz7R8= X-Received: by 2002:a17:90a:b897:b0:253:38bf:9757 with SMTP id o23-20020a17090ab89700b0025338bf9757mr1232850pjr.43.1684266336432; Tue, 16 May 2023 12:45:36 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:36 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 55/80] accel/tcg: Widen tcg-ldst.h addresses to uint64_t Date: Tue, 16 May 2023 12:41:20 -0700 Message-Id: <20230516194145.1749305-56-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102b; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Always pass the target address as uint64_t. Adjust tcg_out_{ld,st}_helper_args to match. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- include/tcg/tcg-ldst.h | 26 +++++++++--------- accel/tcg/cputlb.c | 26 +++++++++--------- accel/tcg/user-exec.c | 26 +++++++++--------- tcg/tcg.c | 62 ++++++++++++++++++++++++++++++++---------- 4 files changed, 87 insertions(+), 53 deletions(-) diff --git a/include/tcg/tcg-ldst.h b/include/tcg/tcg-ldst.h index 7dd57013e9..6ccfe9131d 100644 --- a/include/tcg/tcg-ldst.h +++ b/include/tcg/tcg-ldst.h @@ -26,38 +26,38 @@ #define TCG_LDST_H /* Value zero-extended to tcg register size. */ -tcg_target_ulong helper_ldub_mmu(CPUArchState *env, target_ulong addr, +tcg_target_ulong helper_ldub_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t retaddr); -tcg_target_ulong helper_lduw_mmu(CPUArchState *env, target_ulong addr, +tcg_target_ulong helper_lduw_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t retaddr); -tcg_target_ulong helper_ldul_mmu(CPUArchState *env, target_ulong addr, +tcg_target_ulong helper_ldul_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t retaddr); -uint64_t helper_ldq_mmu(CPUArchState *env, target_ulong addr, +uint64_t helper_ldq_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t retaddr); -Int128 helper_ld16_mmu(CPUArchState *env, target_ulong addr, +Int128 helper_ld16_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t retaddr); /* Value sign-extended to tcg register size. */ -tcg_target_ulong helper_ldsb_mmu(CPUArchState *env, target_ulong addr, +tcg_target_ulong helper_ldsb_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t retaddr); -tcg_target_ulong helper_ldsw_mmu(CPUArchState *env, target_ulong addr, +tcg_target_ulong helper_ldsw_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t retaddr); -tcg_target_ulong helper_ldsl_mmu(CPUArchState *env, target_ulong addr, +tcg_target_ulong helper_ldsl_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t retaddr); /* * Value extended to at least uint32_t, so that some ABIs do not require * zero-extension from uint8_t or uint16_t. */ -void helper_stb_mmu(CPUArchState *env, target_ulong addr, uint32_t val, +void helper_stb_mmu(CPUArchState *env, uint64_t addr, uint32_t val, MemOpIdx oi, uintptr_t retaddr); -void helper_stw_mmu(CPUArchState *env, target_ulong addr, uint32_t val, +void helper_stw_mmu(CPUArchState *env, uint64_t addr, uint32_t val, MemOpIdx oi, uintptr_t retaddr); -void helper_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, +void helper_stl_mmu(CPUArchState *env, uint64_t addr, uint32_t val, MemOpIdx oi, uintptr_t retaddr); -void helper_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, +void helper_stq_mmu(CPUArchState *env, uint64_t addr, uint64_t val, MemOpIdx oi, uintptr_t retaddr); -void helper_st16_mmu(CPUArchState *env, target_ulong addr, Int128 val, +void helper_st16_mmu(CPUArchState *env, uint64_t addr, Int128 val, MemOpIdx oi, uintptr_t retaddr); #endif /* TCG_LDST_H */ diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 49e49f75a4..5440f68deb 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -2367,7 +2367,7 @@ static uint8_t do_ld1_mmu(CPUArchState *env, target_ulong addr, MemOpIdx oi, return do_ld_1(env, &l.page[0], l.mmu_idx, access_type, ra); } -tcg_target_ulong helper_ldub_mmu(CPUArchState *env, target_ulong addr, +tcg_target_ulong helper_ldub_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t retaddr) { tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_8); @@ -2398,7 +2398,7 @@ static uint16_t do_ld2_mmu(CPUArchState *env, target_ulong addr, MemOpIdx oi, return ret; } -tcg_target_ulong helper_lduw_mmu(CPUArchState *env, target_ulong addr, +tcg_target_ulong helper_lduw_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t retaddr) { tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_16); @@ -2425,7 +2425,7 @@ static uint32_t do_ld4_mmu(CPUArchState *env, target_ulong addr, MemOpIdx oi, return ret; } -tcg_target_ulong helper_ldul_mmu(CPUArchState *env, target_ulong addr, +tcg_target_ulong helper_ldul_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t retaddr) { tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_32); @@ -2452,7 +2452,7 @@ static uint64_t do_ld8_mmu(CPUArchState *env, target_ulong addr, MemOpIdx oi, return ret; } -uint64_t helper_ldq_mmu(CPUArchState *env, target_ulong addr, +uint64_t helper_ldq_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t retaddr) { tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_64); @@ -2464,19 +2464,19 @@ uint64_t helper_ldq_mmu(CPUArchState *env, target_ulong addr, * avoid this for 64-bit data, or for 32-bit data on 32-bit host. */ -tcg_target_ulong helper_ldsb_mmu(CPUArchState *env, target_ulong addr, +tcg_target_ulong helper_ldsb_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t retaddr) { return (int8_t)helper_ldub_mmu(env, addr, oi, retaddr); } -tcg_target_ulong helper_ldsw_mmu(CPUArchState *env, target_ulong addr, +tcg_target_ulong helper_ldsw_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t retaddr) { return (int16_t)helper_lduw_mmu(env, addr, oi, retaddr); } -tcg_target_ulong helper_ldsl_mmu(CPUArchState *env, target_ulong addr, +tcg_target_ulong helper_ldsl_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t retaddr) { return (int32_t)helper_ldul_mmu(env, addr, oi, retaddr); @@ -2544,7 +2544,7 @@ static Int128 do_ld16_mmu(CPUArchState *env, target_ulong addr, return ret; } -Int128 helper_ld16_mmu(CPUArchState *env, target_ulong addr, +Int128 helper_ld16_mmu(CPUArchState *env, uint64_t addr, uint32_t oi, uintptr_t retaddr) { tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_128); @@ -2860,7 +2860,7 @@ static void do_st_8(CPUArchState *env, MMULookupPageData *p, uint64_t val, } } -void helper_stb_mmu(CPUArchState *env, target_ulong addr, uint32_t val, +void helper_stb_mmu(CPUArchState *env, uint64_t addr, uint32_t val, MemOpIdx oi, uintptr_t ra) { MMULookupLocals l; @@ -2895,7 +2895,7 @@ static void do_st2_mmu(CPUArchState *env, target_ulong addr, uint16_t val, do_st_1(env, &l.page[1], b, l.mmu_idx, ra); } -void helper_stw_mmu(CPUArchState *env, target_ulong addr, uint32_t val, +void helper_stw_mmu(CPUArchState *env, uint64_t addr, uint32_t val, MemOpIdx oi, uintptr_t retaddr) { tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_16); @@ -2922,7 +2922,7 @@ static void do_st4_mmu(CPUArchState *env, target_ulong addr, uint32_t val, (void) do_st_leN(env, &l.page[1], val, l.mmu_idx, l.memop, ra); } -void helper_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, +void helper_stl_mmu(CPUArchState *env, uint64_t addr, uint32_t val, MemOpIdx oi, uintptr_t retaddr) { tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_32); @@ -2949,7 +2949,7 @@ static void do_st8_mmu(CPUArchState *env, target_ulong addr, uint64_t val, (void) do_st_leN(env, &l.page[1], val, l.mmu_idx, l.memop, ra); } -void helper_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, +void helper_stq_mmu(CPUArchState *env, uint64_t addr, uint64_t val, MemOpIdx oi, uintptr_t retaddr) { tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_64); @@ -3017,7 +3017,7 @@ static void do_st16_mmu(CPUArchState *env, target_ulong addr, Int128 val, } } -void helper_st16_mmu(CPUArchState *env, target_ulong addr, Int128 val, +void helper_st16_mmu(CPUArchState *env, uint64_t addr, Int128 val, MemOpIdx oi, uintptr_t retaddr) { tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_128); diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index 7b824dcde8..9a7afb6f78 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -920,13 +920,13 @@ static uint8_t do_ld1_mmu(CPUArchState *env, abi_ptr addr, return ret; } -tcg_target_ulong helper_ldub_mmu(CPUArchState *env, target_ulong addr, +tcg_target_ulong helper_ldub_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t ra) { return do_ld1_mmu(env, addr, get_memop(oi), ra); } -tcg_target_ulong helper_ldsb_mmu(CPUArchState *env, target_ulong addr, +tcg_target_ulong helper_ldsb_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t ra) { return (int8_t)do_ld1_mmu(env, addr, get_memop(oi), ra); @@ -953,7 +953,7 @@ static uint16_t do_ld2_he_mmu(CPUArchState *env, abi_ptr addr, return ret; } -tcg_target_ulong helper_lduw_mmu(CPUArchState *env, target_ulong addr, +tcg_target_ulong helper_lduw_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t ra) { MemOp mop = get_memop(oi); @@ -965,7 +965,7 @@ tcg_target_ulong helper_lduw_mmu(CPUArchState *env, target_ulong addr, return ret; } -tcg_target_ulong helper_ldsw_mmu(CPUArchState *env, target_ulong addr, +tcg_target_ulong helper_ldsw_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t ra) { MemOp mop = get_memop(oi); @@ -1014,7 +1014,7 @@ static uint32_t do_ld4_he_mmu(CPUArchState *env, abi_ptr addr, return ret; } -tcg_target_ulong helper_ldul_mmu(CPUArchState *env, target_ulong addr, +tcg_target_ulong helper_ldul_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t ra) { MemOp mop = get_memop(oi); @@ -1026,7 +1026,7 @@ tcg_target_ulong helper_ldul_mmu(CPUArchState *env, target_ulong addr, return ret; } -tcg_target_ulong helper_ldsl_mmu(CPUArchState *env, target_ulong addr, +tcg_target_ulong helper_ldsl_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t ra) { MemOp mop = get_memop(oi); @@ -1075,7 +1075,7 @@ static uint64_t do_ld8_he_mmu(CPUArchState *env, abi_ptr addr, return ret; } -uint64_t helper_ldq_mmu(CPUArchState *env, target_ulong addr, +uint64_t helper_ldq_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t ra) { MemOp mop = get_memop(oi); @@ -1124,7 +1124,7 @@ static Int128 do_ld16_he_mmu(CPUArchState *env, abi_ptr addr, return ret; } -Int128 helper_ld16_mmu(CPUArchState *env, target_ulong addr, +Int128 helper_ld16_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t ra) { MemOp mop = get_memop(oi); @@ -1182,7 +1182,7 @@ static void do_st1_mmu(CPUArchState *env, abi_ptr addr, uint8_t val, clear_helper_retaddr(); } -void helper_stb_mmu(CPUArchState *env, target_ulong addr, uint32_t val, +void helper_stb_mmu(CPUArchState *env, uint64_t addr, uint32_t val, MemOpIdx oi, uintptr_t ra) { do_st1_mmu(env, addr, val, get_memop(oi), ra); @@ -1206,7 +1206,7 @@ static void do_st2_he_mmu(CPUArchState *env, abi_ptr addr, uint16_t val, clear_helper_retaddr(); } -void helper_stw_mmu(CPUArchState *env, target_ulong addr, uint32_t val, +void helper_stw_mmu(CPUArchState *env, uint64_t addr, uint32_t val, MemOpIdx oi, uintptr_t ra) { MemOp mop = get_memop(oi); @@ -1248,7 +1248,7 @@ static void do_st4_he_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, clear_helper_retaddr(); } -void helper_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, +void helper_stl_mmu(CPUArchState *env, uint64_t addr, uint32_t val, MemOpIdx oi, uintptr_t ra) { MemOp mop = get_memop(oi); @@ -1290,7 +1290,7 @@ static void do_st8_he_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, clear_helper_retaddr(); } -void helper_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, +void helper_stq_mmu(CPUArchState *env, uint64_t addr, uint64_t val, MemOpIdx oi, uintptr_t ra) { MemOp mop = get_memop(oi); @@ -1332,7 +1332,7 @@ static void do_st16_he_mmu(CPUArchState *env, abi_ptr addr, Int128 val, clear_helper_retaddr(); } -void helper_st16_mmu(CPUArchState *env, target_ulong addr, Int128 val, +void helper_st16_mmu(CPUArchState *env, uint64_t addr, Int128 val, MemOpIdx oi, uintptr_t ra) { MemOp mop = get_memop(oi); diff --git a/tcg/tcg.c b/tcg/tcg.c index da34e5ab22..cdd194639e 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -865,7 +865,7 @@ static TCGHelperInfo info_helper_ld32_mmu = { .flags = TCG_CALL_NO_WG, .typemask = dh_typemask(ttl, 0) /* return tcg_target_ulong */ | dh_typemask(env, 1) - | dh_typemask(tl, 2) /* target_ulong addr */ + | dh_typemask(i64, 2) /* uint64_t addr */ | dh_typemask(i32, 3) /* unsigned oi */ | dh_typemask(ptr, 4) /* uintptr_t ra */ }; @@ -874,7 +874,7 @@ static TCGHelperInfo info_helper_ld64_mmu = { .flags = TCG_CALL_NO_WG, .typemask = dh_typemask(i64, 0) /* return uint64_t */ | dh_typemask(env, 1) - | dh_typemask(tl, 2) /* target_ulong addr */ + | dh_typemask(i64, 2) /* uint64_t addr */ | dh_typemask(i32, 3) /* unsigned oi */ | dh_typemask(ptr, 4) /* uintptr_t ra */ }; @@ -883,7 +883,7 @@ static TCGHelperInfo info_helper_ld128_mmu = { .flags = TCG_CALL_NO_WG, .typemask = dh_typemask(i128, 0) /* return Int128 */ | dh_typemask(env, 1) - | dh_typemask(tl, 2) /* target_ulong addr */ + | dh_typemask(i64, 2) /* uint64_t addr */ | dh_typemask(i32, 3) /* unsigned oi */ | dh_typemask(ptr, 4) /* uintptr_t ra */ }; @@ -892,7 +892,7 @@ static TCGHelperInfo info_helper_st32_mmu = { .flags = TCG_CALL_NO_WG, .typemask = dh_typemask(void, 0) | dh_typemask(env, 1) - | dh_typemask(tl, 2) /* target_ulong addr */ + | dh_typemask(i64, 2) /* uint64_t addr */ | dh_typemask(i32, 3) /* uint32_t data */ | dh_typemask(i32, 4) /* unsigned oi */ | dh_typemask(ptr, 5) /* uintptr_t ra */ @@ -902,7 +902,7 @@ static TCGHelperInfo info_helper_st64_mmu = { .flags = TCG_CALL_NO_WG, .typemask = dh_typemask(void, 0) | dh_typemask(env, 1) - | dh_typemask(tl, 2) /* target_ulong addr */ + | dh_typemask(i64, 2) /* uint64_t addr */ | dh_typemask(i64, 3) /* uint64_t data */ | dh_typemask(i32, 4) /* unsigned oi */ | dh_typemask(ptr, 5) /* uintptr_t ra */ @@ -912,7 +912,7 @@ static TCGHelperInfo info_helper_st128_mmu = { .flags = TCG_CALL_NO_WG, .typemask = dh_typemask(void, 0) | dh_typemask(env, 1) - | dh_typemask(tl, 2) /* target_ulong addr */ + | dh_typemask(i64, 2) /* uint64_t addr */ | dh_typemask(i128, 3) /* Int128 data */ | dh_typemask(i32, 4) /* unsigned oi */ | dh_typemask(ptr, 5) /* uintptr_t ra */ @@ -5595,11 +5595,26 @@ static void tcg_out_ld_helper_args(TCGContext *s, const TCGLabelQemuLdst *ldst, next_arg = 1; loc = &info->in[next_arg]; - nmov = tcg_out_helper_add_mov(mov, loc, TCG_TYPE_TL, TCG_TYPE_TL, - ldst->addrlo_reg, ldst->addrhi_reg); - next_arg += nmov; + if (TCG_TARGET_REG_BITS == 64 || TARGET_LONG_BITS == 64) { + nmov = tcg_out_helper_add_mov(mov, loc, TCG_TYPE_I64, TCG_TYPE_TL, + ldst->addrlo_reg, ldst->addrhi_reg); + tcg_out_helper_load_slots(s, nmov, mov, parm); + next_arg += nmov; + } else { + /* + * 32-bit host with 32-bit guest: zero-extend the guest address + * to 64-bits for the helper by storing the low part, then + * load a zero for the high part. + */ + tcg_out_helper_add_mov(mov, loc + HOST_BIG_ENDIAN, + TCG_TYPE_I32, TCG_TYPE_I32, + ldst->addrlo_reg, -1); + tcg_out_helper_load_slots(s, 1, mov, parm); - tcg_out_helper_load_slots(s, nmov, mov, parm); + tcg_out_helper_load_imm(s, loc[!HOST_BIG_ENDIAN].arg_slot, + TCG_TYPE_I32, 0, parm); + next_arg += 2; + } switch (info->out_kind) { case TCG_CALL_RET_NORMAL: @@ -5753,10 +5768,24 @@ static void tcg_out_st_helper_args(TCGContext *s, const TCGLabelQemuLdst *ldst, /* Handle addr argument. */ loc = &info->in[next_arg]; - n = tcg_out_helper_add_mov(mov, loc, TCG_TYPE_TL, TCG_TYPE_TL, - ldst->addrlo_reg, ldst->addrhi_reg); - next_arg += n; - nmov += n; + if (TCG_TARGET_REG_BITS == 64 || TARGET_LONG_BITS == 64) { + n = tcg_out_helper_add_mov(mov, loc, TCG_TYPE_I64, TCG_TYPE_TL, + ldst->addrlo_reg, ldst->addrhi_reg); + next_arg += n; + nmov += n; + } else { + /* + * 32-bit host with 32-bit guest: zero-extend the guest address + * to 64-bits for the helper by storing the low part. Later, + * after we have processed the register inputs, we will load a + * zero for the high part. + */ + tcg_out_helper_add_mov(mov, loc + HOST_BIG_ENDIAN, + TCG_TYPE_I32, TCG_TYPE_I32, + ldst->addrlo_reg, -1); + next_arg += 2; + nmov += 1; + } /* Handle data argument. */ loc = &info->in[next_arg]; @@ -5801,6 +5830,11 @@ static void tcg_out_st_helper_args(TCGContext *s, const TCGLabelQemuLdst *ldst, g_assert_not_reached(); } + if (TCG_TARGET_REG_BITS == 32 && TARGET_LONG_BITS == 32) { + loc = &info->in[1 + !HOST_BIG_ENDIAN]; + tcg_out_helper_load_imm(s, loc->arg_slot, TCG_TYPE_I32, 0, parm); + } + tcg_out_helper_load_common_args(s, ldst, parm, info, next_arg); } From patchwork Tue May 16 19:41:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682395 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp19562wrt; Tue, 16 May 2023 12:54:06 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ449XGKDgfq42Xoruf6mlGi71mB+XI64vK4nYMkqA+ubeBmd5B5vBKUE/AXQaJmxP03FUze X-Received: by 2002:ac8:5c0c:0:b0:3f3:9080:dfba with SMTP id i12-20020ac85c0c000000b003f39080dfbamr53591108qti.26.1684266845777; Tue, 16 May 2023 12:54:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266845; cv=none; d=google.com; s=arc-20160816; b=VAsiRZSBu1QIp+1anVORs8zc0z/hALqttq7Ty3g1BSBPqT2xMlQ5PKjJB8fgdub4X7 ED+F4T/z8KFqBVj8kNHDAkhBmMtsqzyRnbg7Q5J7Mde4clAEOeV0xUj5MpN+cEHyYmHn obBTjCv7UKvJkPY2w6bLXih7Ts7jZTCivPsaaZnn+eH/AUPk45qJBwi/7H8zB0N2JyOJ PiW/Fbs3+wPo7R5XvCe8FkpkxaJB72UxvESxZujSNn4ViSV+noPq6otLbRnwVnvPEd5j YFsMmHOxvQ4KCJZnpeZK5WjpBsU+uPnk4ZYiy6hlXWOS9XmuBKAiExs7vU9qHeIyx3wJ 3PoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=fSOV4jl3j5hzwAPTwI3x2Fzdh0+s6tsMA+VJ/6oh0dE=; b=E7dSJGjWw/kgWqMDy/77/r1Se4KPcXmY2Ebd61uAS7bf0HFe8J5PuiCd8XS5WgykwA oE2Jg2bmYiYVjP6vczsGtthaFVrfNb6b5bg1IBEcUtFJoiynF+Hk66thXbyn4pK3Cte8 NaTi4xPjVOAA337z/i2y5oFaMtB0VsWbG3WjEIQpijFlJ+2wYBgTAjys+jTg17MvUlIn Tk67Sp5QqqHekmPNvWrBrYMfRxvMm0xQ2/sJo0JJRdAKetYqLPt3ksDu2UrHrVv0D9fI 5/xcqK+xWyoU0cOfRn5TT9Vc1KUSRES/HW0IzcdX0aEuVTXEBGhRB68uvlFE3llDapeZ JA/A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="TNURn/+w"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h3-20020ac85843000000b003ef3aa3a380si11241602qth.798.2023.05.16.12.54.05 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:54:05 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="TNURn/+w"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0cW-0000le-K4; Tue, 16 May 2023 15:46:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cC-0000F8-Nm for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:46 -0400 Received: from mail-pl1-x62f.google.com ([2607:f8b0:4864:20::62f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0c8-0003kD-Fj for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:43 -0400 Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-1ae3f6e5d70so476295ad.1 for ; Tue, 16 May 2023 12:45:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266337; x=1686858337; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fSOV4jl3j5hzwAPTwI3x2Fzdh0+s6tsMA+VJ/6oh0dE=; b=TNURn/+wcOw0pA0vz/YDrAomaGJWsv0oCwqbpP5XUAh3dSj4sYkyXia8NZdzffQhE0 ObzTonhoU8vCoMc2nEYxZlCWzkQMbvzsGf+USr4QNm1+5HWUm2bxIrqmGx1o/PVPNKLs IEbJMQpSSw1FyRG/cSwWl82vWLeuWP8nnLfzsxmRCph3zdevQRbhIXCREjaAp50Yy7qL cqvF7vaKHd80kbBEZLOtCivqrUkkEuNzOjHKX5jJkDNgX0eS8Hc/LaZRQyqO3eO9DDIX eMNo08bcZWXWJ+C8nJiN/SzULs1sjMKWSAVtcCvED//VD8cGnznPU5ubDvDP1bK+Q1hg BxjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266337; x=1686858337; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fSOV4jl3j5hzwAPTwI3x2Fzdh0+s6tsMA+VJ/6oh0dE=; b=d7P8Ievak273/ZRnKt6+DxdhQdfPxMg9qO1xFA+ELXo8Mnx8P1e1hoV30EJM+f6gOo 5iDNwfuH2KrPLZPY0TEgU4dDwNQdENT0qjp1Q13NH10ty/nKXIO0RnJEF3Qw02kh6AUT BW2nRKnR/eXe9rQodzoVgkPqfemwgiOyhFkUpszHjEUzZqHCB7sZ2ZWEsSpkuM+EvGDB YkMrCTZ9yaiWiWUHfCn5G46Xq+gpymnjRW74VNTtJjWVl9SZTs+gTvK+z1OvCEo93mBP cK6QyZqf9i0jTJnbKdB7enMm0kArZA6Z0aPB2Ths+XnulIgiOA0Mq1iOJtQOqM7zOOhs IlrA== X-Gm-Message-State: AC+VfDyioKkfwshSR3HTjr7zWRiKqwx1qguKhsmGcDwEYwRhK8wagm0c vWHvEFGpTrwL2qHga5sf6zILImoKkBon3yKPSpE= X-Received: by 2002:a17:902:eccc:b0:1ac:b449:3528 with SMTP id a12-20020a170902eccc00b001acb4493528mr28624229plh.46.1684266337204; Tue, 16 May 2023 12:45:37 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:36 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 56/80] tcg: Widen helper_{ld,st}_i128 addresses to uint64_t Date: Tue, 16 May 2023 12:41:21 -0700 Message-Id: <20230516194145.1749305-57-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62f; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Always pass the target address as uint64_t. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- accel/tcg/tcg-runtime.h | 4 ++-- accel/tcg/cputlb.c | 5 ++--- accel/tcg/user-exec.c | 5 ++--- tcg/tcg-op-ldst.c | 26 ++++++++++++++++++++++++-- 4 files changed, 30 insertions(+), 10 deletions(-) diff --git a/accel/tcg/tcg-runtime.h b/accel/tcg/tcg-runtime.h index d9adc646c1..0e6c5f55fd 100644 --- a/accel/tcg/tcg-runtime.h +++ b/accel/tcg/tcg-runtime.h @@ -39,8 +39,8 @@ DEF_HELPER_FLAGS_1(exit_atomic, TCG_CALL_NO_WG, noreturn, env) DEF_HELPER_FLAGS_3(memset, TCG_CALL_NO_RWG, ptr, ptr, int, ptr) #endif /* IN_HELPER_PROTO */ -DEF_HELPER_FLAGS_3(ld_i128, TCG_CALL_NO_WG, i128, env, tl, i32) -DEF_HELPER_FLAGS_4(st_i128, TCG_CALL_NO_WG, void, env, tl, i128, i32) +DEF_HELPER_FLAGS_3(ld_i128, TCG_CALL_NO_WG, i128, env, i64, i32) +DEF_HELPER_FLAGS_4(st_i128, TCG_CALL_NO_WG, void, env, i64, i128, i32) DEF_HELPER_FLAGS_5(atomic_cmpxchgb, TCG_CALL_NO_WG, i32, env, tl, i32, i32, i32) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 5440f68deb..ae0fbcdee2 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -2551,7 +2551,7 @@ Int128 helper_ld16_mmu(CPUArchState *env, uint64_t addr, return do_ld16_mmu(env, addr, oi, retaddr); } -Int128 helper_ld_i128(CPUArchState *env, target_ulong addr, uint32_t oi) +Int128 helper_ld_i128(CPUArchState *env, uint64_t addr, uint32_t oi) { return helper_ld16_mmu(env, addr, oi, GETPC()); } @@ -3024,8 +3024,7 @@ void helper_st16_mmu(CPUArchState *env, uint64_t addr, Int128 val, do_st16_mmu(env, addr, val, oi, retaddr); } -void helper_st_i128(CPUArchState *env, target_ulong addr, Int128 val, - MemOpIdx oi) +void helper_st_i128(CPUArchState *env, uint64_t addr, Int128 val, MemOpIdx oi) { helper_st16_mmu(env, addr, val, oi, GETPC()); } diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index 9a7afb6f78..36ad8284a5 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -1136,7 +1136,7 @@ Int128 helper_ld16_mmu(CPUArchState *env, uint64_t addr, return ret; } -Int128 helper_ld_i128(CPUArchState *env, target_ulong addr, MemOpIdx oi) +Int128 helper_ld_i128(CPUArchState *env, uint64_t addr, MemOpIdx oi) { return helper_ld16_mmu(env, addr, oi, GETPC()); } @@ -1343,8 +1343,7 @@ void helper_st16_mmu(CPUArchState *env, uint64_t addr, Int128 val, do_st16_he_mmu(env, addr, val, mop, ra); } -void helper_st_i128(CPUArchState *env, target_ulong addr, - Int128 val, MemOpIdx oi) +void helper_st_i128(CPUArchState *env, uint64_t addr, Int128 val, MemOpIdx oi) { helper_st16_mmu(env, addr, val, oi, GETPC()); } diff --git a/tcg/tcg-op-ldst.c b/tcg/tcg-op-ldst.c index d8503d7da4..aab6dda454 100644 --- a/tcg/tcg-op-ldst.c +++ b/tcg/tcg-op-ldst.c @@ -393,6 +393,24 @@ static void canonicalize_memop_i128_as_i64(MemOp ret[2], MemOp orig) #define tcg_temp_ebb_new tcg_temp_ebb_new_i32 #endif +static TCGv_i64 maybe_extend_addr64(TCGv addr) +{ +#if TARGET_LONG_BITS == 32 + TCGv_i64 a64 = tcg_temp_ebb_new_i64(); + tcg_gen_extu_i32_i64(a64, addr); + return a64; +#else + return addr; +#endif +} + +static void maybe_free_addr64(TCGv_i64 a64) +{ +#if TARGET_LONG_BITS == 32 + tcg_temp_free_i64(a64); +#endif +} + void tcg_gen_qemu_ld_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) { const MemOpIdx oi = make_memop_idx(memop, idx); @@ -467,7 +485,9 @@ void tcg_gen_qemu_ld_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) tcg_gen_bswap64_i64(y, y); } } else { - gen_helper_ld_i128(val, cpu_env, addr, tcg_constant_i32(oi)); + TCGv_i64 a64 = maybe_extend_addr64(addr); + gen_helper_ld_i128(val, cpu_env, a64, tcg_constant_i32(oi)); + maybe_free_addr64(a64); } plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_R); @@ -547,7 +567,9 @@ void tcg_gen_qemu_st_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) } tcg_temp_free(addr_p8); } else { - gen_helper_st_i128(cpu_env, addr, val, tcg_constant_i32(oi)); + TCGv_i64 a64 = maybe_extend_addr64(addr); + gen_helper_st_i128(cpu_env, a64, val, tcg_constant_i32(oi)); + maybe_free_addr64(a64); } plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_W); From patchwork Tue May 16 19:41:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682394 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp19471wrt; Tue, 16 May 2023 12:53:44 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4t5VwDEN3vVvKkuG/IVUSBYrR6ATS1HJShwQXDhptOe/T5aFOhWtc/keVG2sRxDf61XUyB X-Received: by 2002:a05:622a:11cb:b0:3f3:8cf1:62ca with SMTP id n11-20020a05622a11cb00b003f38cf162camr53533864qtk.46.1684266823856; Tue, 16 May 2023 12:53:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266823; cv=none; d=google.com; s=arc-20160816; b=uageAmrfYT3dwCCt0EK4UkVZOJGs+KSdrvvLOCdZQXv3F99KfCdE36hzTjylkFpJtE mLP/QzSLmZkP7TrNXjozQvBuEcV4z4eV756xBOT0sSkOp/y8nmgKJ9RNAHdZnEIJid4U WD/VGUiPGrl0o8Ii7LaceDYcOHfwJYL1p0cidrO2TPF4MDJo7WF7QK2xLHhkAjFyUWaO /PCmKLPqxZXrweBZqLn5fh8Ht4a0ssaV9peZge1AXWz51Z1zqppg71z/P2j5kgy0O12j otBjx3nbGNdrO3moTwd1HB+8MaOBCt79SpsKvFOS9B7tA/lTUgcAfN+gYTke1OC0i2Nu 3XlQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=DvNheCiB+f6TAMadKaJTWUJqSqhItFZZi8+lYf6xl88=; b=JdXhqgyFXeTeKCKSrRdhjeGDO/qUek9XehdoTBwflSOxptu3LRk066lBakiU4m7hJw TDFQECUYcAa/EkrFujEmXlNBNMJOFFcTuyhhsTlopoEbH5msyuSVBGgq/K8GosgK/PrA PKm3+ISV13Vy+aI7SVzbeqVWfgwqjrk/NpCgyWhKnki9HaT4J3QMrz2tfmANYcnkr+Z4 jf4U6Pzw/RXI58m1FYCC1T6QAD94tVex42WidsAQ2Jjz+V0+wXEemeQrlK82k/27JjDc VxDauVFdSTdcnLmTO7j61MKtlKQ6Bz9+fu7JXAVyVTlXv8otJ2dcHgjz4yD80a5GKtEq Pf5A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=zqEgVJyK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y21-20020a37e315000000b0075953c7471csi270074qki.14.2023.05.16.12.53.43 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:53:43 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=zqEgVJyK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0eJ-0002kD-6T; Tue, 16 May 2023 15:47:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cE-0000Hb-IF for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:49 -0400 Received: from mail-pj1-x1033.google.com ([2607:f8b0:4864:20::1033]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0c8-0003kL-Hi for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:44 -0400 Received: by mail-pj1-x1033.google.com with SMTP id 98e67ed59e1d1-24e09b4153eso41968a91.2 for ; Tue, 16 May 2023 12:45:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266338; x=1686858338; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DvNheCiB+f6TAMadKaJTWUJqSqhItFZZi8+lYf6xl88=; b=zqEgVJyKhVLXhcMilDsfJCJ3VIJcRx6U3pnC4oB2j6wlWgrF5Y1cmXgYjFIYtPlL4/ s13eQnXrqQT52l5ChMN9F3j4g/vpN9LHUUhPUcFqXCpWAd3QCh3ecnvhHSu3MNjsgaIs CCAgt9DQ5YQxkv+wCc7TPXcSjAT+e19kvuqXkEEbcjV3ZYcYvwc2xoYmgvcCce0qrlsd okAY1AMLHswGyoefqgeQ08aXSKsq7x/fefII24Pfaeyd8wh2nCpxJc4tXlq1aSHyHGaW zcshqX13tuCe6mxhu8eoL20Ai9Xxe16M3teEdyS0dS6BsQJT+pjh28gSZKSIbu4Hz3RJ SRng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266338; x=1686858338; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DvNheCiB+f6TAMadKaJTWUJqSqhItFZZi8+lYf6xl88=; b=clK+L1+oYkd/YZsdAgmsFpQOVUgLQDRBOmWjjO47GRjKUwKlonQ7qTn4P9R6c+D4aE oL8MEk0pndUCQpt7SzTtYTYKb+PkNnOeT8V0vKS5OYUVRv0nq6DfYyBA28qaicb7HcQP fXY51LP1Zwc3K9BzUiPskn/kcH57VH2JuLl0LBJWb+TJI9Ba+WzMOpAyQUKF4u2Rg9es r0xONgidTlDseglkaag64xRykD8WeNUESuV5dQpIEtaWOxEnhZsLOqfHXfzIrC8It2dg NwyXPT2JpIl2/nGzYO8ACPlGIShJOdr08zEZk0kq9SKlAhI7P9aNmnmJCVvYdM9Yw8Ch kP4A== X-Gm-Message-State: AC+VfDxDwkyRnn+A3ifgP8SkfcYvdGs/7hOFviQgTrOl0QyEtSSzclOj fyJB2arAHiN+9IqKX3t81o4TwGsHjSFKun6S+Ao= X-Received: by 2002:a17:90a:e68b:b0:250:8f09:75f2 with SMTP id s11-20020a17090ae68b00b002508f0975f2mr28857938pjy.22.1684266337927; Tue, 16 May 2023 12:45:37 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:37 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 57/80] tcg: Widen helper_atomic_* addresses to uint64_t Date: Tue, 16 May 2023 12:41:22 -0700 Message-Id: <20230516194145.1749305-58-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1033; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1033.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Always pass the target address as uint64_t. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- accel/tcg/tcg-runtime.h | 46 +++++++++++++++++------------------ tcg/tcg-op-ldst.c | 38 ++++++++++++++++++++--------- accel/tcg/atomic_common.c.inc | 14 +++++------ 3 files changed, 57 insertions(+), 41 deletions(-) diff --git a/accel/tcg/tcg-runtime.h b/accel/tcg/tcg-runtime.h index 0e6c5f55fd..6f8c2061d0 100644 --- a/accel/tcg/tcg-runtime.h +++ b/accel/tcg/tcg-runtime.h @@ -43,61 +43,61 @@ DEF_HELPER_FLAGS_3(ld_i128, TCG_CALL_NO_WG, i128, env, i64, i32) DEF_HELPER_FLAGS_4(st_i128, TCG_CALL_NO_WG, void, env, i64, i128, i32) DEF_HELPER_FLAGS_5(atomic_cmpxchgb, TCG_CALL_NO_WG, - i32, env, tl, i32, i32, i32) + i32, env, i64, i32, i32, i32) DEF_HELPER_FLAGS_5(atomic_cmpxchgw_be, TCG_CALL_NO_WG, - i32, env, tl, i32, i32, i32) + i32, env, i64, i32, i32, i32) DEF_HELPER_FLAGS_5(atomic_cmpxchgw_le, TCG_CALL_NO_WG, - i32, env, tl, i32, i32, i32) + i32, env, i64, i32, i32, i32) DEF_HELPER_FLAGS_5(atomic_cmpxchgl_be, TCG_CALL_NO_WG, - i32, env, tl, i32, i32, i32) + i32, env, i64, i32, i32, i32) DEF_HELPER_FLAGS_5(atomic_cmpxchgl_le, TCG_CALL_NO_WG, - i32, env, tl, i32, i32, i32) + i32, env, i64, i32, i32, i32) #ifdef CONFIG_ATOMIC64 DEF_HELPER_FLAGS_5(atomic_cmpxchgq_be, TCG_CALL_NO_WG, - i64, env, tl, i64, i64, i32) + i64, env, i64, i64, i64, i32) DEF_HELPER_FLAGS_5(atomic_cmpxchgq_le, TCG_CALL_NO_WG, - i64, env, tl, i64, i64, i32) + i64, env, i64, i64, i64, i32) #endif #ifdef CONFIG_CMPXCHG128 DEF_HELPER_FLAGS_5(atomic_cmpxchgo_be, TCG_CALL_NO_WG, - i128, env, tl, i128, i128, i32) + i128, env, i64, i128, i128, i32) DEF_HELPER_FLAGS_5(atomic_cmpxchgo_le, TCG_CALL_NO_WG, - i128, env, tl, i128, i128, i32) + i128, env, i64, i128, i128, i32) #endif DEF_HELPER_FLAGS_5(nonatomic_cmpxchgo_be, TCG_CALL_NO_WG, - i128, env, tl, i128, i128, i32) + i128, env, i64, i128, i128, i32) DEF_HELPER_FLAGS_5(nonatomic_cmpxchgo_le, TCG_CALL_NO_WG, - i128, env, tl, i128, i128, i32) + i128, env, i64, i128, i128, i32) #ifdef CONFIG_ATOMIC64 #define GEN_ATOMIC_HELPERS(NAME) \ DEF_HELPER_FLAGS_4(glue(glue(atomic_, NAME), b), \ - TCG_CALL_NO_WG, i32, env, tl, i32, i32) \ + TCG_CALL_NO_WG, i32, env, i64, i32, i32) \ DEF_HELPER_FLAGS_4(glue(glue(atomic_, NAME), w_le), \ - TCG_CALL_NO_WG, i32, env, tl, i32, i32) \ + TCG_CALL_NO_WG, i32, env, i64, i32, i32) \ DEF_HELPER_FLAGS_4(glue(glue(atomic_, NAME), w_be), \ - TCG_CALL_NO_WG, i32, env, tl, i32, i32) \ + TCG_CALL_NO_WG, i32, env, i64, i32, i32) \ DEF_HELPER_FLAGS_4(glue(glue(atomic_, NAME), l_le), \ - TCG_CALL_NO_WG, i32, env, tl, i32, i32) \ + TCG_CALL_NO_WG, i32, env, i64, i32, i32) \ DEF_HELPER_FLAGS_4(glue(glue(atomic_, NAME), l_be), \ - TCG_CALL_NO_WG, i32, env, tl, i32, i32) \ + TCG_CALL_NO_WG, i32, env, i64, i32, i32) \ DEF_HELPER_FLAGS_4(glue(glue(atomic_, NAME), q_le), \ - TCG_CALL_NO_WG, i64, env, tl, i64, i32) \ + TCG_CALL_NO_WG, i64, env, i64, i64, i32) \ DEF_HELPER_FLAGS_4(glue(glue(atomic_, NAME), q_be), \ - TCG_CALL_NO_WG, i64, env, tl, i64, i32) + TCG_CALL_NO_WG, i64, env, i64, i64, i32) #else #define GEN_ATOMIC_HELPERS(NAME) \ DEF_HELPER_FLAGS_4(glue(glue(atomic_, NAME), b), \ - TCG_CALL_NO_WG, i32, env, tl, i32, i32) \ + TCG_CALL_NO_WG, i32, env, i64, i32, i32) \ DEF_HELPER_FLAGS_4(glue(glue(atomic_, NAME), w_le), \ - TCG_CALL_NO_WG, i32, env, tl, i32, i32) \ + TCG_CALL_NO_WG, i32, env, i64, i32, i32) \ DEF_HELPER_FLAGS_4(glue(glue(atomic_, NAME), w_be), \ - TCG_CALL_NO_WG, i32, env, tl, i32, i32) \ + TCG_CALL_NO_WG, i32, env, i64, i32, i32) \ DEF_HELPER_FLAGS_4(glue(glue(atomic_, NAME), l_le), \ - TCG_CALL_NO_WG, i32, env, tl, i32, i32) \ + TCG_CALL_NO_WG, i32, env, i64, i32, i32) \ DEF_HELPER_FLAGS_4(glue(glue(atomic_, NAME), l_be), \ - TCG_CALL_NO_WG, i32, env, tl, i32, i32) + TCG_CALL_NO_WG, i32, env, i64, i32, i32) #endif /* CONFIG_ATOMIC64 */ GEN_ATOMIC_HELPERS(fetch_add) diff --git a/tcg/tcg-op-ldst.c b/tcg/tcg-op-ldst.c index aab6dda454..ca57a2779d 100644 --- a/tcg/tcg-op-ldst.c +++ b/tcg/tcg-op-ldst.c @@ -623,15 +623,15 @@ static void tcg_gen_ext_i64(TCGv_i64 ret, TCGv_i64 val, MemOp opc) } } -typedef void (*gen_atomic_cx_i32)(TCGv_i32, TCGv_env, TCGv, +typedef void (*gen_atomic_cx_i32)(TCGv_i32, TCGv_env, TCGv_i64, TCGv_i32, TCGv_i32, TCGv_i32); -typedef void (*gen_atomic_cx_i64)(TCGv_i64, TCGv_env, TCGv, +typedef void (*gen_atomic_cx_i64)(TCGv_i64, TCGv_env, TCGv_i64, TCGv_i64, TCGv_i64, TCGv_i32); -typedef void (*gen_atomic_cx_i128)(TCGv_i128, TCGv_env, TCGv, +typedef void (*gen_atomic_cx_i128)(TCGv_i128, TCGv_env, TCGv_i64, TCGv_i128, TCGv_i128, TCGv_i32); -typedef void (*gen_atomic_op_i32)(TCGv_i32, TCGv_env, TCGv, +typedef void (*gen_atomic_op_i32)(TCGv_i32, TCGv_env, TCGv_i64, TCGv_i32, TCGv_i32); -typedef void (*gen_atomic_op_i64)(TCGv_i64, TCGv_env, TCGv, +typedef void (*gen_atomic_op_i64)(TCGv_i64, TCGv_env, TCGv_i64, TCGv_i64, TCGv_i32); #ifdef CONFIG_ATOMIC64 @@ -682,6 +682,7 @@ void tcg_gen_atomic_cmpxchg_i32(TCGv_i32 retv, TCGv addr, TCGv_i32 cmpv, TCGv_i32 newv, TCGArg idx, MemOp memop) { gen_atomic_cx_i32 gen; + TCGv_i64 a64; MemOpIdx oi; if (!(tcg_ctx->gen_tb->cflags & CF_PARALLEL)) { @@ -694,7 +695,9 @@ void tcg_gen_atomic_cmpxchg_i32(TCGv_i32 retv, TCGv addr, TCGv_i32 cmpv, tcg_debug_assert(gen != NULL); oi = make_memop_idx(memop & ~MO_SIGN, idx); - gen(retv, cpu_env, addr, cmpv, newv, tcg_constant_i32(oi)); + a64 = maybe_extend_addr64(addr); + gen(retv, cpu_env, a64, cmpv, newv, tcg_constant_i32(oi)); + maybe_free_addr64(a64); if (memop & MO_SIGN) { tcg_gen_ext_i32(retv, retv, memop); @@ -750,7 +753,9 @@ void tcg_gen_atomic_cmpxchg_i64(TCGv_i64 retv, TCGv addr, TCGv_i64 cmpv, gen = table_cmpxchg[memop & (MO_SIZE | MO_BSWAP)]; if (gen) { MemOpIdx oi = make_memop_idx(memop, idx); - gen(retv, cpu_env, addr, cmpv, newv, tcg_constant_i32(oi)); + TCGv_i64 a64 = maybe_extend_addr64(addr); + gen(retv, cpu_env, a64, cmpv, newv, tcg_constant_i32(oi)); + maybe_free_addr64(a64); return; } @@ -802,11 +807,14 @@ void tcg_gen_nonatomic_cmpxchg_i128(TCGv_i128 retv, TCGv addr, TCGv_i128 cmpv, ? gen_helper_nonatomic_cmpxchgo_le : gen_helper_nonatomic_cmpxchgo_be); MemOpIdx oi = make_memop_idx(memop, idx); + TCGv_i64 a64; tcg_debug_assert((memop & MO_SIZE) == MO_128); tcg_debug_assert((memop & MO_SIGN) == 0); - gen(retv, cpu_env, addr, cmpv, newv, tcg_constant_i32(oi)); + a64 = maybe_extend_addr64(addr); + gen(retv, cpu_env, a64, cmpv, newv, tcg_constant_i32(oi)); + maybe_free_addr64(a64); } else { TCGv_i128 oldv = tcg_temp_ebb_new_i128(); TCGv_i128 tmpv = tcg_temp_ebb_new_i128(); @@ -854,7 +862,9 @@ void tcg_gen_atomic_cmpxchg_i128(TCGv_i128 retv, TCGv addr, TCGv_i128 cmpv, if (gen) { MemOpIdx oi = make_memop_idx(memop, idx); - gen(retv, cpu_env, addr, cmpv, newv, tcg_constant_i32(oi)); + TCGv_i64 a64 = maybe_extend_addr64(addr); + gen(retv, cpu_env, a64, cmpv, newv, tcg_constant_i32(oi)); + maybe_free_addr64(a64); return; } @@ -892,6 +902,7 @@ static void do_atomic_op_i32(TCGv_i32 ret, TCGv addr, TCGv_i32 val, TCGArg idx, MemOp memop, void * const table[]) { gen_atomic_op_i32 gen; + TCGv_i64 a64; MemOpIdx oi; memop = tcg_canonicalize_memop(memop, 0, 0); @@ -900,7 +911,9 @@ static void do_atomic_op_i32(TCGv_i32 ret, TCGv addr, TCGv_i32 val, tcg_debug_assert(gen != NULL); oi = make_memop_idx(memop & ~MO_SIGN, idx); - gen(ret, cpu_env, addr, val, tcg_constant_i32(oi)); + a64 = maybe_extend_addr64(addr); + gen(ret, cpu_env, a64, val, tcg_constant_i32(oi)); + maybe_free_addr64(a64); if (memop & MO_SIGN) { tcg_gen_ext_i32(ret, ret, memop); @@ -934,13 +947,16 @@ static void do_atomic_op_i64(TCGv_i64 ret, TCGv addr, TCGv_i64 val, if ((memop & MO_SIZE) == MO_64) { #ifdef CONFIG_ATOMIC64 gen_atomic_op_i64 gen; + TCGv_i64 a64; MemOpIdx oi; gen = table[memop & (MO_SIZE | MO_BSWAP)]; tcg_debug_assert(gen != NULL); oi = make_memop_idx(memop & ~MO_SIGN, idx); - gen(ret, cpu_env, addr, val, tcg_constant_i32(oi)); + a64 = maybe_extend_addr64(addr); + gen(ret, cpu_env, a64, val, tcg_constant_i32(oi)); + maybe_free_addr64(a64); #else gen_helper_exit_atomic(cpu_env); /* Produce a result, so that we have a well-formed opcode stream diff --git a/accel/tcg/atomic_common.c.inc b/accel/tcg/atomic_common.c.inc index 8f2ce43ee6..fe0eea018f 100644 --- a/accel/tcg/atomic_common.c.inc +++ b/accel/tcg/atomic_common.c.inc @@ -13,20 +13,20 @@ * See the COPYING file in the top-level directory. */ -static void atomic_trace_rmw_post(CPUArchState *env, target_ulong addr, +static void atomic_trace_rmw_post(CPUArchState *env, uint64_t addr, MemOpIdx oi) { qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_RW); } #if HAVE_ATOMIC128 -static void atomic_trace_ld_post(CPUArchState *env, target_ulong addr, +static void atomic_trace_ld_post(CPUArchState *env, uint64_t addr, MemOpIdx oi) { qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); } -static void atomic_trace_st_post(CPUArchState *env, target_ulong addr, +static void atomic_trace_st_post(CPUArchState *env, uint64_t addr, MemOpIdx oi) { qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); @@ -40,7 +40,7 @@ static void atomic_trace_st_post(CPUArchState *env, target_ulong addr, */ #define CMPXCHG_HELPER(OP, TYPE) \ - TYPE HELPER(atomic_##OP)(CPUArchState *env, target_ulong addr, \ + TYPE HELPER(atomic_##OP)(CPUArchState *env, uint64_t addr, \ TYPE oldv, TYPE newv, uint32_t oi) \ { return cpu_atomic_##OP##_mmu(env, addr, oldv, newv, oi, GETPC()); } @@ -62,7 +62,7 @@ CMPXCHG_HELPER(cmpxchgo_le, Int128) #undef CMPXCHG_HELPER -Int128 HELPER(nonatomic_cmpxchgo_be)(CPUArchState *env, target_ulong addr, +Int128 HELPER(nonatomic_cmpxchgo_be)(CPUArchState *env, uint64_t addr, Int128 cmpv, Int128 newv, uint32_t oi) { #if TCG_TARGET_REG_BITS == 32 @@ -82,7 +82,7 @@ Int128 HELPER(nonatomic_cmpxchgo_be)(CPUArchState *env, target_ulong addr, #endif } -Int128 HELPER(nonatomic_cmpxchgo_le)(CPUArchState *env, target_ulong addr, +Int128 HELPER(nonatomic_cmpxchgo_le)(CPUArchState *env, uint64_t addr, Int128 cmpv, Int128 newv, uint32_t oi) { #if TCG_TARGET_REG_BITS == 32 @@ -103,7 +103,7 @@ Int128 HELPER(nonatomic_cmpxchgo_le)(CPUArchState *env, target_ulong addr, } #define ATOMIC_HELPER(OP, TYPE) \ - TYPE HELPER(glue(atomic_,OP))(CPUArchState *env, target_ulong addr, \ + TYPE HELPER(glue(atomic_,OP))(CPUArchState *env, uint64_t addr, \ TYPE val, uint32_t oi) \ { return glue(glue(cpu_atomic_,OP),_mmu)(env, addr, val, oi, GETPC()); } From patchwork Tue May 16 19:41:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682397 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp19621wrt; Tue, 16 May 2023 12:54:14 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7OUZgLqvvQq21UuxXaiOdDEDo+eUFwrRZfKnbH3u2HdiHtjLyUFdMV7NgK394hfOnB41xO X-Received: by 2002:a05:6214:528b:b0:5ef:50a3:f9ca with SMTP id kj11-20020a056214528b00b005ef50a3f9camr53621570qvb.47.1684266854537; Tue, 16 May 2023 12:54:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266854; cv=none; d=google.com; s=arc-20160816; b=odcwWs3V+sZFXOW4qY6a7VqwR6jRNGP8XQ+S9tBrAPcdDokajdBXqLnXG5qUyGDzgc hWDAXjo+dFq9QWwwHmEfOjonlkR29ZVHSHavq+EDMyiG0imSi2XoHe8n12Qja5cqlzis E2ugaknKeixDr412vSUR0sdZ45ndLpH3qT22dhQq/p9ZcQqGR7whl/KYuk6Wwt5pGtCK 3y4pVBAmYrtGhtM1ENkuC8ft1kqBwILj7d1EH/hPFFuXp2pyRvi7pzbDvMiU6Zt48YDR ZjGnbfAJkq1Q+ALmG6IqHqLInyDgnS1pFRbbXvboxx+/EvA1voP0N9xnSTKbM5ZToWEP XX6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=LtDld2gFk+pS05MnfqGIihYYTTfEqLTj2+RJM8YOMsA=; b=OVKtiV8YZ9n2HxYhkBZcYUClyo+/ioQJojSZMcTSrRGGQL/rFH5MFQi9ZmTldABzeA a+N3b/04w2EMetQetKjae2W2cX2Yt187dPd7BBhZSSSjgwpxKmKH2qXXaEqP7HCxWJL9 Ynmn5sOxN6iKlXPTwgSzNqNr+pCa2BXKaVvTJ3zZ+Qq24+n7eci80G3A08Ib5aDXDKtC t+JgS5/yxXKHEPUhyRNqp7ts/EpW6GDy18yr6dY4Z5MOeSqHvx8uK2TgI5SwMpgL9BEj bwNZZERBrOUfBz+UkXoYijlRNXHVh+i7aWtiDC0yVsxbX1tXWUaPv6nulxgc3AqZAMe+ OBJw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=FTaIJf7b; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id e17-20020a05620a12d100b007467fa3ff23si230531qkl.352.2023.05.16.12.54.14 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:54:14 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=FTaIJf7b; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0eR-0002qH-Kg; Tue, 16 May 2023 15:48:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cE-0000HZ-IG for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:49 -0400 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cA-0003kY-Dd for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:45 -0400 Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-1aaff9c93a5so433675ad.2 for ; Tue, 16 May 2023 12:45:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266339; x=1686858339; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LtDld2gFk+pS05MnfqGIihYYTTfEqLTj2+RJM8YOMsA=; b=FTaIJf7bNpCRrBz0i6HQSxKfZPSKGA1lFQd60j0qTIIQPvklnkoWwHmWWZRuGet2XV Cwv1iuckqLtKhg4vJ/IO4eaw+g1KcAKcCDqS0mLji94rlNbhxP+EtSe/Y9qFpJO4jzSX 3ACQdMn/HFwUSq+VIopmKpIdsIvRL2W6+dYrIUlCdgWAWOe1zHmm8GYCRLMHjjIxIKo5 IO2XbeI5k/pcizPtxsGaykdPis5pMNfUDIXCGeBOis6gRs4MLHRpZEUUlTR+atK4vttp 3urFgdSRZBKTVOJ+7/2jkN+KA/HD/dPcALF+g4GFO98UFUQ27NLmKp9nb+70Q+OCXyXH sqyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266339; x=1686858339; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LtDld2gFk+pS05MnfqGIihYYTTfEqLTj2+RJM8YOMsA=; b=kM0kGNkXzOu7zD89RTS/zxC92BMEr4eY/K+MntH1bYSxbBlsTFqI2Ug7S7/Fp6hzSx W5tMI9eDWlgPZF1PkluG+ZAY6lxLXQfLwQv/+agx0TSDmswe3jTG0kKiVPwE1nDxG21D TRAy1JwqykjwDq4QvYWvIn2ZQKLrhpelUZeHRZV4QWhhqpOmXS3gg20+luOccEthvlaA FRB4QCd14zegWIcxCgVJJ7lS+ixdGDJUmF0yA8DkbGj4WUw+GIgrt1VCM1IzLce09lyL QkNuVyUyxkFLfJBKoS84RYx8e2aRYQTyPaSeb707+tuyoIsz8BKSkS/o+U3y+BhiFB4/ TSeg== X-Gm-Message-State: AC+VfDzK756xgyeJMFjI96JPZRPI0kEMYE6D5vF279ps8UJXCVUA7Xm8 aOU0BbuWCvS7LsO/oduubNthnZT0kJmmFM+OjnE= X-Received: by 2002:a17:902:6ac3:b0:1ac:93ba:a5ec with SMTP id i3-20020a1709026ac300b001ac93baa5ecmr24773619plt.52.1684266338805; Tue, 16 May 2023 12:45:38 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:38 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 58/80] tcg: Widen tcg_gen_code pc_start argument to uint64_t Date: Tue, 16 May 2023 12:41:23 -0700 Message-Id: <20230516194145.1749305-59-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::634; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x634.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- include/tcg/tcg.h | 2 +- tcg/tcg.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h index f40de4177d..42a3c05607 100644 --- a/include/tcg/tcg.h +++ b/include/tcg/tcg.h @@ -852,7 +852,7 @@ void tcg_register_thread(void); void tcg_prologue_init(TCGContext *s); void tcg_func_start(TCGContext *s); -int tcg_gen_code(TCGContext *s, TranslationBlock *tb, target_ulong pc_start); +int tcg_gen_code(TCGContext *s, TranslationBlock *tb, uint64_t pc_start); void tb_target_set_jmp_target(const TranslationBlock *, int, uintptr_t, uintptr_t); diff --git a/tcg/tcg.c b/tcg/tcg.c index cdd194639e..800dcba159 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -5947,7 +5947,7 @@ int64_t tcg_cpu_exec_time(void) #endif -int tcg_gen_code(TCGContext *s, TranslationBlock *tb, target_ulong pc_start) +int tcg_gen_code(TCGContext *s, TranslationBlock *tb, uint64_t pc_start) { #ifdef CONFIG_PROFILER TCGProfile *prof = &s->prof; From patchwork Tue May 16 19:41:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682412 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp20565wrt; Tue, 16 May 2023 12:57:37 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ54Z3Zu9ZwLBB2xiFLZYxl841U4DLLBpi+6jD1uS9qU/hb/rskB+n4RJoo3ufToiw2YzeVV X-Received: by 2002:a5d:4004:0:b0:2c5:3cd2:b8e with SMTP id n4-20020a5d4004000000b002c53cd20b8emr27293838wrp.1.1684267057639; Tue, 16 May 2023 12:57:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267057; cv=none; d=google.com; s=arc-20160816; b=pAdf0LmEvFM8z6StH9yJnAkaETmvEcUWj5YYlGhD4hz4XTpjBcmbFBPZuphMc1N8tO hrWz393uveBFvUUwJpiznMvAgK16QvTLchknNiDpUIXPbhWm/HcDecMPOB+fVrUK94N/ Pi0FZ2/efZk7dzeRWvIoVg1ZTCUHBBunzUNVB8BYq/S8zjURVZDUlV39J66WvHznySue nvScGKAH7CmRTfrvZZHf+83t+9huj7EWZ22+VHgTbYIto5V1QTHXZkwoWC8h1AyHtQ8q pb8FXcBUDBLh2gwNilEQdaQX9Cs/5cwYlFN7oLDeTeE6cYkG7UOGh9Y8W9/2TX6+t/Jo ZWtg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=4fVGxJLZnm/9x1zczI0K5yPA644Oq34wOtBnW72R26I=; b=ACAcaMEAZjI345mjCRSOQUcq0jn7e1ZdYvyC5mNXgV4vlcgSXcCA3z4pEKQYQ1mSCP pc/0cEX31icDxtcacSsbrU8fHcAlZokyrshuOtNOWldTzXr60WqpwzhZyXe8B7PKks8Z xNkPPAj+cOlZZa9sM3iYvdi88UCbVSLmlfhMv383QhmVRWiTeMkUqX/PVRwJ/CSI118Y Y7IdLSY2WwPgArKf0AbFY3xGNG4YWwQ9Yr+btG9iAFow3az5uRP8oBYV0SDhuVCXhlex EHPoBQMFvODn5Vp//cV37s0cIxDHCDe9j9dgB1ipIIGifcLfTY5Bx3JyUNVXSNGK3Riy njEA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=BPCrYin+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z17-20020adfec91000000b003075e827333si140398wrn.1023.2023.05.16.12.57.37 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:57:37 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=BPCrYin+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0eM-0002mb-Vt; Tue, 16 May 2023 15:47:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cI-0000Lm-ED for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:52 -0400 Received: from mail-pf1-x42e.google.com ([2607:f8b0:4864:20::42e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cC-0003kg-32 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:48 -0400 Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-64ab2a37812so8129768b3a.1 for ; Tue, 16 May 2023 12:45:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266339; x=1686858339; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4fVGxJLZnm/9x1zczI0K5yPA644Oq34wOtBnW72R26I=; b=BPCrYin+SEGQkYn+HsOVDVVfjGfmjuVns6WVGhROOvX8bruyLMAAVLpwYd5ev2veZZ XsXdvh5L6Dg8Aws5JX0E8TWXKyEzlziUEKr6wCWzFW/9e6im4Jh3IOPbvgPUpN+KUB/G dNpTCRT3vECSvXKI93sGwURUcjCDN7HcOzKc+ZRm5UnyC70Q1r/qXQpViQQeAlCrO/GK iDzcY4HhWSmDMo1bmcAMywXYBtn/HgICcPvzv+RVCwJhgbsoB+2yoIn+U7pogtGefpUB i8YofOGQwQ42lG76k2yU0hDtw7GL46SHr/1955WXIrl0rv21uKDiigmyO/tgMBJYFVmi 6i6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266339; x=1686858339; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4fVGxJLZnm/9x1zczI0K5yPA644Oq34wOtBnW72R26I=; b=I87uvfQYDhU5/Y1kkmsw6GA8V+r3368Y/8DWhcnyHILHJGPWPgSEdYQCBKcCU4xjHI FrN1Md/XfotlmzwlPwSnqAh6wHPOIUSj0re4EZr/fAWjL1m2zhRAxD6GHlBgaSXSrgY9 0R1fo4DQuQJ6FvKSsZw6s2Tib3r6bx8IzhZY0XxkihT8lCJcN+zkiVuXgffpJxyI9FSy +ha+GZuwPUs1g8Xv07Df7tr8eXBQmjjRDF159uq9N9PAhWGxAJW3Jq9Fq/scOQRs6mTV ekOTwz1tvHA74V31lNkEG6kVE6LwcLF5UUs7xEzYMnFyr2mfcyVgbq7oytAjfeM252CL fgOw== X-Gm-Message-State: AC+VfDzK63dP9rD/PnHhuHEorIxlmJX0cPF/f+4X7w99Tys0rfYUNClG JDkXE19i481GPpnDdMzf/1VzRj2l54EJGncfVis= X-Received: by 2002:a17:903:1208:b0:1aa:d235:6dd4 with SMTP id l8-20020a170903120800b001aad2356dd4mr50774163plh.19.1684266339710; Tue, 16 May 2023 12:45:39 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:39 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 59/80] accel/tcg: Merge gen_mem_wrapped with plugin_gen_empty_mem_callback Date: Tue, 16 May 2023 12:41:24 -0700 Message-Id: <20230516194145.1749305-60-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42e; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org As gen_mem_wrapped is only used in plugin_gen_empty_mem_callback, we can avoid the curiosity of union mem_gen_fn by inlining it. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- accel/tcg/plugin-gen.c | 30 ++++++------------------------ 1 file changed, 6 insertions(+), 24 deletions(-) diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c index 5efb8db258..04facd6305 100644 --- a/accel/tcg/plugin-gen.c +++ b/accel/tcg/plugin-gen.c @@ -202,35 +202,17 @@ static void plugin_gen_empty_callback(enum plugin_gen_from from) } } -union mem_gen_fn { - void (*mem_fn)(TCGv, uint32_t); - void (*inline_fn)(void); -}; - -static void gen_mem_wrapped(enum plugin_gen_cb type, - const union mem_gen_fn *f, TCGv addr, - uint32_t info, bool is_mem) +void plugin_gen_empty_mem_callback(TCGv addr, uint32_t info) { enum qemu_plugin_mem_rw rw = get_plugin_meminfo_rw(info); - gen_plugin_cb_start(PLUGIN_GEN_FROM_MEM, type, rw); - if (is_mem) { - f->mem_fn(addr, info); - } else { - f->inline_fn(); - } + gen_plugin_cb_start(PLUGIN_GEN_FROM_MEM, PLUGIN_GEN_CB_MEM, rw); + gen_empty_mem_cb(addr, info); tcg_gen_plugin_cb_end(); -} -void plugin_gen_empty_mem_callback(TCGv addr, uint32_t info) -{ - union mem_gen_fn fn; - - fn.mem_fn = gen_empty_mem_cb; - gen_mem_wrapped(PLUGIN_GEN_CB_MEM, &fn, addr, info, true); - - fn.inline_fn = gen_empty_inline_cb; - gen_mem_wrapped(PLUGIN_GEN_CB_INLINE, &fn, 0, info, false); + gen_plugin_cb_start(PLUGIN_GEN_FROM_MEM, PLUGIN_GEN_CB_INLINE, rw); + gen_empty_inline_cb(); + tcg_gen_plugin_cb_end(); } static TCGOp *find_op(TCGOp *op, TCGOpcode opc) From patchwork Tue May 16 19:41:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682420 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp21059wrt; Tue, 16 May 2023 12:59:29 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ60wAjIXAFjkG88Hatoxd+yl5NBYv5NVjubG5hV0mIbrS0wlRUVScy1PbnE5RvsVRL1ooWQ X-Received: by 2002:a05:600c:cf:b0:3f4:29c2:1cd0 with SMTP id u15-20020a05600c00cf00b003f429c21cd0mr18176038wmm.25.1684267168979; Tue, 16 May 2023 12:59:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267168; cv=none; d=google.com; s=arc-20160816; b=WrR/pOnA3hEI1YPbd+aHdNTC2Ho+/P+1veMVxU/x6KEEZ1f/6E4OIFtZYediECTA+B B2YUU5UK1tv+mYyf04v+f1eIKD39HiZUcRdRoEsDSD1Xk1EtoEgyxA97+Nded0/L8lMe JOCDL4sQi0bwy3yRe+01QVhVLf8MEpY2zP2NZ6JGwTL2vC1Ez2qyA7wu7jB1gbqY55ua o32kC3weID6rK/6LiCQ/Cme4W222yNdisZJc16fbWa7+rRnAaJboMeVNp7AUJz+WWo+p jYeDMwWCZnCraFlHc7iPXepuOK7wFColNz2baKDaJwksHtms77y1dVMUgIwv9+qhLtYp yDxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=bikJAOfrqavvtaG/OvbZaIR8VQshwzSewGm6mEpCNLg=; b=g/lHNn6CmX0OZVhTdLCkjj66KgKYFEYFsNc7H+pkCPsWxt7NzJ6uDB6L90cpi+sb0A hlifgufE0kx6Zw9JWq0keWEY+NSpbBLWBEdMa+jRu4tq8A1/wVB6+MM3HK2qJsNGNfjZ xoFcq9ESdukFRFmZx9IN10geWj4T4Ti4Z0lh0TWwuxU1OMebIqq+mnUNI7zVIGwYo17z r9t1hHdk+0JQ918SYgF0p6WTQ07hBr+aMaPBmieKL2t5RUDZ9pKKvIMBbMzlK2derNZH sYY39IDK2Dpt6K7NnOi/7TUQQu5gQz32c14AEGF3Uz8RMI+sMm5Grd9g6rZma+J25b1W rdZg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qymJs5hF; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l17-20020a05600c1d1100b003f42792e5fesi183942wms.9.2023.05.16.12.59.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:59:28 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qymJs5hF; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0eK-0002lb-GT; Tue, 16 May 2023 15:47:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cI-0000Lg-F7 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:52 -0400 Received: from mail-pj1-x1035.google.com ([2607:f8b0:4864:20::1035]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cC-0003ks-BJ for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:46 -0400 Received: by mail-pj1-x1035.google.com with SMTP id 98e67ed59e1d1-24e1d272b09so49421a91.1 for ; Tue, 16 May 2023 12:45:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266340; x=1686858340; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bikJAOfrqavvtaG/OvbZaIR8VQshwzSewGm6mEpCNLg=; b=qymJs5hFQcBNk7pgtlWLdEnvMh88D878yzujIu2US9RswMaMfdIsGG7O5pbJd4BXSx 2j8W7ns3Pr/FD7bmJqR3ATH/5qYpJYjU4OHOdnjYTg0r+2wdpQ+9HvpUzg6NkoeyJrU6 yWAg0g5BD8PyI3STTkyCywvCxMtO3G7lK1SIxT3IhEn5w4jPW/cg731/NAMU1RGH0m9D x9+EUEG2B/TaFUIpcEm6+F2CTwKxB0qm+KDN56cze3T84l+SuvApWPpmSFGFGYEXUtTN 17dXCMrlcT/tjx05eAZ6GS9zaC8Z6CSh3vM6ReviZRxLdAglDbKAd7SXTvLiH+MxBoJK qbug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266340; x=1686858340; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bikJAOfrqavvtaG/OvbZaIR8VQshwzSewGm6mEpCNLg=; b=SUQijEXW1WkyTwCZRwux9NkQgFj2HDTWLgOqMOsl69C/OcPB6gIQNjA7BmUdgN7VYT CLiXCiXNhciEUyS/nn9QRMgn/HMUhitQLrPu2eb5bU3oXX6rpO7dsLPd1gYjpFcyfnoP GoY9F1Ks9WjPs/96SQYeM0+TiLV3wPhQbl5XZV+N8uxqohNRh8hbodUO5Jy/D0BTXfWe dnNZE5yB5IzF87EL37VQ1DX7Nx2ONdLsCxykYJVAq4yWgp5vVTq/kOwbQLqu8bRhbJ8r XkPOROkXl+IijC2BJyjTLdH7tco1dWbt4ROMZN1fJetpKUXxysP0KvuCV54OFD/oQ4BT eRtQ== X-Gm-Message-State: AC+VfDxEDEMoj2lQ0zsbSXUSlwsDmn1CMBAyIKAJKh/j5q60Bw5QtXUv JviD8Q38AH0njL6Gebzoknsc2jwXmSbQyYSZiL0= X-Received: by 2002:a17:90a:d24f:b0:24e:4a1a:39a4 with SMTP id o15-20020a17090ad24f00b0024e4a1a39a4mr39553102pjw.17.1684266340481; Tue, 16 May 2023 12:45:40 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:40 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 60/80] accel/tcg: Merge do_gen_mem_cb into caller Date: Tue, 16 May 2023 12:41:25 -0700 Message-Id: <20230516194145.1749305-61-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1035; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1035.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org As do_gen_mem_cb is called once, merge it into gen_empty_mem_cb. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- accel/tcg/plugin-gen.c | 39 +++++++++++++++++---------------------- 1 file changed, 17 insertions(+), 22 deletions(-) diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c index 04facd6305..907c5004a4 100644 --- a/accel/tcg/plugin-gen.c +++ b/accel/tcg/plugin-gen.c @@ -92,27 +92,6 @@ void HELPER(plugin_vcpu_mem_cb)(unsigned int vcpu_index, void *userdata) { } -static void do_gen_mem_cb(TCGv vaddr, uint32_t info) -{ - TCGv_i32 cpu_index = tcg_temp_ebb_new_i32(); - TCGv_i32 meminfo = tcg_temp_ebb_new_i32(); - TCGv_i64 vaddr64 = tcg_temp_ebb_new_i64(); - TCGv_ptr udata = tcg_temp_ebb_new_ptr(); - - tcg_gen_movi_i32(meminfo, info); - tcg_gen_movi_ptr(udata, 0); - tcg_gen_ld_i32(cpu_index, cpu_env, - -offsetof(ArchCPU, env) + offsetof(CPUState, cpu_index)); - tcg_gen_extu_tl_i64(vaddr64, vaddr); - - gen_helper_plugin_vcpu_mem_cb(cpu_index, meminfo, vaddr64, udata); - - tcg_temp_free_ptr(udata); - tcg_temp_free_i64(vaddr64); - tcg_temp_free_i32(meminfo); - tcg_temp_free_i32(cpu_index); -} - static void gen_empty_udata_cb(void) { TCGv_i32 cpu_index = tcg_temp_ebb_new_i32(); @@ -147,7 +126,23 @@ static void gen_empty_inline_cb(void) static void gen_empty_mem_cb(TCGv addr, uint32_t info) { - do_gen_mem_cb(addr, info); + TCGv_i32 cpu_index = tcg_temp_ebb_new_i32(); + TCGv_i32 meminfo = tcg_temp_ebb_new_i32(); + TCGv_i64 addr64 = tcg_temp_ebb_new_i64(); + TCGv_ptr udata = tcg_temp_ebb_new_ptr(); + + tcg_gen_movi_i32(meminfo, info); + tcg_gen_movi_ptr(udata, 0); + tcg_gen_ld_i32(cpu_index, cpu_env, + -offsetof(ArchCPU, env) + offsetof(CPUState, cpu_index)); + tcg_gen_extu_tl_i64(addr64, addr); + + gen_helper_plugin_vcpu_mem_cb(cpu_index, meminfo, addr64, udata); + + tcg_temp_free_ptr(udata); + tcg_temp_free_i64(addr64); + tcg_temp_free_i32(meminfo); + tcg_temp_free_i32(cpu_index); } /* From patchwork Tue May 16 19:41:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682399 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp19758wrt; Tue, 16 May 2023 12:54:42 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ69uhScrTUdmtIj5J4VvD2iGA8kkFrZ8XqF0zQrNM+YXXFiiLdRAzdmanS64BnIyzSRxu1B X-Received: by 2002:a05:6214:628:b0:5ef:4254:d6f0 with SMTP id a8-20020a056214062800b005ef4254d6f0mr66640084qvx.36.1684266881773; Tue, 16 May 2023 12:54:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266881; cv=none; d=google.com; s=arc-20160816; b=VLNTAizD5qetgDMofvp//06xNQxz0T5HDpsMr6KMJhW7qVNEBYo0A1TnEjudKTrQIr EqKd2yKzwTadpY44UIml+3da7ca9GzyG153C4Ba0wR63u9cJEiDIsLULn1tQvykODyzE 1h2XquhTgNNlB//etRrxq/DgWLi7G0sDvRgUSl2sUY6dZmLHtXYlKEPMsaoQuTUD613q LAGXg5uQUhoirBhc1NcjrDAyGk55i3NA9yDxmh1OGmsp0wQyBG30I/5OejYlwP1jXoLS Q1xa00DQbGVhJTMdPoFoqTUS2YdObVfVbbpyvF1vNa+fvk7Ao0KmBCNi3u1xTKDT+bBk kCxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=wuME4Q0uyj8eqINyik0cwsf020/cK1vI3bJfJHG03XY=; b=rvevUEJbkGsgu0oRGF207yMw3I/CqVEmE/eSmpqZwpdnnSPre8/W2sJ1YShNXW2UbZ PSXvZLpFGZRlNte6vzcgJogEifygmvqOraJtXRYoGatcXj6Ckt+HKE/TNc2WUbFPQDVR fMY/3HB+5gk/0E00MLYNyXaTfCb2sIMCaiSmschH/4QK3NuFmNjlRI3YHybYLESNzZCz Ow+Ll/bBl25KSqqSudw2b+VFLI6pkx3PEy41jpXrwmy3YniYv50T04d6wcfU5Pgp1x9q K9/xNlAYKTqm/GhjqRCbdslyDp/dkT00EPMy9ycv/7wgJgMR3Abk6KJS6cc+xP4MjeFc ZHYQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MYYreJyO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z27-20020a05620a101b00b0075938b14760si211947qkj.396.2023.05.16.12.54.41 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:54:41 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MYYreJyO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0eK-0002lX-Fv; Tue, 16 May 2023 15:47:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cI-0000Lk-DD for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:52 -0400 Received: from mail-pj1-x102d.google.com ([2607:f8b0:4864:20::102d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cD-0003kw-Cp for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:48 -0400 Received: by mail-pj1-x102d.google.com with SMTP id 98e67ed59e1d1-24ded4b33d7so21887a91.3 for ; Tue, 16 May 2023 12:45:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266341; x=1686858341; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wuME4Q0uyj8eqINyik0cwsf020/cK1vI3bJfJHG03XY=; b=MYYreJyO/ie479vEeEk6NixhZOLMixFy1P9G90NT+LsB7f+SfwP3kEq6iOP85RgBkl FJfYNrvzmERmYG3FL3Kwfss6Nk4SH1O2E8erOF6RdLz12EqbTntQZbMwxOc9RNJTDOBL V2qZoBJ3UFRwq1MuwIW/cfUeZsah+3fNZd8m3hAU4fwVGeRHdn0fve3QP5fEBOYFls60 gW0l+UGSbBwTLq+r8hkcPoHf58yUUNy8aDdBXLsAb4+cb3cV248HgfK5ODONpkillxWa 7qsxTDwA8ceKm1Q4f+5ChhlR+zH1Wkib9JzkMOsrzGy54K7X7EXgj5UvE8Lpc53++0fz KkAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266341; x=1686858341; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wuME4Q0uyj8eqINyik0cwsf020/cK1vI3bJfJHG03XY=; b=b27DujgwYHf6d7WLHa1snhAwWWIMxXj2hGD6v5uENDuHraQvLf0JfbE9UUcRAG9VUL WrRWokDV8dY5WLZWEUS8NchjRoa79BoXs2qXNtzK0rgm/vUCaUdhNsRPrpqk3TLs0CbX 0RHKgRqnsRWjSfJTfZxvhzon05OzNLwLuCqY3WLLJh0K6EsKV/jrzGlQyBbAHO/vZqi4 F4jmy3E4T48LQbO1g886u6gVX5HGp1Bnpw+xJIsWnlGqbpp74l4oBCHQhv7c6tEB/uTK JgY+NmgdK/nh0zAyyD1mpPApbCQ646yVh7uu0WpXsAZRTlPpmIplC1a0nFoNDxRlK35G tvsw== X-Gm-Message-State: AC+VfDxk/EQAizSF/FafWzvAcSS9WUJR6EJCbVKr3T4FOojw5m4afl4e lIy6mVJYtmAS6LPIcQeaKbLcMa44IGyfXlPX3Ow= X-Received: by 2002:a17:90a:b793:b0:250:6bd9:d0d with SMTP id m19-20020a17090ab79300b002506bd90d0dmr32874155pjr.15.1684266341199; Tue, 16 May 2023 12:45:41 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:40 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 61/80] tcg: Reduce copies for plugin_gen_mem_callbacks Date: Tue, 16 May 2023 12:41:26 -0700 Message-Id: <20230516194145.1749305-62-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102d; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org We only need to make copies for loads, when the destination overlaps the address. For now, only eliminate the copy for stores and 128-bit loads. Rename plugin_prep_mem_callbacks to plugin_maybe_preserve_addr, returning NULL if no copy is made. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- tcg/tcg-op-ldst.c | 38 ++++++++++++++++++++------------------ 1 file changed, 20 insertions(+), 18 deletions(-) diff --git a/tcg/tcg-op-ldst.c b/tcg/tcg-op-ldst.c index ca57a2779d..b695d2954e 100644 --- a/tcg/tcg-op-ldst.c +++ b/tcg/tcg-op-ldst.c @@ -114,7 +114,8 @@ static void tcg_gen_req_mo(TCGBar type) } } -static inline TCGv plugin_prep_mem_callbacks(TCGv vaddr) +/* Only required for loads, where value might overlap addr. */ +static TCGv plugin_maybe_preserve_addr(TCGv vaddr) { #ifdef CONFIG_PLUGIN if (tcg_ctx->plugin_insn != NULL) { @@ -124,17 +125,20 @@ static inline TCGv plugin_prep_mem_callbacks(TCGv vaddr) return temp; } #endif - return vaddr; + return NULL; } -static void plugin_gen_mem_callbacks(TCGv vaddr, MemOpIdx oi, - enum qemu_plugin_mem_rw rw) +static void +plugin_gen_mem_callbacks(TCGv copy_addr, TCGv orig_addr, MemOpIdx oi, + enum qemu_plugin_mem_rw rw) { #ifdef CONFIG_PLUGIN if (tcg_ctx->plugin_insn != NULL) { qemu_plugin_meminfo_t info = make_plugin_meminfo(oi, rw); - plugin_gen_empty_mem_callback(vaddr, info); - tcg_temp_free(vaddr); + plugin_gen_empty_mem_callback(copy_addr ? : orig_addr, info); + if (copy_addr) { + tcg_temp_free(copy_addr); + } } #endif } @@ -143,6 +147,7 @@ void tcg_gen_qemu_ld_i32(TCGv_i32 val, TCGv addr, TCGArg idx, MemOp memop) { MemOp orig_memop; MemOpIdx oi; + TCGv copy_addr; tcg_gen_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); memop = tcg_canonicalize_memop(memop, 0, 0); @@ -157,9 +162,9 @@ void tcg_gen_qemu_ld_i32(TCGv_i32 val, TCGv addr, TCGArg idx, MemOp memop) } } - addr = plugin_prep_mem_callbacks(addr); + copy_addr = plugin_maybe_preserve_addr(addr); gen_ldst_i32(INDEX_op_qemu_ld_i32, val, addr, memop, idx); - plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_R); + plugin_gen_mem_callbacks(copy_addr, addr, oi, QEMU_PLUGIN_MEM_R); if ((orig_memop ^ memop) & MO_BSWAP) { switch (orig_memop & MO_SIZE) { @@ -202,13 +207,12 @@ void tcg_gen_qemu_st_i32(TCGv_i32 val, TCGv addr, TCGArg idx, MemOp memop) memop &= ~MO_BSWAP; } - addr = plugin_prep_mem_callbacks(addr); if (TCG_TARGET_HAS_qemu_st8_i32 && (memop & MO_SIZE) == MO_8) { gen_ldst_i32(INDEX_op_qemu_st8_i32, val, addr, memop, idx); } else { gen_ldst_i32(INDEX_op_qemu_st_i32, val, addr, memop, idx); } - plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_W); + plugin_gen_mem_callbacks(NULL, addr, oi, QEMU_PLUGIN_MEM_W); if (swap) { tcg_temp_free_i32(swap); @@ -219,6 +223,7 @@ void tcg_gen_qemu_ld_i64(TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop) { MemOp orig_memop; MemOpIdx oi; + TCGv copy_addr; if (TCG_TARGET_REG_BITS == 32 && (memop & MO_SIZE) < MO_64) { tcg_gen_qemu_ld_i32(TCGV_LOW(val), addr, idx, memop); @@ -243,9 +248,9 @@ void tcg_gen_qemu_ld_i64(TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop) } } - addr = plugin_prep_mem_callbacks(addr); + copy_addr = plugin_maybe_preserve_addr(addr); gen_ldst_i64(INDEX_op_qemu_ld_i64, val, addr, memop, idx); - plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_R); + plugin_gen_mem_callbacks(copy_addr, addr, oi, QEMU_PLUGIN_MEM_R); if ((orig_memop ^ memop) & MO_BSWAP) { int flags = (orig_memop & MO_SIGN @@ -300,9 +305,8 @@ void tcg_gen_qemu_st_i64(TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop) memop &= ~MO_BSWAP; } - addr = plugin_prep_mem_callbacks(addr); gen_ldst_i64(INDEX_op_qemu_st_i64, val, addr, memop, idx); - plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_W); + plugin_gen_mem_callbacks(NULL, addr, oi, QEMU_PLUGIN_MEM_W); if (swap) { tcg_temp_free_i64(swap); @@ -419,7 +423,6 @@ void tcg_gen_qemu_ld_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) tcg_debug_assert((memop & MO_SIGN) == 0); tcg_gen_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); - addr = plugin_prep_mem_callbacks(addr); /* TODO: For now, force 32-bit hosts to use the helper. */ if (TCG_TARGET_HAS_qemu_ldst_i128 && TCG_TARGET_REG_BITS == 64) { @@ -490,7 +493,7 @@ void tcg_gen_qemu_ld_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) maybe_free_addr64(a64); } - plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_R); + plugin_gen_mem_callbacks(NULL, addr, oi, QEMU_PLUGIN_MEM_R); } void tcg_gen_qemu_st_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) @@ -501,7 +504,6 @@ void tcg_gen_qemu_st_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) tcg_debug_assert((memop & MO_SIGN) == 0); tcg_gen_req_mo(TCG_MO_ST_LD | TCG_MO_ST_ST); - addr = plugin_prep_mem_callbacks(addr); /* TODO: For now, force 32-bit hosts to use the helper. */ @@ -572,7 +574,7 @@ void tcg_gen_qemu_st_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) maybe_free_addr64(a64); } - plugin_gen_mem_callbacks(addr, oi, QEMU_PLUGIN_MEM_W); + plugin_gen_mem_callbacks(NULL, addr, oi, QEMU_PLUGIN_MEM_W); } static void tcg_gen_ext_i32(TCGv_i32 ret, TCGv_i32 val, MemOp opc) From patchwork Tue May 16 19:41:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682409 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp20531wrt; Tue, 16 May 2023 12:57:30 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7OtqBfj0zlR3RhuvIQPS7DvB6JPsdCbtIEyAxZFHkjDTJg6y9fcVGR/9cwSLDdZr1DyXh2 X-Received: by 2002:ad4:5ae5:0:b0:621:1b1d:7dde with SMTP id c5-20020ad45ae5000000b006211b1d7ddemr64313047qvh.8.1684267050284; Tue, 16 May 2023 12:57:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267050; cv=none; d=google.com; s=arc-20160816; b=a2wsRdBGNONY8w8ghrhnW8Xe4x/JKAdeGPdLAWVyJR8rrE7eam7VmCSSilrT25HEeg XMslUhIC+DHOsi+zNbJdw39LZBUaVrzIVstNbSTlc1TSWD5efuDMr9z9L1T8e6A1DRn4 e57W5b+KH1OCAa+zj/5F2xmGr/jDAvJ84p1YcYJKDNN90zsFEulsVz2veeD7o6qeo/xZ M9JUHqwEbiaNRFnR5wOQYzxXFDYXBAXGc+3SVFx/WpIv5BSb3SnpSMjYAwIgD2o7001Y FvV/T1ju0Zl8B+kEJwI4hrHkqHIM+93WZ3mnqzofJQsdk/SRreLclNHmVhHe65WrmRr/ Ddwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=5JyxwW/97nzLH4qcsKUT7VsDKiOxHavIc9pVCUEzi9E=; b=Ax5n8/VDbO1zTI+TJyxta11pEJR8XyMczzsjORdSJ3bRbFHQXkHKuHseBWnRREX1Cj qc8r5t1kfZOyK8Cj3eMzXv2HKpoBRvIat4qjy5sh18QMCrBvVNQHI7Uj9SIFlKxyNfPK jAo6/dyEqIPVopw8zyOwEME/XWJ/9PKjo3cnr/Q7nX02FYpwnbwryqkKausqvxRwWdQC VPwITY2CQVphKxleZbEacSv4XR0V/HcetYx71+Or+DfcWVa0tA/HPUr8pP7dUtJyH3Yw vhOJV3XAt4U4FJb7y/UkWh1IqX2/WS+bqCuYYbB8PjOnuHdKJllpJaFjndVitnt23jVi NCdw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Okfajnhe; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 10-20020a0562140d0a00b0061ada608558si12400672qvh.275.2023.05.16.12.57.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:57:30 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Okfajnhe; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0ev-0003Q1-PP; Tue, 16 May 2023 15:48:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cI-0000Lr-GB for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:52 -0400 Received: from mail-pg1-x52c.google.com ([2607:f8b0:4864:20::52c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cD-0003hv-Cw for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:48 -0400 Received: by mail-pg1-x52c.google.com with SMTP id 41be03b00d2f7-517ab9a4a13so13458148a12.1 for ; Tue, 16 May 2023 12:45:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266342; x=1686858342; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5JyxwW/97nzLH4qcsKUT7VsDKiOxHavIc9pVCUEzi9E=; b=Okfajnhe5xsCo2ySERBG1mGCTK9C7gMSpcB0EDrEWAyV7TilI/k6vev0j5YIQNNJRU 6J71ahXbvOpZRbxG3iC6LoZPJoA8TmGuKS58Jk2Lk6f5vzN3EDlnvozWqdNKMKPYD0AC b5w0073sH/nZMrh6kjrR+40SOwA1ARWlUjp2yKLjQHWpB4npIm+KJVOTrdv0V9VFiFXT cWVztUnJtBir++pU9z0fiDm4+tiGtFQBKs0RRC7PsqmdcoOk1num31cRI5BwvamvApsW cQoXnIRTzI16MNuS3J/YH6LjkXYxl+yCiCsDhQ9Lg1Zi1OBDgnYTK7fzaa8+Ci2ws2so BQxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266342; x=1686858342; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5JyxwW/97nzLH4qcsKUT7VsDKiOxHavIc9pVCUEzi9E=; b=YcPFJNsD+GSPwtyzqca3YpWQ9MZ88vGrhoJJfPOY1yu0/HNZki4jbEjwYUBhBuv7RK z03Ivz2lxde0eXWkpmTy10vADRG5plJ3hhJsUXo0YPPm3XJMoiYTmy+PoGIMjAyioTKV lGfmO+nB4Npt0hV1GbouroLLE19bOKqpAAZu7aeGzE++KhjKNwKbg+JX4j6nxvRJTrA5 1jOQSH2G2zlSKs1wpkyMueh8+Gt6xcICCCtTpF+9fpL08vGKNLBWyMTW8BqlxIH1r+yc elgMlMnfsEErnxNH45dVcohiJzwWG87zDsD2gWiApQ04J3ZM6Wtaoy+5tT3Xljilysjd QCIA== X-Gm-Message-State: AC+VfDwBpg5W2nDpiCgaDsAL4Ydx2iSWlgEWiIa5KsQHNy8ZyTUh7rIk XEPbXeVqRL02F/CVWRrgUyQGGDP3lwzF6L+lGTo= X-Received: by 2002:a17:90a:3ec2:b0:253:3b17:de90 with SMTP id k60-20020a17090a3ec200b002533b17de90mr924968pjc.21.1684266342104; Tue, 16 May 2023 12:45:42 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:41 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 62/80] accel/tcg: Widen plugin_gen_empty_mem_callback to i64 Date: Tue, 16 May 2023 12:41:27 -0700 Message-Id: <20230516194145.1749305-63-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52c; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Since we do this inside gen_empty_mem_cb anyway, let's do this earlier inside tcg expansion. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- include/exec/plugin-gen.h | 4 ++-- accel/tcg/plugin-gen.c | 9 +++------ tcg/tcg-op-ldst.c | 28 ++++++++++++++++++++-------- 3 files changed, 25 insertions(+), 16 deletions(-) diff --git a/include/exec/plugin-gen.h b/include/exec/plugin-gen.h index 5f5506f1cc..3af0168e65 100644 --- a/include/exec/plugin-gen.h +++ b/include/exec/plugin-gen.h @@ -27,7 +27,7 @@ void plugin_gen_insn_start(CPUState *cpu, const struct DisasContextBase *db); void plugin_gen_insn_end(void); void plugin_gen_disable_mem_helpers(void); -void plugin_gen_empty_mem_callback(TCGv addr, uint32_t info); +void plugin_gen_empty_mem_callback(TCGv_i64 addr, uint32_t info); static inline void plugin_insn_append(abi_ptr pc, const void *from, size_t size) { @@ -69,7 +69,7 @@ static inline void plugin_gen_tb_end(CPUState *cpu) static inline void plugin_gen_disable_mem_helpers(void) { } -static inline void plugin_gen_empty_mem_callback(TCGv addr, uint32_t info) +static inline void plugin_gen_empty_mem_callback(TCGv_i64 addr, uint32_t info) { } static inline void plugin_insn_append(abi_ptr pc, const void *from, size_t size) diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c index 907c5004a4..34be1b940c 100644 --- a/accel/tcg/plugin-gen.c +++ b/accel/tcg/plugin-gen.c @@ -124,23 +124,20 @@ static void gen_empty_inline_cb(void) tcg_temp_free_i64(val); } -static void gen_empty_mem_cb(TCGv addr, uint32_t info) +static void gen_empty_mem_cb(TCGv_i64 addr, uint32_t info) { TCGv_i32 cpu_index = tcg_temp_ebb_new_i32(); TCGv_i32 meminfo = tcg_temp_ebb_new_i32(); - TCGv_i64 addr64 = tcg_temp_ebb_new_i64(); TCGv_ptr udata = tcg_temp_ebb_new_ptr(); tcg_gen_movi_i32(meminfo, info); tcg_gen_movi_ptr(udata, 0); tcg_gen_ld_i32(cpu_index, cpu_env, -offsetof(ArchCPU, env) + offsetof(CPUState, cpu_index)); - tcg_gen_extu_tl_i64(addr64, addr); - gen_helper_plugin_vcpu_mem_cb(cpu_index, meminfo, addr64, udata); + gen_helper_plugin_vcpu_mem_cb(cpu_index, meminfo, addr, udata); tcg_temp_free_ptr(udata); - tcg_temp_free_i64(addr64); tcg_temp_free_i32(meminfo); tcg_temp_free_i32(cpu_index); } @@ -197,7 +194,7 @@ static void plugin_gen_empty_callback(enum plugin_gen_from from) } } -void plugin_gen_empty_mem_callback(TCGv addr, uint32_t info) +void plugin_gen_empty_mem_callback(TCGv_i64 addr, uint32_t info) { enum qemu_plugin_mem_rw rw = get_plugin_meminfo_rw(info); diff --git a/tcg/tcg-op-ldst.c b/tcg/tcg-op-ldst.c index b695d2954e..2d5e98971d 100644 --- a/tcg/tcg-op-ldst.c +++ b/tcg/tcg-op-ldst.c @@ -115,13 +115,13 @@ static void tcg_gen_req_mo(TCGBar type) } /* Only required for loads, where value might overlap addr. */ -static TCGv plugin_maybe_preserve_addr(TCGv vaddr) +static TCGv_i64 plugin_maybe_preserve_addr(TCGv vaddr) { #ifdef CONFIG_PLUGIN if (tcg_ctx->plugin_insn != NULL) { /* Save a copy of the vaddr for use after a load. */ - TCGv temp = tcg_temp_new(); - tcg_gen_mov_tl(temp, vaddr); + TCGv_i64 temp = tcg_temp_ebb_new_i64(); + tcg_gen_extu_tl_i64(temp, vaddr); return temp; } #endif @@ -129,16 +129,28 @@ static TCGv plugin_maybe_preserve_addr(TCGv vaddr) } static void -plugin_gen_mem_callbacks(TCGv copy_addr, TCGv orig_addr, MemOpIdx oi, +plugin_gen_mem_callbacks(TCGv_i64 copy_addr, TCGv orig_addr, MemOpIdx oi, enum qemu_plugin_mem_rw rw) { #ifdef CONFIG_PLUGIN if (tcg_ctx->plugin_insn != NULL) { qemu_plugin_meminfo_t info = make_plugin_meminfo(oi, rw); - plugin_gen_empty_mem_callback(copy_addr ? : orig_addr, info); + +#if TARGET_LONG_BITS == 64 if (copy_addr) { - tcg_temp_free(copy_addr); + plugin_gen_empty_mem_callback(copy_addr, info); + tcg_temp_free_i64(copy_addr); + } else { + plugin_gen_empty_mem_callback(orig_addr, info); } +#else + if (!copy_addr) { + copy_addr = tcg_temp_ebb_new_i64(); + tcg_gen_extu_tl_i64(copy_addr, orig_addr); + } + plugin_gen_empty_mem_callback(copy_addr, info); + tcg_temp_free_i64(copy_addr); +#endif } #endif } @@ -147,7 +159,7 @@ void tcg_gen_qemu_ld_i32(TCGv_i32 val, TCGv addr, TCGArg idx, MemOp memop) { MemOp orig_memop; MemOpIdx oi; - TCGv copy_addr; + TCGv_i64 copy_addr; tcg_gen_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); memop = tcg_canonicalize_memop(memop, 0, 0); @@ -223,7 +235,7 @@ void tcg_gen_qemu_ld_i64(TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop) { MemOp orig_memop; MemOpIdx oi; - TCGv copy_addr; + TCGv_i64 copy_addr; if (TCG_TARGET_REG_BITS == 32 && (memop & MO_SIZE) < MO_64) { tcg_gen_qemu_ld_i32(TCGV_LOW(val), addr, idx, memop); From patchwork Tue May 16 19:41:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682435 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp23642wrt; Tue, 16 May 2023 13:05:12 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4Zd3alWrF2p8JvCm7AZ9zcsAFBWYU7xpWA/xBMG5vL0rKyuKzT8mPO5Ls+xxUV3a3Bi5CB X-Received: by 2002:adf:f552:0:b0:306:772:5c2e with SMTP id j18-20020adff552000000b0030607725c2emr23847456wrp.70.1684267512547; Tue, 16 May 2023 13:05:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267512; cv=none; d=google.com; s=arc-20160816; b=jQmXPBb1gz1nU6ksa9ET+fNszErY4FM2JKaEiGLcG2s7Ns4FO1wScxtKQOTWeHctky fqeyBRssIXIXojKyAv6uK5LfwTpRUhy2rjhFuDLKh8UsCYFI4WpEYMCg/VOEthKk//5C dvy3XDl4sazlHcgBSIo5FYftNwXXhhopbcqNs5mar0gwXr6uVCPT5s2JZZ0PfITrXW+C +haOoS28ueYQ4IB09gf1C4fyKZPXfiJL4EADRNJDPTYgFMwLxKLgMPoVrSoER6AoFxBL qFOnqKkah/nITEv018RxnvIO36ZcgRujim8sdjx+xPOoVXRTcQWYdvxr9bWbWt5ae5lW O3Zg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=joEze47TBGXX0770sgjaIGXAA3+n4j9+h7GefAjVJ1s=; b=QapF5P6LpDgc155YU0WZ/UrJ1+VxI9tNwbR25F0OU/SQeb63T9bIgWHI6pijbM7AZS pyCqTnwzCjxOnldc0tt17vTCMV8GhEpTqiboBw9YfFGAA7Sallr5pZPLvL0wN6A9bjo+ vlWLuY2UdFm8m3VjWFdAG5gJQTjiMyK5o52FJHWTdrAXtNKtjpO0qyJw9wUTjbcX7TL2 p1zOg64BaBxLIMRkH7HLC2HTiJ1K+8GQySiMV7mhw9UC5wDSYbwv+OCQLkzq9HI5A3z2 yzHRdTjQc5j74QhB4jFx0mvrVBkhyW1BS37aVgj2nZT0V1at0C8YY8hZhNRAyTRgSCFv QCSg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="ZBc/Uc7l"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i10-20020a05600c354a00b003f51bbe6ef1si151796wmq.78.2023.05.16.13.05.12 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 13:05:12 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="ZBc/Uc7l"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0cn-000148-QF; Tue, 16 May 2023 15:46:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cI-0000Ll-Eb for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:52 -0400 Received: from mail-pj1-x1029.google.com ([2607:f8b0:4864:20::1029]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cC-0003lH-EW for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:48 -0400 Received: by mail-pj1-x1029.google.com with SMTP id 98e67ed59e1d1-2533b600d35so58243a91.1 for ; Tue, 16 May 2023 12:45:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266343; x=1686858343; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=joEze47TBGXX0770sgjaIGXAA3+n4j9+h7GefAjVJ1s=; b=ZBc/Uc7lvnUykiQfXp7OzLqGjzVFcGIdowXxpO0wzpKIVCpmpxb3epYMg3Pua/pTKL CqEr99hKM8O7iBrZSq7muMH05QotI4ivaBSZ/Sppbt72xu6SzXf5TJMGgjYVeYkfI9cO kAitAC8+I6Gmulnanu8hubq9E0zlzVL9PZR+6VXkDsc7Zq8N6yfPRCvky7lGOLZzBqD9 6UAf/toWK7GIRX/XXybLoWFQxKWJEcV/IgIxnxayKLVPubNmzcZyBSRslU20wXD6lkGX 0pr9rU2FtqORlgHKTri2m8bDQnnm7fKFj8I1W1dA5CNDzMEgGnmBfiDj9ruqjyVsKmYG qiaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266343; x=1686858343; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=joEze47TBGXX0770sgjaIGXAA3+n4j9+h7GefAjVJ1s=; b=SaFDC+7Z0pE0Wbo0uFXqueIJhGrQjBy3BqkhIpJlHJhTxGzNiYo1nrve998WuunKRs cHnDZH7qRxT/MukH9/eGqC4mV6iu5I7LO6XCASxAyNeNVGK23sVahsde7DzJ0VGOkxFb jX5J61DDJz846JO/o4RvvdAhGdSwED8ykKW+7K5W04uT2U07gu/9dYV2ZCkOsK/hV19R OZivfqB4/aPBBq4BgN5BScW75wG9JvchfJlK3eBf419BoQFkXjEZLNpzmxpbjrtbUPUs HlhimQBfzxE2jBTyYmcIpRpuhFn61NoA7kKJa9BBWixKO46jVDweyyL2XLLhxVIuhzfd 1XnA== X-Gm-Message-State: AC+VfDx4ptlGadp3X6YSlOzLM1JADHPKZdkVhNJyobaz4Sb9FN2CPTQh XJFpqY2mUJcrlSa20E2TDJP0tGH5m544z8GfTw0= X-Received: by 2002:a17:90b:2313:b0:253:2dc5:4e12 with SMTP id mt19-20020a17090b231300b002532dc54e12mr2463204pjb.46.1684266342887; Tue, 16 May 2023 12:45:42 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:42 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 63/80] tcg: Add addr_type to TCGContext Date: Tue, 16 May 2023 12:41:28 -0700 Message-Id: <20230516194145.1749305-64-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1029; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1029.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org This will enable replacement of TARGET_LONG_BITS within tcg/. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- include/tcg/tcg.h | 1 + accel/tcg/translate-all.c | 2 ++ tcg/tcg.c | 3 +++ 3 files changed, 6 insertions(+) diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h index 42a3c05607..b9748fd0c5 100644 --- a/include/tcg/tcg.h +++ b/include/tcg/tcg.h @@ -558,6 +558,7 @@ struct TCGContext { int nb_temps; int nb_indirects; int nb_ops; + TCGType addr_type; /* TCG_TYPE_I32 or TCG_TYPE_I64 */ TCGRegSet reserved_regs; intptr_t current_frame_offset; diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c index 7b7d9a5fff..99a9d0e34f 100644 --- a/accel/tcg/translate-all.c +++ b/accel/tcg/translate-all.c @@ -356,6 +356,8 @@ TranslationBlock *tb_gen_code(CPUState *cpu, tb_set_page_addr0(tb, phys_pc); tb_set_page_addr1(tb, -1); tcg_ctx->gen_tb = tb; + tcg_ctx->addr_type = TCG_TYPE_TL; + tb_overflow: #ifdef CONFIG_PROFILER diff --git a/tcg/tcg.c b/tcg/tcg.c index 800dcba159..dfe5dde63d 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -1520,6 +1520,9 @@ void tcg_func_start(TCGContext *s) QTAILQ_INIT(&s->ops); QTAILQ_INIT(&s->free_ops); QSIMPLEQ_INIT(&s->labels); + + tcg_debug_assert(s->addr_type == TCG_TYPE_I32 || + s->addr_type == TCG_TYPE_I64); } static TCGTemp *tcg_temp_alloc(TCGContext *s) From patchwork Tue May 16 19:41:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682432 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp22999wrt; Tue, 16 May 2023 13:03:36 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4int1A5cmcdy3qWzEOoWP3gfVjjzVplsJYZykuG+dY/61Pnsa4UJgr+y4lQ8i/phS6QXdC X-Received: by 2002:a5d:63c8:0:b0:306:2b56:d268 with SMTP id c8-20020a5d63c8000000b003062b56d268mr25586993wrw.15.1684267416673; Tue, 16 May 2023 13:03:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267416; cv=none; d=google.com; s=arc-20160816; b=PIdQzTRXOnA2b25/Ct4+nECAwFusEgc97H8dFAzoAeZBsqqk4B71wa34JmETeadl9u G0SekqdbN1XoTMxlvhPYMJ3GKbDNxvt44mS8vpDAoYNxDYKSU7MLgBUVQYfUcFDTLdqi LKl9Qe09+VOqvSbYDTS6mv4+Ab+crzYRK4dzQd0SdL3KQL2cFghYT6QqSqP7F88hijEu yV91T72OnM8o1+/Z5jD9mUyKpc/UuHTWp0AFZINXEN2qCn5zVw7mTu/WO9YHv91+88yu e2xIPDMZTyiEbITX5jWp0Sr9bCTjslR1A9b99a5wVwjfI81cB5Z27QgBFQI6kFXImFOD roZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=/ojCK0LNJKJYp6jdH9Or6ZqPPjbgWADgsJhhQdni33Y=; b=T2GM57qKUaNo+naoVezW5eM8yOIYn65NiRaR7rbsB0oUXLLR86sDmuxe0fJG2CO1z2 +LqWPzxrb1J5U4IBsl6ae7CP61+60MSNY/l+9mzS6LLOjKPgcidHjPfzN1kniVd3jzR1 g6jM8V48GwpiTLL4vB5yeqEMLVlRNuty7F8BbL8vawLPfd5vlo58o8IOOLQ8U1t7DZwm tqbqxZAV+h886dD3sYrXZu5KfMou+6kTCrxsQSrCcVzmVPYwOBIXT6C48NH/lD4P95Pb 0Wc0u+4zdAl0r/30lWuIbByCuIJuXadyIihf7nlcDTxnCEJDSgGsNt778EzN1aBRipSk efwQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rzzHkcQj; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f2-20020adffcc2000000b00306401d8cdbsi226950wrs.166.2023.05.16.13.03.36 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 13:03:36 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rzzHkcQj; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0cZ-0000qy-5Q; Tue, 16 May 2023 15:46:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cI-0000M8-GC for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:52 -0400 Received: from mail-pj1-x102e.google.com ([2607:f8b0:4864:20::102e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cD-0003iP-Lv for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:49 -0400 Received: by mail-pj1-x102e.google.com with SMTP id 98e67ed59e1d1-24df161f84bso39442a91.3 for ; Tue, 16 May 2023 12:45:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266344; x=1686858344; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/ojCK0LNJKJYp6jdH9Or6ZqPPjbgWADgsJhhQdni33Y=; b=rzzHkcQjQlcUYKQ+HPDBvL9xupgQUMG59a3UbZhXor9hyxV4DJM+mrrMjV6W8E6yXG WCawRPiWusNvNE4JC+lQfQ8UGudHOSQKPeQys44umm2Fe2vXKBVrry82sM1TCjfio8cw pIQudEcGcdjNA4hb3yxjzyyI+9XQaEaYIw2PbdeO9cb7nInm2enlFgpWJW3HR8J60EVN iY+avlPqYV//w4xRhXUjgqL/we6+hn4Q/vl8rQrdPc9lk7hpdn2rx5bsn57SS5x2ShBM +yWOHNUVd132mbqUKCmXNwkN1IiFnURWgwKYcf3GBCEdAjQ9VpDjZZJZ+Fhb3YqbHdMy HOpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266344; x=1686858344; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/ojCK0LNJKJYp6jdH9Or6ZqPPjbgWADgsJhhQdni33Y=; b=QPDkPl0llom57bLkZu76apFj1EK/beOaejgZQ+l0G19+J/uKQwMfNMEqSs2+Y4DWYI eyJdgmb+//XEP+nnMunFLD17dodcj7sXcTbAC75NYnb5/Jl+wohiiw5qCjBi6WYwkWHz 1CNcgnxA8uMD9/OmT2oJsRgDonCdI3ZA32gBMBD1eJ/3coUVlQPvDPbQa+Mt2/YaEnLD v/XPLUUzeoZx0L6zY6uvOmzU8e9UleuyWEcXUTqbW95XWK6Hw2vAEILpGJJX2NKzOLMN cft/ZXRsUQyaFnYQfLjaoZNuNNStpwk4Cl6Qx7qGe7sGi/zDU+kdfye2OibXuU1/s+k0 zasw== X-Gm-Message-State: AC+VfDxU02NlmQnPEmMnlTnEG50lz7NLW3kaoqL+mMOtmjMuZkGTjw8/ 5QuMN4Y6AoxeAULt8R6SccmyX2b8b9lkkyGlwaw= X-Received: by 2002:a17:90a:1bcd:b0:23d:bbcb:c97f with SMTP id r13-20020a17090a1bcd00b0023dbbcbc97fmr37554231pjr.1.1684266343639; Tue, 16 May 2023 12:45:43 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:43 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 64/80] tcg: Remove TCGv from tcg_gen_qemu_{ld,st}_* Date: Tue, 16 May 2023 12:41:29 -0700 Message-Id: <20230516194145.1749305-65-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102e; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Expand from TCGv to TCGTemp inline in the translators, and validate that the size matches tcg_ctx->addr_type. These inlines will eventually be seen only by target-specific code. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- include/tcg/tcg-op.h | 50 ++++++- tcg/tcg-op-ldst.c | 343 ++++++++++++++++++++++++++----------------- 2 files changed, 251 insertions(+), 142 deletions(-) diff --git a/include/tcg/tcg-op.h b/include/tcg/tcg-op.h index de3b70aa84..e556450ba9 100644 --- a/include/tcg/tcg-op.h +++ b/include/tcg/tcg-op.h @@ -803,22 +803,60 @@ static inline void tcg_gen_plugin_cb_end(void) #define tcg_temp_new() tcg_temp_new_i32() #define tcg_global_mem_new tcg_global_mem_new_i32 #define tcg_temp_free tcg_temp_free_i32 +#define tcgv_tl_temp tcgv_i32_temp #define tcg_gen_qemu_ld_tl tcg_gen_qemu_ld_i32 #define tcg_gen_qemu_st_tl tcg_gen_qemu_st_i32 #else #define tcg_temp_new() tcg_temp_new_i64() #define tcg_global_mem_new tcg_global_mem_new_i64 #define tcg_temp_free tcg_temp_free_i64 +#define tcgv_tl_temp tcgv_i64_temp #define tcg_gen_qemu_ld_tl tcg_gen_qemu_ld_i64 #define tcg_gen_qemu_st_tl tcg_gen_qemu_st_i64 #endif -void tcg_gen_qemu_ld_i32(TCGv_i32, TCGv, TCGArg, MemOp); -void tcg_gen_qemu_st_i32(TCGv_i32, TCGv, TCGArg, MemOp); -void tcg_gen_qemu_ld_i64(TCGv_i64, TCGv, TCGArg, MemOp); -void tcg_gen_qemu_st_i64(TCGv_i64, TCGv, TCGArg, MemOp); -void tcg_gen_qemu_ld_i128(TCGv_i128, TCGv, TCGArg, MemOp); -void tcg_gen_qemu_st_i128(TCGv_i128, TCGv, TCGArg, MemOp); +void tcg_gen_qemu_ld_i32_chk(TCGv_i32, TCGTemp *, TCGArg, MemOp, TCGType); +void tcg_gen_qemu_st_i32_chk(TCGv_i32, TCGTemp *, TCGArg, MemOp, TCGType); +void tcg_gen_qemu_ld_i64_chk(TCGv_i64, TCGTemp *, TCGArg, MemOp, TCGType); +void tcg_gen_qemu_st_i64_chk(TCGv_i64, TCGTemp *, TCGArg, MemOp, TCGType); +void tcg_gen_qemu_ld_i128_chk(TCGv_i128, TCGTemp *, TCGArg, MemOp, TCGType); +void tcg_gen_qemu_st_i128_chk(TCGv_i128, TCGTemp *, TCGArg, MemOp, TCGType); + +static inline void +tcg_gen_qemu_ld_i32(TCGv_i32 v, TCGv a, TCGArg i, MemOp m) +{ + tcg_gen_qemu_ld_i32_chk(v, tcgv_tl_temp(a), i, m, TCG_TYPE_TL); +} + +static inline void +tcg_gen_qemu_st_i32(TCGv_i32 v, TCGv a, TCGArg i, MemOp m) +{ + tcg_gen_qemu_st_i32_chk(v, tcgv_tl_temp(a), i, m, TCG_TYPE_TL); +} + +static inline void +tcg_gen_qemu_ld_i64(TCGv_i64 v, TCGv a, TCGArg i, MemOp m) +{ + tcg_gen_qemu_ld_i64_chk(v, tcgv_tl_temp(a), i, m, TCG_TYPE_TL); +} + +static inline void +tcg_gen_qemu_st_i64(TCGv_i64 v, TCGv a, TCGArg i, MemOp m) +{ + tcg_gen_qemu_st_i64_chk(v, tcgv_tl_temp(a), i, m, TCG_TYPE_TL); +} + +static inline void +tcg_gen_qemu_ld_i128(TCGv_i128 v, TCGv a, TCGArg i, MemOp m) +{ + tcg_gen_qemu_ld_i128_chk(v, tcgv_tl_temp(a), i, m, TCG_TYPE_TL); +} + +static inline void +tcg_gen_qemu_st_i128(TCGv_i128 v, TCGv a, TCGArg i, MemOp m) +{ + tcg_gen_qemu_st_i128_chk(v, tcgv_tl_temp(a), i, m, TCG_TYPE_TL); +} void tcg_gen_atomic_cmpxchg_i32(TCGv_i32, TCGv, TCGv_i32, TCGv_i32, TCGArg, MemOp); diff --git a/tcg/tcg-op-ldst.c b/tcg/tcg-op-ldst.c index 2d5e98971d..84a03bf6ed 100644 --- a/tcg/tcg-op-ldst.c +++ b/tcg/tcg-op-ldst.c @@ -68,39 +68,38 @@ static inline MemOp tcg_canonicalize_memop(MemOp op, bool is64, bool st) return op; } -static void gen_ldst_i32(TCGOpcode opc, TCGv_i32 val, TCGv addr, - MemOp memop, TCGArg idx) +static void gen_ldst(TCGOpcode opc, TCGTemp *vl, TCGTemp *vh, + TCGTemp *addr, MemOpIdx oi) { - MemOpIdx oi = make_memop_idx(memop, idx); -#if TARGET_LONG_BITS == 32 - tcg_gen_op3i_i32(opc, val, addr, oi); -#else - if (TCG_TARGET_REG_BITS == 32) { - tcg_gen_op4i_i32(opc, val, TCGV_LOW(addr), TCGV_HIGH(addr), oi); + if (TCG_TARGET_REG_BITS == 64 || tcg_ctx->addr_type == TCG_TYPE_I32) { + if (vh) { + tcg_gen_op4(opc, temp_arg(vl), temp_arg(vh), temp_arg(addr), oi); + } else { + tcg_gen_op3(opc, temp_arg(vl), temp_arg(addr), oi); + } } else { - tcg_gen_op3(opc, tcgv_i32_arg(val), tcgv_i64_arg(addr), oi); + /* See TCGV_LOW/HIGH. */ + TCGTemp *al = addr + HOST_BIG_ENDIAN; + TCGTemp *ah = addr + !HOST_BIG_ENDIAN; + + if (vh) { + tcg_gen_op5(opc, temp_arg(vl), temp_arg(vh), + temp_arg(al), temp_arg(ah), oi); + } else { + tcg_gen_op4(opc, temp_arg(vl), temp_arg(al), temp_arg(ah), oi); + } } -#endif } -static void gen_ldst_i64(TCGOpcode opc, TCGv_i64 val, TCGv addr, - MemOp memop, TCGArg idx) +static void gen_ldst_i64(TCGOpcode opc, TCGv_i64 v, TCGTemp *addr, MemOpIdx oi) { - MemOpIdx oi = make_memop_idx(memop, idx); -#if TARGET_LONG_BITS == 32 if (TCG_TARGET_REG_BITS == 32) { - tcg_gen_op4i_i32(opc, TCGV_LOW(val), TCGV_HIGH(val), addr, oi); + TCGTemp *vl = tcgv_i32_temp(TCGV_LOW(v)); + TCGTemp *vh = tcgv_i32_temp(TCGV_HIGH(v)); + gen_ldst(opc, vl, vh, addr, oi); } else { - tcg_gen_op3(opc, tcgv_i64_arg(val), tcgv_i32_arg(addr), oi); + gen_ldst(opc, tcgv_i64_temp(v), NULL, addr, oi); } -#else - if (TCG_TARGET_REG_BITS == 32) { - tcg_gen_op5i_i32(opc, TCGV_LOW(val), TCGV_HIGH(val), - TCGV_LOW(addr), TCGV_HIGH(addr), oi); - } else { - tcg_gen_op3i_i64(opc, val, addr, oi); - } -#endif } static void tcg_gen_req_mo(TCGBar type) @@ -115,13 +114,17 @@ static void tcg_gen_req_mo(TCGBar type) } /* Only required for loads, where value might overlap addr. */ -static TCGv_i64 plugin_maybe_preserve_addr(TCGv vaddr) +static TCGv_i64 plugin_maybe_preserve_addr(TCGTemp *addr) { #ifdef CONFIG_PLUGIN if (tcg_ctx->plugin_insn != NULL) { /* Save a copy of the vaddr for use after a load. */ TCGv_i64 temp = tcg_temp_ebb_new_i64(); - tcg_gen_extu_tl_i64(temp, vaddr); + if (tcg_ctx->addr_type == TCG_TYPE_I32) { + tcg_gen_extu_i32_i64(temp, temp_tcgv_i32(addr)); + } else { + tcg_gen_mov_i64(temp, temp_tcgv_i64(addr)); + } return temp; } #endif @@ -129,54 +132,55 @@ static TCGv_i64 plugin_maybe_preserve_addr(TCGv vaddr) } static void -plugin_gen_mem_callbacks(TCGv_i64 copy_addr, TCGv orig_addr, MemOpIdx oi, +plugin_gen_mem_callbacks(TCGv_i64 copy_addr, TCGTemp *orig_addr, MemOpIdx oi, enum qemu_plugin_mem_rw rw) { #ifdef CONFIG_PLUGIN if (tcg_ctx->plugin_insn != NULL) { qemu_plugin_meminfo_t info = make_plugin_meminfo(oi, rw); -#if TARGET_LONG_BITS == 64 - if (copy_addr) { + if (tcg_ctx->addr_type == TCG_TYPE_I32) { + if (!copy_addr) { + copy_addr = tcg_temp_ebb_new_i64(); + tcg_gen_extu_i32_i64(copy_addr, temp_tcgv_i32(orig_addr)); + } plugin_gen_empty_mem_callback(copy_addr, info); tcg_temp_free_i64(copy_addr); } else { - plugin_gen_empty_mem_callback(orig_addr, info); + if (copy_addr) { + plugin_gen_empty_mem_callback(copy_addr, info); + tcg_temp_free_i64(copy_addr); + } else { + plugin_gen_empty_mem_callback(temp_tcgv_i64(orig_addr), info); + } } -#else - if (!copy_addr) { - copy_addr = tcg_temp_ebb_new_i64(); - tcg_gen_extu_tl_i64(copy_addr, orig_addr); - } - plugin_gen_empty_mem_callback(copy_addr, info); - tcg_temp_free_i64(copy_addr); -#endif } #endif } -void tcg_gen_qemu_ld_i32(TCGv_i32 val, TCGv addr, TCGArg idx, MemOp memop) +static void tcg_gen_qemu_ld_i32_int(TCGv_i32 val, TCGTemp *addr, + TCGArg idx, MemOp memop) { MemOp orig_memop; - MemOpIdx oi; + MemOpIdx orig_oi, oi; TCGv_i64 copy_addr; tcg_gen_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); - memop = tcg_canonicalize_memop(memop, 0, 0); - oi = make_memop_idx(memop, idx); + orig_memop = memop = tcg_canonicalize_memop(memop, 0, 0); + orig_oi = oi = make_memop_idx(memop, idx); - orig_memop = memop; if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { memop &= ~MO_BSWAP; /* The bswap primitive benefits from zero-extended input. */ if ((memop & MO_SSIZE) == MO_SW) { memop &= ~MO_SIGN; } + oi = make_memop_idx(memop, idx); } copy_addr = plugin_maybe_preserve_addr(addr); - gen_ldst_i32(INDEX_op_qemu_ld_i32, val, addr, memop, idx); - plugin_gen_mem_callbacks(copy_addr, addr, oi, QEMU_PLUGIN_MEM_R); + gen_ldst(INDEX_op_qemu_ld_i32, tcgv_i32_temp(val), NULL, addr, oi); + plugin_gen_mem_callbacks(copy_addr, addr, orig_oi, QEMU_PLUGIN_MEM_R); if ((orig_memop ^ memop) & MO_BSWAP) { switch (orig_memop & MO_SIZE) { @@ -194,14 +198,24 @@ void tcg_gen_qemu_ld_i32(TCGv_i32 val, TCGv addr, TCGArg idx, MemOp memop) } } -void tcg_gen_qemu_st_i32(TCGv_i32 val, TCGv addr, TCGArg idx, MemOp memop) +void tcg_gen_qemu_ld_i32_chk(TCGv_i32 val, TCGTemp *addr, TCGArg idx, + MemOp memop, TCGType addr_type) +{ + tcg_debug_assert(addr_type == tcg_ctx->addr_type); + tcg_debug_assert((memop & MO_SIZE) <= MO_32); + tcg_gen_qemu_ld_i32_int(val, addr, idx, memop); +} + +static void tcg_gen_qemu_st_i32_int(TCGv_i32 val, TCGTemp *addr, + TCGArg idx, MemOp memop) { TCGv_i32 swap = NULL; - MemOpIdx oi; + MemOpIdx orig_oi, oi; + TCGOpcode opc; tcg_gen_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST); memop = tcg_canonicalize_memop(memop, 0, 1); - oi = make_memop_idx(memop, idx); + orig_oi = oi = make_memop_idx(memop, idx); if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { swap = tcg_temp_ebb_new_i32(); @@ -217,28 +231,39 @@ void tcg_gen_qemu_st_i32(TCGv_i32 val, TCGv addr, TCGArg idx, MemOp memop) } val = swap; memop &= ~MO_BSWAP; + oi = make_memop_idx(memop, idx); } if (TCG_TARGET_HAS_qemu_st8_i32 && (memop & MO_SIZE) == MO_8) { - gen_ldst_i32(INDEX_op_qemu_st8_i32, val, addr, memop, idx); + opc = INDEX_op_qemu_st8_i32; } else { - gen_ldst_i32(INDEX_op_qemu_st_i32, val, addr, memop, idx); + opc = INDEX_op_qemu_st_i32; } - plugin_gen_mem_callbacks(NULL, addr, oi, QEMU_PLUGIN_MEM_W); + gen_ldst(opc, tcgv_i32_temp(val), NULL, addr, oi); + plugin_gen_mem_callbacks(NULL, addr, orig_oi, QEMU_PLUGIN_MEM_W); if (swap) { tcg_temp_free_i32(swap); } } -void tcg_gen_qemu_ld_i64(TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop) +void tcg_gen_qemu_st_i32_chk(TCGv_i32 val, TCGTemp *addr, TCGArg idx, + MemOp memop, TCGType addr_type) +{ + tcg_debug_assert(addr_type == tcg_ctx->addr_type); + tcg_debug_assert((memop & MO_SIZE) <= MO_32); + tcg_gen_qemu_st_i32_int(val, addr, idx, memop); +} + +static void tcg_gen_qemu_ld_i64_int(TCGv_i64 val, TCGTemp *addr, + TCGArg idx, MemOp memop) { MemOp orig_memop; - MemOpIdx oi; + MemOpIdx orig_oi, oi; TCGv_i64 copy_addr; if (TCG_TARGET_REG_BITS == 32 && (memop & MO_SIZE) < MO_64) { - tcg_gen_qemu_ld_i32(TCGV_LOW(val), addr, idx, memop); + tcg_gen_qemu_ld_i32_int(TCGV_LOW(val), addr, idx, memop); if (memop & MO_SIGN) { tcg_gen_sari_i32(TCGV_HIGH(val), TCGV_LOW(val), 31); } else { @@ -248,21 +273,21 @@ void tcg_gen_qemu_ld_i64(TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop) } tcg_gen_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); - memop = tcg_canonicalize_memop(memop, 1, 0); - oi = make_memop_idx(memop, idx); + orig_memop = memop = tcg_canonicalize_memop(memop, 1, 0); + orig_oi = oi = make_memop_idx(memop, idx); - orig_memop = memop; if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { memop &= ~MO_BSWAP; /* The bswap primitive benefits from zero-extended input. */ if ((memop & MO_SIGN) && (memop & MO_SIZE) < MO_64) { memop &= ~MO_SIGN; } + oi = make_memop_idx(memop, idx); } copy_addr = plugin_maybe_preserve_addr(addr); - gen_ldst_i64(INDEX_op_qemu_ld_i64, val, addr, memop, idx); - plugin_gen_mem_callbacks(copy_addr, addr, oi, QEMU_PLUGIN_MEM_R); + gen_ldst_i64(INDEX_op_qemu_ld_i64, val, addr, oi); + plugin_gen_mem_callbacks(copy_addr, addr, orig_oi, QEMU_PLUGIN_MEM_R); if ((orig_memop ^ memop) & MO_BSWAP) { int flags = (orig_memop & MO_SIGN @@ -284,19 +309,28 @@ void tcg_gen_qemu_ld_i64(TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop) } } -void tcg_gen_qemu_st_i64(TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop) +void tcg_gen_qemu_ld_i64_chk(TCGv_i64 val, TCGTemp *addr, TCGArg idx, + MemOp memop, TCGType addr_type) +{ + tcg_debug_assert(addr_type == tcg_ctx->addr_type); + tcg_debug_assert((memop & MO_SIZE) <= MO_64); + tcg_gen_qemu_ld_i64_int(val, addr, idx, memop); +} + +static void tcg_gen_qemu_st_i64_int(TCGv_i64 val, TCGTemp *addr, + TCGArg idx, MemOp memop) { TCGv_i64 swap = NULL; - MemOpIdx oi; + MemOpIdx orig_oi, oi; if (TCG_TARGET_REG_BITS == 32 && (memop & MO_SIZE) < MO_64) { - tcg_gen_qemu_st_i32(TCGV_LOW(val), addr, idx, memop); + tcg_gen_qemu_st_i32_int(TCGV_LOW(val), addr, idx, memop); return; } tcg_gen_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST); memop = tcg_canonicalize_memop(memop, 1, 1); - oi = make_memop_idx(memop, idx); + orig_oi = oi = make_memop_idx(memop, idx); if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { swap = tcg_temp_ebb_new_i64(); @@ -315,16 +349,25 @@ void tcg_gen_qemu_st_i64(TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop) } val = swap; memop &= ~MO_BSWAP; + oi = make_memop_idx(memop, idx); } - gen_ldst_i64(INDEX_op_qemu_st_i64, val, addr, memop, idx); - plugin_gen_mem_callbacks(NULL, addr, oi, QEMU_PLUGIN_MEM_W); + gen_ldst_i64(INDEX_op_qemu_st_i64, val, addr, oi); + plugin_gen_mem_callbacks(NULL, addr, orig_oi, QEMU_PLUGIN_MEM_W); if (swap) { tcg_temp_free_i64(swap); } } +void tcg_gen_qemu_st_i64_chk(TCGv_i64 val, TCGTemp *addr, TCGArg idx, + MemOp memop, TCGType addr_type) +{ + tcg_debug_assert(addr_type == tcg_ctx->addr_type); + tcg_debug_assert((memop & MO_SIZE) <= MO_64); + tcg_gen_qemu_st_i64_int(val, addr, idx, memop); +} + /* * Return true if @mop, without knowledge of the pointer alignment, * does not require 16-byte atomicity, and it would be adventagous @@ -360,9 +403,6 @@ static void canonicalize_memop_i128_as_i64(MemOp ret[2], MemOp orig) { MemOp mop_1 = orig, mop_2; - tcg_debug_assert((orig & MO_SIZE) == MO_128); - tcg_debug_assert((orig & MO_SIGN) == 0); - /* Reduce the size to 64-bit. */ mop_1 = (mop_1 & ~MO_SIZE) | MO_64; @@ -403,12 +443,6 @@ static void canonicalize_memop_i128_as_i64(MemOp ret[2], MemOp orig) ret[1] = mop_2; } -#if TARGET_LONG_BITS == 64 -#define tcg_temp_ebb_new tcg_temp_ebb_new_i64 -#else -#define tcg_temp_ebb_new tcg_temp_ebb_new_i32 -#endif - static TCGv_i64 maybe_extend_addr64(TCGv addr) { #if TARGET_LONG_BITS == 32 @@ -427,39 +461,32 @@ static void maybe_free_addr64(TCGv_i64 a64) #endif } -void tcg_gen_qemu_ld_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) +static void tcg_gen_qemu_ld_i128_int(TCGv_i128 val, TCGTemp *addr, + TCGArg idx, MemOp memop) { - const MemOpIdx oi = make_memop_idx(memop, idx); - - tcg_debug_assert((memop & MO_SIZE) == MO_128); - tcg_debug_assert((memop & MO_SIGN) == 0); + const MemOpIdx orig_oi = make_memop_idx(memop, idx); + TCGv_i64 ext_addr = NULL; tcg_gen_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); /* TODO: For now, force 32-bit hosts to use the helper. */ if (TCG_TARGET_HAS_qemu_ldst_i128 && TCG_TARGET_REG_BITS == 64) { TCGv_i64 lo, hi; - TCGArg addr_arg; - MemOpIdx adj_oi; bool need_bswap = false; + MemOpIdx oi = orig_oi; if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { lo = TCGV128_HIGH(val); hi = TCGV128_LOW(val); - adj_oi = make_memop_idx(memop & ~MO_BSWAP, idx); + oi = make_memop_idx(memop & ~MO_BSWAP, idx); need_bswap = true; } else { lo = TCGV128_LOW(val); hi = TCGV128_HIGH(val); - adj_oi = oi; } -#if TARGET_LONG_BITS == 32 - addr_arg = tcgv_i32_arg(addr); -#else - addr_arg = tcgv_i64_arg(addr); -#endif - tcg_gen_op4ii_i64(INDEX_op_qemu_ld_i128, lo, hi, addr_arg, adj_oi); + gen_ldst(INDEX_op_qemu_ld_i128, tcgv_i64_temp(lo), + tcgv_i64_temp(hi), addr, oi); if (need_bswap) { tcg_gen_bswap64_i64(lo, lo); @@ -467,10 +494,13 @@ void tcg_gen_qemu_ld_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) } } else if (use_two_i64_for_i128(memop)) { MemOp mop[2]; - TCGv addr_p8; + TCGTemp *addr_p8; TCGv_i64 x, y; + MemOpIdx oi; + bool need_bswap; canonicalize_memop_i128_as_i64(mop, memop); + need_bswap = (mop[0] ^ memop) & MO_BSWAP; /* * Since there are no global TCGv_i128, there is no visible state @@ -485,35 +515,56 @@ void tcg_gen_qemu_ld_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) y = TCGV128_LOW(val); } - gen_ldst_i64(INDEX_op_qemu_ld_i64, x, addr, mop[0], idx); + oi = make_memop_idx(mop[0], idx); + gen_ldst_i64(INDEX_op_qemu_ld_i64, x, addr, oi); - if ((mop[0] ^ memop) & MO_BSWAP) { + if (need_bswap) { tcg_gen_bswap64_i64(x, x); } - addr_p8 = tcg_temp_ebb_new(); - tcg_gen_addi_tl(addr_p8, addr, 8); - gen_ldst_i64(INDEX_op_qemu_ld_i64, y, addr_p8, mop[1], idx); - tcg_temp_free(addr_p8); + if (tcg_ctx->addr_type == TCG_TYPE_I32) { + TCGv_i32 t = tcg_temp_ebb_new_i32(); + tcg_gen_addi_i32(t, temp_tcgv_i32(addr), 8); + addr_p8 = tcgv_i32_temp(t); + } else { + TCGv_i64 t = tcg_temp_ebb_new_i64(); + tcg_gen_addi_i64(t, temp_tcgv_i64(addr), 8); + addr_p8 = tcgv_i64_temp(t); + } - if ((mop[0] ^ memop) & MO_BSWAP) { + gen_ldst_i64(INDEX_op_qemu_ld_i64, y, addr_p8, oi); + tcg_temp_free_internal(addr_p8); + + if (need_bswap) { tcg_gen_bswap64_i64(y, y); } } else { - TCGv_i64 a64 = maybe_extend_addr64(addr); - gen_helper_ld_i128(val, cpu_env, a64, tcg_constant_i32(oi)); - maybe_free_addr64(a64); + if (tcg_ctx->addr_type == TCG_TYPE_I32) { + ext_addr = tcg_temp_ebb_new_i64(); + tcg_gen_extu_i32_i64(ext_addr, temp_tcgv_i32(addr)); + addr = tcgv_i64_temp(ext_addr); + } + gen_helper_ld_i128(val, cpu_env, temp_tcgv_i64(addr), + tcg_constant_i32(orig_oi)); } - plugin_gen_mem_callbacks(NULL, addr, oi, QEMU_PLUGIN_MEM_R); + plugin_gen_mem_callbacks(ext_addr, addr, orig_oi, QEMU_PLUGIN_MEM_R); } -void tcg_gen_qemu_st_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) +void tcg_gen_qemu_ld_i128_chk(TCGv_i128 val, TCGTemp *addr, TCGArg idx, + MemOp memop, TCGType addr_type) { - const MemOpIdx oi = make_memop_idx(memop, idx); - + tcg_debug_assert(addr_type == tcg_ctx->addr_type); tcg_debug_assert((memop & MO_SIZE) == MO_128); tcg_debug_assert((memop & MO_SIGN) == 0); + tcg_gen_qemu_ld_i128_int(val, addr, idx, memop); +} + +static void tcg_gen_qemu_st_i128_int(TCGv_i128 val, TCGTemp *addr, + TCGArg idx, MemOp memop) +{ + const MemOpIdx orig_oi = make_memop_idx(memop, idx); + TCGv_i64 ext_addr = NULL; tcg_gen_req_mo(TCG_MO_ST_LD | TCG_MO_ST_ST); @@ -521,29 +572,23 @@ void tcg_gen_qemu_st_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) if (TCG_TARGET_HAS_qemu_ldst_i128 && TCG_TARGET_REG_BITS == 64) { TCGv_i64 lo, hi; - TCGArg addr_arg; - MemOpIdx adj_oi; + MemOpIdx oi = orig_oi; bool need_bswap = false; if ((memop & MO_BSWAP) && !tcg_target_has_memory_bswap(memop)) { - lo = tcg_temp_new_i64(); - hi = tcg_temp_new_i64(); + lo = tcg_temp_ebb_new_i64(); + hi = tcg_temp_ebb_new_i64(); tcg_gen_bswap64_i64(lo, TCGV128_HIGH(val)); tcg_gen_bswap64_i64(hi, TCGV128_LOW(val)); - adj_oi = make_memop_idx(memop & ~MO_BSWAP, idx); + oi = make_memop_idx(memop & ~MO_BSWAP, idx); need_bswap = true; } else { lo = TCGV128_LOW(val); hi = TCGV128_HIGH(val); - adj_oi = oi; } -#if TARGET_LONG_BITS == 32 - addr_arg = tcgv_i32_arg(addr); -#else - addr_arg = tcgv_i64_arg(addr); -#endif - tcg_gen_op4ii_i64(INDEX_op_qemu_st_i128, lo, hi, addr_arg, adj_oi); + gen_ldst(INDEX_op_qemu_st_i128, tcgv_i64_temp(lo), + tcgv_i64_temp(hi), addr, oi); if (need_bswap) { tcg_temp_free_i64(lo); @@ -551,8 +596,8 @@ void tcg_gen_qemu_st_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) } } else if (use_two_i64_for_i128(memop)) { MemOp mop[2]; - TCGv addr_p8; - TCGv_i64 x, y; + TCGTemp *addr_p8; + TCGv_i64 x, y, b = NULL; canonicalize_memop_i128_as_i64(mop, memop); @@ -564,29 +609,55 @@ void tcg_gen_qemu_st_i128(TCGv_i128 val, TCGv addr, TCGArg idx, MemOp memop) y = TCGV128_LOW(val); } - addr_p8 = tcg_temp_ebb_new(); if ((mop[0] ^ memop) & MO_BSWAP) { - TCGv_i64 t = tcg_temp_ebb_new_i64(); - - tcg_gen_bswap64_i64(t, x); - gen_ldst_i64(INDEX_op_qemu_st_i64, t, addr, mop[0], idx); - tcg_gen_bswap64_i64(t, y); - tcg_gen_addi_tl(addr_p8, addr, 8); - gen_ldst_i64(INDEX_op_qemu_st_i64, t, addr_p8, mop[1], idx); - tcg_temp_free_i64(t); - } else { - gen_ldst_i64(INDEX_op_qemu_st_i64, x, addr, mop[0], idx); - tcg_gen_addi_tl(addr_p8, addr, 8); - gen_ldst_i64(INDEX_op_qemu_st_i64, y, addr_p8, mop[1], idx); + b = tcg_temp_ebb_new_i64(); + tcg_gen_bswap64_i64(b, x); + x = b; } - tcg_temp_free(addr_p8); + gen_ldst_i64(INDEX_op_qemu_st_i64, x, addr, + make_memop_idx(mop[0], idx)); + + if (tcg_ctx->addr_type == TCG_TYPE_I32) { + TCGv_i32 t = tcg_temp_ebb_new_i32(); + tcg_gen_addi_i32(t, temp_tcgv_i32(addr), 8); + addr_p8 = tcgv_i32_temp(t); + } else { + TCGv_i64 t = tcg_temp_ebb_new_i64(); + tcg_gen_addi_i64(t, temp_tcgv_i64(addr), 8); + addr_p8 = tcgv_i64_temp(t); + } + + if (b) { + tcg_gen_bswap64_i64(b, y); + y = b; + } + gen_ldst_i64(INDEX_op_qemu_st_i64, y, addr_p8, + make_memop_idx(mop[1], idx)); + + if (b) { + tcg_temp_free_i64(b); + } + tcg_temp_free_internal(addr_p8); } else { - TCGv_i64 a64 = maybe_extend_addr64(addr); - gen_helper_st_i128(cpu_env, a64, val, tcg_constant_i32(oi)); - maybe_free_addr64(a64); + if (tcg_ctx->addr_type == TCG_TYPE_I32) { + ext_addr = tcg_temp_ebb_new_i64(); + tcg_gen_extu_i32_i64(ext_addr, temp_tcgv_i32(addr)); + addr = tcgv_i64_temp(ext_addr); + } + gen_helper_st_i128(cpu_env, temp_tcgv_i64(addr), val, + tcg_constant_i32(orig_oi)); } - plugin_gen_mem_callbacks(NULL, addr, oi, QEMU_PLUGIN_MEM_W); + plugin_gen_mem_callbacks(ext_addr, addr, orig_oi, QEMU_PLUGIN_MEM_W); +} + +void tcg_gen_qemu_st_i128_chk(TCGv_i128 val, TCGTemp *addr, TCGArg idx, + MemOp memop, TCGType addr_type) +{ + tcg_debug_assert(addr_type == tcg_ctx->addr_type); + tcg_debug_assert((memop & MO_SIZE) == MO_128); + tcg_debug_assert((memop & MO_SIGN) == 0); + tcg_gen_qemu_st_i128_int(val, addr, idx, memop); } static void tcg_gen_ext_i32(TCGv_i32 ret, TCGv_i32 val, MemOp opc) From patchwork Tue May 16 19:41:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682407 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp20495wrt; Tue, 16 May 2023 12:57:24 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4IsXhOo7kDv8fIPlo9LVDgiILPQ/z+Kh1yGaWxz9/EEpEp7sNEfUClY/dzqBiXtE9SQqkM X-Received: by 2002:ad4:5ce8:0:b0:5ef:8b3a:5d7 with SMTP id iv8-20020ad45ce8000000b005ef8b3a05d7mr65474779qvb.5.1684267044137; Tue, 16 May 2023 12:57:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267044; cv=none; d=google.com; s=arc-20160816; b=N8r8jB+eBx8XeYSoBGaZHh0kqvF3XBdAUg1daEpNVBh6/4R8vZhmrvLwYmNh0H0GCB EFNN3psE+2w9L+8plqWqtu+uGrntIhfRl7jiwch+njsfgvyc0GETnpEkVPNg0DE2ovcz SlftexoDp/Sr11pw4MlkwZzIWGM6VLI19xoSk0Uxzfno38W8QG0Z7UdF3zjcBtdsvCuv t1BnuqSDSOqb/7TBKWtT4tyaa9EXvd63eeSb1XsodKANVCzt02h27Pw+7boWHLxYIPDh zZLzPSqqtlNJT5KzoloJG5nwNUPMkBLWJJE8A2thohbuJHhSCkh6etGEj+lCdZq26h4b +v4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=UxwdqiALkoPj7zCiSvoG+AA0MPWC8VroDhK0qwZ8X3s=; b=MlPx5+WZVfI7r6X+k6YX/MMWJYHCs69HLH72kObLOhxSgwoC8HKJpQKrrt292FXVMC ERcsv/+Dtk5bm1ERUaoOffBm1ITFjttxvDHgrOu3mdZ7dTMn3PdVd5uDfbZKloR2dVbw 13aEX+C3NNtj1UflWSHBAwdrqg31O3ElCz47KAjnmn7bShJ9hvVXgjtrHmCTn6X0iREe 1GyXJlmQcr0ziLwonUer96dzaKzOFloJS6mroEewJmHZF8zu0OzNSUTNHN3SuwFVV2kX S4Dc8Tk2k6iylR5ljsIaqktutzlQPyXHCtJlOLbXg15DSIdObf7xRhGGivn/UhPKSc3h d/dA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="K7iw/zdQ"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id fv13-20020a056214240d00b005f03ef77442si12261001qvb.305.2023.05.16.12.57.23 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:57:24 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="K7iw/zdQ"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0dQ-0002Uz-D7; Tue, 16 May 2023 15:47:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cK-0000O9-HT for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:53 -0400 Received: from mail-pj1-x1029.google.com ([2607:f8b0:4864:20::1029]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cE-0003ls-7a for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:50 -0400 Received: by mail-pj1-x1029.google.com with SMTP id 98e67ed59e1d1-24df161f84bso39447a91.3 for ; Tue, 16 May 2023 12:45:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266344; x=1686858344; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UxwdqiALkoPj7zCiSvoG+AA0MPWC8VroDhK0qwZ8X3s=; b=K7iw/zdQrBnI8GDVFlyQx6tBh+Un2TcFnqQxCv6RsR8WjJbUaCdj+TX1fXibI/zqJY aPARHzdP46BCxbj8nDdvdYliXHXlCOH9gIPuxKI/YfbWHlEFyD/Hniu/UJgj7aga4tTV V+CuucMQYCgb1S9nPt2bBdmK5F42+yYpeN5vbzqbpZzCPwojXCB9dMH6IKXbLZBbU18M V8uOXnVuwHDEL4vSpujZ12rAsfJAplHu9TGph/mI+v5ZyVR6RcfLjbTVs7PnN9ZeQGTx ULUxcASSCeauZ64ZcwhuDRHpwijeAwDqYXEKVb6aRwmp1auuO5NIDUz/jRklkr8j+kDR 4Y/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266344; x=1686858344; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UxwdqiALkoPj7zCiSvoG+AA0MPWC8VroDhK0qwZ8X3s=; b=VgLgXAyUqFg6qu3TDTmkiAgXbv6nLZXoXRaNXPfLswWirseXs0QW46HDUduiLjb0dV FtgaKrEu22x9Au+yI55SGwWglPvIhiwN5pQrblurQ91E+3Rc3z3K/q7Y3XyRFqj/bjKN At+6Yz2DHVtyBpICvvmq+8NZcEpGEQDczp/cdcDhZsLBtBPaB8dtMdiLKHvn02eoNBhO ztL09tJaD5VjrMa26u6wLXoXhkUWNhGp9ykR4Y504peOT/bjdmb65PbxR/vOEke7+CpM Yy06qdntWgFfY1RrTgx+YQVMqlbJGnvhecsNYNODFF/yuRRkG1nren/JjmnTjGki0K0D 4QyA== X-Gm-Message-State: AC+VfDw77S3X56UyEWyxuZ62IoBVxy++hcq3Cv3uMYeQfv30aYOzqxBe qTiW+716RMrU9+UZ2ZUElU4ow/r7PncIu/N7Ov8= X-Received: by 2002:a17:90a:4611:b0:23d:3549:82b7 with SMTP id w17-20020a17090a461100b0023d354982b7mr36673367pjg.46.1684266344398; Tue, 16 May 2023 12:45:44 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:44 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 65/80] tcg: Remove TCGv from tcg_gen_atomic_* Date: Tue, 16 May 2023 12:41:30 -0700 Message-Id: <20230516194145.1749305-66-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1029; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1029.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Expand from TCGv to TCGTemp inline in the translators, and validate that the size matches tcg_ctx->addr_type. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- include/tcg/tcg-op.h | 184 ++++++++++++++++++++++++++++++---------- tcg/tcg-op-ldst.c | 198 ++++++++++++++++++++++++++++--------------- 2 files changed, 267 insertions(+), 115 deletions(-) diff --git a/include/tcg/tcg-op.h b/include/tcg/tcg-op.h index e556450ba9..35c5700183 100644 --- a/include/tcg/tcg-op.h +++ b/include/tcg/tcg-op.h @@ -858,56 +858,148 @@ tcg_gen_qemu_st_i128(TCGv_i128 v, TCGv a, TCGArg i, MemOp m) tcg_gen_qemu_st_i128_chk(v, tcgv_tl_temp(a), i, m, TCG_TYPE_TL); } -void tcg_gen_atomic_cmpxchg_i32(TCGv_i32, TCGv, TCGv_i32, TCGv_i32, - TCGArg, MemOp); -void tcg_gen_atomic_cmpxchg_i64(TCGv_i64, TCGv, TCGv_i64, TCGv_i64, - TCGArg, MemOp); -void tcg_gen_atomic_cmpxchg_i128(TCGv_i128, TCGv, TCGv_i128, TCGv_i128, - TCGArg, MemOp); +void tcg_gen_atomic_cmpxchg_i32_chk(TCGv_i32, TCGTemp *, TCGv_i32, TCGv_i32, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_cmpxchg_i64_chk(TCGv_i64, TCGTemp *, TCGv_i64, TCGv_i64, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_cmpxchg_i128_chk(TCGv_i128, TCGTemp *, TCGv_i128, + TCGv_i128, TCGArg, MemOp, TCGType); -void tcg_gen_nonatomic_cmpxchg_i32(TCGv_i32, TCGv, TCGv_i32, TCGv_i32, - TCGArg, MemOp); -void tcg_gen_nonatomic_cmpxchg_i64(TCGv_i64, TCGv, TCGv_i64, TCGv_i64, - TCGArg, MemOp); -void tcg_gen_nonatomic_cmpxchg_i128(TCGv_i128, TCGv, TCGv_i128, TCGv_i128, - TCGArg, MemOp); +void tcg_gen_nonatomic_cmpxchg_i32_chk(TCGv_i32, TCGTemp *, TCGv_i32, TCGv_i32, + TCGArg, MemOp, TCGType); +void tcg_gen_nonatomic_cmpxchg_i64_chk(TCGv_i64, TCGTemp *, TCGv_i64, TCGv_i64, + TCGArg, MemOp, TCGType); +void tcg_gen_nonatomic_cmpxchg_i128_chk(TCGv_i128, TCGTemp *, TCGv_i128, + TCGv_i128, TCGArg, MemOp, TCGType); -void tcg_gen_atomic_xchg_i32(TCGv_i32, TCGv, TCGv_i32, TCGArg, MemOp); -void tcg_gen_atomic_xchg_i64(TCGv_i64, TCGv, TCGv_i64, TCGArg, MemOp); +void tcg_gen_atomic_xchg_i32_chk(TCGv_i32, TCGTemp *, TCGv_i32, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_xchg_i64_chk(TCGv_i64, TCGTemp *, TCGv_i64, + TCGArg, MemOp, TCGType); -void tcg_gen_atomic_fetch_add_i32(TCGv_i32, TCGv, TCGv_i32, TCGArg, MemOp); -void tcg_gen_atomic_fetch_add_i64(TCGv_i64, TCGv, TCGv_i64, TCGArg, MemOp); -void tcg_gen_atomic_fetch_and_i32(TCGv_i32, TCGv, TCGv_i32, TCGArg, MemOp); -void tcg_gen_atomic_fetch_and_i64(TCGv_i64, TCGv, TCGv_i64, TCGArg, MemOp); -void tcg_gen_atomic_fetch_or_i32(TCGv_i32, TCGv, TCGv_i32, TCGArg, MemOp); -void tcg_gen_atomic_fetch_or_i64(TCGv_i64, TCGv, TCGv_i64, TCGArg, MemOp); -void tcg_gen_atomic_fetch_xor_i32(TCGv_i32, TCGv, TCGv_i32, TCGArg, MemOp); -void tcg_gen_atomic_fetch_xor_i64(TCGv_i64, TCGv, TCGv_i64, TCGArg, MemOp); -void tcg_gen_atomic_fetch_smin_i32(TCGv_i32, TCGv, TCGv_i32, TCGArg, MemOp); -void tcg_gen_atomic_fetch_smin_i64(TCGv_i64, TCGv, TCGv_i64, TCGArg, MemOp); -void tcg_gen_atomic_fetch_umin_i32(TCGv_i32, TCGv, TCGv_i32, TCGArg, MemOp); -void tcg_gen_atomic_fetch_umin_i64(TCGv_i64, TCGv, TCGv_i64, TCGArg, MemOp); -void tcg_gen_atomic_fetch_smax_i32(TCGv_i32, TCGv, TCGv_i32, TCGArg, MemOp); -void tcg_gen_atomic_fetch_smax_i64(TCGv_i64, TCGv, TCGv_i64, TCGArg, MemOp); -void tcg_gen_atomic_fetch_umax_i32(TCGv_i32, TCGv, TCGv_i32, TCGArg, MemOp); -void tcg_gen_atomic_fetch_umax_i64(TCGv_i64, TCGv, TCGv_i64, TCGArg, MemOp); +void tcg_gen_atomic_fetch_add_i32_chk(TCGv_i32, TCGTemp *, TCGv_i32, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_fetch_add_i64_chk(TCGv_i64, TCGTemp *, TCGv_i64, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_fetch_and_i32_chk(TCGv_i32, TCGTemp *, TCGv_i32, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_fetch_and_i64_chk(TCGv_i64, TCGTemp *, TCGv_i64, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_fetch_or_i32_chk(TCGv_i32, TCGTemp *, TCGv_i32, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_fetch_or_i64_chk(TCGv_i64, TCGTemp *, TCGv_i64, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_fetch_xor_i32_chk(TCGv_i32, TCGTemp *, TCGv_i32, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_fetch_xor_i64_chk(TCGv_i64, TCGTemp *, TCGv_i64, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_fetch_smin_i32_chk(TCGv_i32, TCGTemp *, TCGv_i32, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_fetch_smin_i64_chk(TCGv_i64, TCGTemp *, TCGv_i64, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_fetch_umin_i32_chk(TCGv_i32, TCGTemp *, TCGv_i32, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_fetch_umin_i64_chk(TCGv_i64, TCGTemp *, TCGv_i64, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_fetch_smax_i32_chk(TCGv_i32, TCGTemp *, TCGv_i32, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_fetch_smax_i64_chk(TCGv_i64, TCGTemp *, TCGv_i64, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_fetch_umax_i32_chk(TCGv_i32, TCGTemp *, TCGv_i32, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_fetch_umax_i64_chk(TCGv_i64, TCGTemp *, TCGv_i64, + TCGArg, MemOp, TCGType); -void tcg_gen_atomic_add_fetch_i32(TCGv_i32, TCGv, TCGv_i32, TCGArg, MemOp); -void tcg_gen_atomic_add_fetch_i64(TCGv_i64, TCGv, TCGv_i64, TCGArg, MemOp); -void tcg_gen_atomic_and_fetch_i32(TCGv_i32, TCGv, TCGv_i32, TCGArg, MemOp); -void tcg_gen_atomic_and_fetch_i64(TCGv_i64, TCGv, TCGv_i64, TCGArg, MemOp); -void tcg_gen_atomic_or_fetch_i32(TCGv_i32, TCGv, TCGv_i32, TCGArg, MemOp); -void tcg_gen_atomic_or_fetch_i64(TCGv_i64, TCGv, TCGv_i64, TCGArg, MemOp); -void tcg_gen_atomic_xor_fetch_i32(TCGv_i32, TCGv, TCGv_i32, TCGArg, MemOp); -void tcg_gen_atomic_xor_fetch_i64(TCGv_i64, TCGv, TCGv_i64, TCGArg, MemOp); -void tcg_gen_atomic_smin_fetch_i32(TCGv_i32, TCGv, TCGv_i32, TCGArg, MemOp); -void tcg_gen_atomic_smin_fetch_i64(TCGv_i64, TCGv, TCGv_i64, TCGArg, MemOp); -void tcg_gen_atomic_umin_fetch_i32(TCGv_i32, TCGv, TCGv_i32, TCGArg, MemOp); -void tcg_gen_atomic_umin_fetch_i64(TCGv_i64, TCGv, TCGv_i64, TCGArg, MemOp); -void tcg_gen_atomic_smax_fetch_i32(TCGv_i32, TCGv, TCGv_i32, TCGArg, MemOp); -void tcg_gen_atomic_smax_fetch_i64(TCGv_i64, TCGv, TCGv_i64, TCGArg, MemOp); -void tcg_gen_atomic_umax_fetch_i32(TCGv_i32, TCGv, TCGv_i32, TCGArg, MemOp); -void tcg_gen_atomic_umax_fetch_i64(TCGv_i64, TCGv, TCGv_i64, TCGArg, MemOp); +void tcg_gen_atomic_add_fetch_i32_chk(TCGv_i32, TCGTemp *, TCGv_i32, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_add_fetch_i64_chk(TCGv_i64, TCGTemp *, TCGv_i64, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_and_fetch_i32_chk(TCGv_i32, TCGTemp *, TCGv_i32, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_and_fetch_i64_chk(TCGv_i64, TCGTemp *, TCGv_i64, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_or_fetch_i32_chk(TCGv_i32, TCGTemp *, TCGv_i32, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_or_fetch_i64_chk(TCGv_i64, TCGTemp *, TCGv_i64, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_xor_fetch_i32_chk(TCGv_i32, TCGTemp *, TCGv_i32, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_xor_fetch_i64_chk(TCGv_i64, TCGTemp *, TCGv_i64, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_smin_fetch_i32_chk(TCGv_i32, TCGTemp *, TCGv_i32, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_smin_fetch_i64_chk(TCGv_i64, TCGTemp *, TCGv_i64, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_umin_fetch_i32_chk(TCGv_i32, TCGTemp *, TCGv_i32, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_umin_fetch_i64_chk(TCGv_i64, TCGTemp *, TCGv_i64, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_smax_fetch_i32_chk(TCGv_i32, TCGTemp *, TCGv_i32, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_smax_fetch_i64_chk(TCGv_i64, TCGTemp *, TCGv_i64, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_umax_fetch_i32_chk(TCGv_i32, TCGTemp *, TCGv_i32, + TCGArg, MemOp, TCGType); +void tcg_gen_atomic_umax_fetch_i64_chk(TCGv_i64, TCGTemp *, TCGv_i64, + TCGArg, MemOp, TCGType); + +#define DEF_ATOMIC2(N, S) \ + static inline void N##_##S(TCGv_##S r, TCGv a, TCGv_##S v, \ + TCGArg i, MemOp m) \ + { N##_##S##_chk(r, tcgv_tl_temp(a), v, i, m, TCG_TYPE_TL); } + +#define DEF_ATOMIC3(N, S) \ + static inline void N##_##S(TCGv_##S r, TCGv a, TCGv_##S o, \ + TCGv_##S n, TCGArg i, MemOp m) \ + { N##_##S##_chk(r, tcgv_tl_temp(a), o, n, i, m, TCG_TYPE_TL); } + +DEF_ATOMIC3(tcg_gen_atomic_cmpxchg, i32) +DEF_ATOMIC3(tcg_gen_atomic_cmpxchg, i64) +DEF_ATOMIC3(tcg_gen_atomic_cmpxchg, i128) + +DEF_ATOMIC3(tcg_gen_nonatomic_cmpxchg, i32) +DEF_ATOMIC3(tcg_gen_nonatomic_cmpxchg, i64) +DEF_ATOMIC3(tcg_gen_nonatomic_cmpxchg, i128) + +DEF_ATOMIC2(tcg_gen_atomic_xchg, i32) +DEF_ATOMIC2(tcg_gen_atomic_xchg, i64) + +DEF_ATOMIC2(tcg_gen_atomic_fetch_add, i32) +DEF_ATOMIC2(tcg_gen_atomic_fetch_add, i64) +DEF_ATOMIC2(tcg_gen_atomic_fetch_and, i32) +DEF_ATOMIC2(tcg_gen_atomic_fetch_and, i64) +DEF_ATOMIC2(tcg_gen_atomic_fetch_or, i32) +DEF_ATOMIC2(tcg_gen_atomic_fetch_or, i64) +DEF_ATOMIC2(tcg_gen_atomic_fetch_xor, i32) +DEF_ATOMIC2(tcg_gen_atomic_fetch_xor, i64) +DEF_ATOMIC2(tcg_gen_atomic_fetch_smin, i32) +DEF_ATOMIC2(tcg_gen_atomic_fetch_smin, i64) +DEF_ATOMIC2(tcg_gen_atomic_fetch_umin, i32) +DEF_ATOMIC2(tcg_gen_atomic_fetch_umin, i64) +DEF_ATOMIC2(tcg_gen_atomic_fetch_smax, i32) +DEF_ATOMIC2(tcg_gen_atomic_fetch_smax, i64) +DEF_ATOMIC2(tcg_gen_atomic_fetch_umax, i32) +DEF_ATOMIC2(tcg_gen_atomic_fetch_umax, i64) + +DEF_ATOMIC2(tcg_gen_atomic_add_fetch, i32) +DEF_ATOMIC2(tcg_gen_atomic_add_fetch, i64) +DEF_ATOMIC2(tcg_gen_atomic_and_fetch, i32) +DEF_ATOMIC2(tcg_gen_atomic_and_fetch, i64) +DEF_ATOMIC2(tcg_gen_atomic_or_fetch, i32) +DEF_ATOMIC2(tcg_gen_atomic_or_fetch, i64) +DEF_ATOMIC2(tcg_gen_atomic_xor_fetch, i32) +DEF_ATOMIC2(tcg_gen_atomic_xor_fetch, i64) +DEF_ATOMIC2(tcg_gen_atomic_smin_fetch, i32) +DEF_ATOMIC2(tcg_gen_atomic_smin_fetch, i64) +DEF_ATOMIC2(tcg_gen_atomic_umin_fetch, i32) +DEF_ATOMIC2(tcg_gen_atomic_umin_fetch, i64) +DEF_ATOMIC2(tcg_gen_atomic_smax_fetch, i32) +DEF_ATOMIC2(tcg_gen_atomic_smax_fetch, i64) +DEF_ATOMIC2(tcg_gen_atomic_umax_fetch, i32) +DEF_ATOMIC2(tcg_gen_atomic_umax_fetch, i64) + +#undef DEF_ATOMIC2 +#undef DEF_ATOMIC3 void tcg_gen_mov_vec(TCGv_vec, TCGv_vec); void tcg_gen_dup_i32_vec(unsigned vece, TCGv_vec, TCGv_i32); diff --git a/tcg/tcg-op-ldst.c b/tcg/tcg-op-ldst.c index 84a03bf6ed..281a80db41 100644 --- a/tcg/tcg-op-ldst.c +++ b/tcg/tcg-op-ldst.c @@ -443,22 +443,21 @@ static void canonicalize_memop_i128_as_i64(MemOp ret[2], MemOp orig) ret[1] = mop_2; } -static TCGv_i64 maybe_extend_addr64(TCGv addr) +static TCGv_i64 maybe_extend_addr64(TCGTemp *addr) { -#if TARGET_LONG_BITS == 32 - TCGv_i64 a64 = tcg_temp_ebb_new_i64(); - tcg_gen_extu_i32_i64(a64, addr); - return a64; -#else - return addr; -#endif + if (tcg_ctx->addr_type == TCG_TYPE_I32) { + TCGv_i64 a64 = tcg_temp_ebb_new_i64(); + tcg_gen_extu_i32_i64(a64, temp_tcgv_i32(addr)); + return a64; + } + return temp_tcgv_i64(addr); } static void maybe_free_addr64(TCGv_i64 a64) { -#if TARGET_LONG_BITS == 32 - tcg_temp_free_i64(a64); -#endif + if (tcg_ctx->addr_type == TCG_TYPE_I32) { + tcg_temp_free_i64(a64); + } } static void tcg_gen_qemu_ld_i128_int(TCGv_i128 val, TCGTemp *addr, @@ -742,17 +741,18 @@ static void * const table_cmpxchg[(MO_SIZE | MO_BSWAP) + 1] = { WITH_ATOMIC128([MO_128 | MO_BE] = gen_helper_atomic_cmpxchgo_be) }; -void tcg_gen_nonatomic_cmpxchg_i32(TCGv_i32 retv, TCGv addr, TCGv_i32 cmpv, - TCGv_i32 newv, TCGArg idx, MemOp memop) +static void tcg_gen_nonatomic_cmpxchg_i32_int(TCGv_i32 retv, TCGTemp *addr, + TCGv_i32 cmpv, TCGv_i32 newv, + TCGArg idx, MemOp memop) { TCGv_i32 t1 = tcg_temp_ebb_new_i32(); TCGv_i32 t2 = tcg_temp_ebb_new_i32(); tcg_gen_ext_i32(t2, cmpv, memop & MO_SIZE); - tcg_gen_qemu_ld_i32(t1, addr, idx, memop & ~MO_SIGN); + tcg_gen_qemu_ld_i32_int(t1, addr, idx, memop & ~MO_SIGN); tcg_gen_movcond_i32(TCG_COND_EQ, t2, t1, t2, newv, t1); - tcg_gen_qemu_st_i32(t2, addr, idx, memop); + tcg_gen_qemu_st_i32_int(t2, addr, idx, memop); tcg_temp_free_i32(t2); if (memop & MO_SIGN) { @@ -763,15 +763,26 @@ void tcg_gen_nonatomic_cmpxchg_i32(TCGv_i32 retv, TCGv addr, TCGv_i32 cmpv, tcg_temp_free_i32(t1); } -void tcg_gen_atomic_cmpxchg_i32(TCGv_i32 retv, TCGv addr, TCGv_i32 cmpv, - TCGv_i32 newv, TCGArg idx, MemOp memop) +void tcg_gen_nonatomic_cmpxchg_i32_chk(TCGv_i32 retv, TCGTemp *addr, + TCGv_i32 cmpv, TCGv_i32 newv, + TCGArg idx, MemOp memop, + TCGType addr_type) +{ + tcg_debug_assert(addr_type == tcg_ctx->addr_type); + tcg_debug_assert((memop & MO_SIZE) <= MO_32); + tcg_gen_nonatomic_cmpxchg_i32_int(retv, addr, cmpv, newv, idx, memop); +} + +static void tcg_gen_atomic_cmpxchg_i32_int(TCGv_i32 retv, TCGTemp *addr, + TCGv_i32 cmpv, TCGv_i32 newv, + TCGArg idx, MemOp memop) { gen_atomic_cx_i32 gen; TCGv_i64 a64; MemOpIdx oi; if (!(tcg_ctx->gen_tb->cflags & CF_PARALLEL)) { - tcg_gen_nonatomic_cmpxchg_i32(retv, addr, cmpv, newv, idx, memop); + tcg_gen_nonatomic_cmpxchg_i32_int(retv, addr, cmpv, newv, idx, memop); return; } @@ -789,14 +800,25 @@ void tcg_gen_atomic_cmpxchg_i32(TCGv_i32 retv, TCGv addr, TCGv_i32 cmpv, } } -void tcg_gen_nonatomic_cmpxchg_i64(TCGv_i64 retv, TCGv addr, TCGv_i64 cmpv, - TCGv_i64 newv, TCGArg idx, MemOp memop) +void tcg_gen_atomic_cmpxchg_i32_chk(TCGv_i32 retv, TCGTemp *addr, + TCGv_i32 cmpv, TCGv_i32 newv, + TCGArg idx, MemOp memop, + TCGType addr_type) +{ + tcg_debug_assert(addr_type == tcg_ctx->addr_type); + tcg_debug_assert((memop & MO_SIZE) <= MO_32); + tcg_gen_atomic_cmpxchg_i32_int(retv, addr, cmpv, newv, idx, memop); +} + +static void tcg_gen_nonatomic_cmpxchg_i64_int(TCGv_i64 retv, TCGTemp *addr, + TCGv_i64 cmpv, TCGv_i64 newv, + TCGArg idx, MemOp memop) { TCGv_i64 t1, t2; if (TCG_TARGET_REG_BITS == 32 && (memop & MO_SIZE) < MO_64) { - tcg_gen_nonatomic_cmpxchg_i32(TCGV_LOW(retv), addr, TCGV_LOW(cmpv), - TCGV_LOW(newv), idx, memop); + tcg_gen_nonatomic_cmpxchg_i32_int(TCGV_LOW(retv), addr, TCGV_LOW(cmpv), + TCGV_LOW(newv), idx, memop); if (memop & MO_SIGN) { tcg_gen_sari_i32(TCGV_HIGH(retv), TCGV_LOW(retv), 31); } else { @@ -810,9 +832,9 @@ void tcg_gen_nonatomic_cmpxchg_i64(TCGv_i64 retv, TCGv addr, TCGv_i64 cmpv, tcg_gen_ext_i64(t2, cmpv, memop & MO_SIZE); - tcg_gen_qemu_ld_i64(t1, addr, idx, memop & ~MO_SIGN); + tcg_gen_qemu_ld_i64_int(t1, addr, idx, memop & ~MO_SIGN); tcg_gen_movcond_i64(TCG_COND_EQ, t2, t1, t2, newv, t1); - tcg_gen_qemu_st_i64(t2, addr, idx, memop); + tcg_gen_qemu_st_i64_int(t2, addr, idx, memop); tcg_temp_free_i64(t2); if (memop & MO_SIGN) { @@ -823,11 +845,22 @@ void tcg_gen_nonatomic_cmpxchg_i64(TCGv_i64 retv, TCGv addr, TCGv_i64 cmpv, tcg_temp_free_i64(t1); } -void tcg_gen_atomic_cmpxchg_i64(TCGv_i64 retv, TCGv addr, TCGv_i64 cmpv, - TCGv_i64 newv, TCGArg idx, MemOp memop) +void tcg_gen_nonatomic_cmpxchg_i64_chk(TCGv_i64 retv, TCGTemp *addr, + TCGv_i64 cmpv, TCGv_i64 newv, + TCGArg idx, MemOp memop, + TCGType addr_type) +{ + tcg_debug_assert(addr_type == tcg_ctx->addr_type); + tcg_debug_assert((memop & MO_SIZE) <= MO_64); + tcg_gen_nonatomic_cmpxchg_i64_int(retv, addr, cmpv, newv, idx, memop); +} + +static void tcg_gen_atomic_cmpxchg_i64_int(TCGv_i64 retv, TCGTemp *addr, + TCGv_i64 cmpv, TCGv_i64 newv, + TCGArg idx, MemOp memop) { if (!(tcg_ctx->gen_tb->cflags & CF_PARALLEL)) { - tcg_gen_nonatomic_cmpxchg_i64(retv, addr, cmpv, newv, idx, memop); + tcg_gen_nonatomic_cmpxchg_i64_int(retv, addr, cmpv, newv, idx, memop); return; } @@ -856,8 +889,8 @@ void tcg_gen_atomic_cmpxchg_i64(TCGv_i64 retv, TCGv addr, TCGv_i64 cmpv, } if (TCG_TARGET_REG_BITS == 32) { - tcg_gen_atomic_cmpxchg_i32(TCGV_LOW(retv), addr, TCGV_LOW(cmpv), - TCGV_LOW(newv), idx, memop); + tcg_gen_atomic_cmpxchg_i32_int(TCGV_LOW(retv), addr, TCGV_LOW(cmpv), + TCGV_LOW(newv), idx, memop); if (memop & MO_SIGN) { tcg_gen_sari_i32(TCGV_HIGH(retv), TCGV_LOW(retv), 31); } else { @@ -870,7 +903,8 @@ void tcg_gen_atomic_cmpxchg_i64(TCGv_i64 retv, TCGv addr, TCGv_i64 cmpv, tcg_gen_extrl_i64_i32(c32, cmpv); tcg_gen_extrl_i64_i32(n32, newv); - tcg_gen_atomic_cmpxchg_i32(r32, addr, c32, n32, idx, memop & ~MO_SIGN); + tcg_gen_atomic_cmpxchg_i32_int(r32, addr, c32, n32, + idx, memop & ~MO_SIGN); tcg_temp_free_i32(c32); tcg_temp_free_i32(n32); @@ -883,8 +917,18 @@ void tcg_gen_atomic_cmpxchg_i64(TCGv_i64 retv, TCGv addr, TCGv_i64 cmpv, } } -void tcg_gen_nonatomic_cmpxchg_i128(TCGv_i128 retv, TCGv addr, TCGv_i128 cmpv, - TCGv_i128 newv, TCGArg idx, MemOp memop) +void tcg_gen_atomic_cmpxchg_i64_chk(TCGv_i64 retv, TCGTemp *addr, + TCGv_i64 cmpv, TCGv_i64 newv, + TCGArg idx, MemOp memop, TCGType addr_type) +{ + tcg_debug_assert(addr_type == tcg_ctx->addr_type); + tcg_debug_assert((memop & MO_SIZE) <= MO_64); + tcg_gen_atomic_cmpxchg_i64_int(retv, addr, cmpv, newv, idx, memop); +} + +static void tcg_gen_nonatomic_cmpxchg_i128_int(TCGv_i128 retv, TCGTemp *addr, + TCGv_i128 cmpv, TCGv_i128 newv, + TCGArg idx, MemOp memop) { if (TCG_TARGET_REG_BITS == 32) { /* Inline expansion below is simply too large for 32-bit hosts. */ @@ -892,12 +936,8 @@ void tcg_gen_nonatomic_cmpxchg_i128(TCGv_i128 retv, TCGv addr, TCGv_i128 cmpv, ? gen_helper_nonatomic_cmpxchgo_le : gen_helper_nonatomic_cmpxchgo_be); MemOpIdx oi = make_memop_idx(memop, idx); - TCGv_i64 a64; + TCGv_i64 a64 = maybe_extend_addr64(addr); - tcg_debug_assert((memop & MO_SIZE) == MO_128); - tcg_debug_assert((memop & MO_SIGN) == 0); - - a64 = maybe_extend_addr64(addr); gen(retv, cpu_env, a64, cmpv, newv, tcg_constant_i32(oi)); maybe_free_addr64(a64); } else { @@ -907,7 +947,7 @@ void tcg_gen_nonatomic_cmpxchg_i128(TCGv_i128 retv, TCGv addr, TCGv_i128 cmpv, TCGv_i64 t1 = tcg_temp_ebb_new_i64(); TCGv_i64 z = tcg_constant_i64(0); - tcg_gen_qemu_ld_i128(oldv, addr, idx, memop); + tcg_gen_qemu_ld_i128_int(oldv, addr, idx, memop); /* Compare i128 */ tcg_gen_xor_i64(t0, TCGV128_LOW(oldv), TCGV128_LOW(cmpv)); @@ -921,7 +961,7 @@ void tcg_gen_nonatomic_cmpxchg_i128(TCGv_i128 retv, TCGv addr, TCGv_i128 cmpv, TCGV128_HIGH(newv), TCGV128_HIGH(oldv)); /* Unconditional writeback. */ - tcg_gen_qemu_st_i128(tmpv, addr, idx, memop); + tcg_gen_qemu_st_i128_int(tmpv, addr, idx, memop); tcg_gen_mov_i128(retv, oldv); tcg_temp_free_i64(t0); @@ -931,20 +971,28 @@ void tcg_gen_nonatomic_cmpxchg_i128(TCGv_i128 retv, TCGv addr, TCGv_i128 cmpv, } } -void tcg_gen_atomic_cmpxchg_i128(TCGv_i128 retv, TCGv addr, TCGv_i128 cmpv, - TCGv_i128 newv, TCGArg idx, MemOp memop) +void tcg_gen_nonatomic_cmpxchg_i128_chk(TCGv_i128 retv, TCGTemp *addr, + TCGv_i128 cmpv, TCGv_i128 newv, + TCGArg idx, MemOp memop, + TCGType addr_type) +{ + tcg_debug_assert(addr_type == tcg_ctx->addr_type); + tcg_debug_assert((memop & (MO_SIZE | MO_SIGN)) == MO_128); + tcg_gen_nonatomic_cmpxchg_i128_int(retv, addr, cmpv, newv, idx, memop); +} + +static void tcg_gen_atomic_cmpxchg_i128_int(TCGv_i128 retv, TCGTemp *addr, + TCGv_i128 cmpv, TCGv_i128 newv, + TCGArg idx, MemOp memop) { gen_atomic_cx_i128 gen; if (!(tcg_ctx->gen_tb->cflags & CF_PARALLEL)) { - tcg_gen_nonatomic_cmpxchg_i128(retv, addr, cmpv, newv, idx, memop); + tcg_gen_nonatomic_cmpxchg_i128_int(retv, addr, cmpv, newv, idx, memop); return; } - tcg_debug_assert((memop & MO_SIZE) == MO_128); - tcg_debug_assert((memop & MO_SIGN) == 0); gen = table_cmpxchg[memop & (MO_SIZE | MO_BSWAP)]; - if (gen) { MemOpIdx oi = make_memop_idx(memop, idx); TCGv_i64 a64 = maybe_extend_addr64(addr); @@ -964,7 +1012,17 @@ void tcg_gen_atomic_cmpxchg_i128(TCGv_i128 retv, TCGv addr, TCGv_i128 cmpv, tcg_gen_movi_i64(TCGV128_HIGH(retv), 0); } -static void do_nonatomic_op_i32(TCGv_i32 ret, TCGv addr, TCGv_i32 val, +void tcg_gen_atomic_cmpxchg_i128_chk(TCGv_i128 retv, TCGTemp *addr, + TCGv_i128 cmpv, TCGv_i128 newv, + TCGArg idx, MemOp memop, + TCGType addr_type) +{ + tcg_debug_assert(addr_type == tcg_ctx->addr_type); + tcg_debug_assert((memop & (MO_SIZE | MO_SIGN)) == MO_128); + tcg_gen_atomic_cmpxchg_i128_int(retv, addr, cmpv, newv, idx, memop); +} + +static void do_nonatomic_op_i32(TCGv_i32 ret, TCGTemp *addr, TCGv_i32 val, TCGArg idx, MemOp memop, bool new_val, void (*gen)(TCGv_i32, TCGv_i32, TCGv_i32)) { @@ -973,17 +1031,17 @@ static void do_nonatomic_op_i32(TCGv_i32 ret, TCGv addr, TCGv_i32 val, memop = tcg_canonicalize_memop(memop, 0, 0); - tcg_gen_qemu_ld_i32(t1, addr, idx, memop); + tcg_gen_qemu_ld_i32_int(t1, addr, idx, memop); tcg_gen_ext_i32(t2, val, memop); gen(t2, t1, t2); - tcg_gen_qemu_st_i32(t2, addr, idx, memop); + tcg_gen_qemu_st_i32_int(t2, addr, idx, memop); tcg_gen_ext_i32(ret, (new_val ? t2 : t1), memop); tcg_temp_free_i32(t1); tcg_temp_free_i32(t2); } -static void do_atomic_op_i32(TCGv_i32 ret, TCGv addr, TCGv_i32 val, +static void do_atomic_op_i32(TCGv_i32 ret, TCGTemp *addr, TCGv_i32 val, TCGArg idx, MemOp memop, void * const table[]) { gen_atomic_op_i32 gen; @@ -1005,7 +1063,7 @@ static void do_atomic_op_i32(TCGv_i32 ret, TCGv addr, TCGv_i32 val, } } -static void do_nonatomic_op_i64(TCGv_i64 ret, TCGv addr, TCGv_i64 val, +static void do_nonatomic_op_i64(TCGv_i64 ret, TCGTemp *addr, TCGv_i64 val, TCGArg idx, MemOp memop, bool new_val, void (*gen)(TCGv_i64, TCGv_i64, TCGv_i64)) { @@ -1014,40 +1072,36 @@ static void do_nonatomic_op_i64(TCGv_i64 ret, TCGv addr, TCGv_i64 val, memop = tcg_canonicalize_memop(memop, 1, 0); - tcg_gen_qemu_ld_i64(t1, addr, idx, memop); + tcg_gen_qemu_ld_i64_int(t1, addr, idx, memop); tcg_gen_ext_i64(t2, val, memop); gen(t2, t1, t2); - tcg_gen_qemu_st_i64(t2, addr, idx, memop); + tcg_gen_qemu_st_i64_int(t2, addr, idx, memop); tcg_gen_ext_i64(ret, (new_val ? t2 : t1), memop); tcg_temp_free_i64(t1); tcg_temp_free_i64(t2); } -static void do_atomic_op_i64(TCGv_i64 ret, TCGv addr, TCGv_i64 val, +static void do_atomic_op_i64(TCGv_i64 ret, TCGTemp *addr, TCGv_i64 val, TCGArg idx, MemOp memop, void * const table[]) { memop = tcg_canonicalize_memop(memop, 1, 0); if ((memop & MO_SIZE) == MO_64) { -#ifdef CONFIG_ATOMIC64 - gen_atomic_op_i64 gen; - TCGv_i64 a64; - MemOpIdx oi; + gen_atomic_op_i64 gen = table[memop & (MO_SIZE | MO_BSWAP)]; - gen = table[memop & (MO_SIZE | MO_BSWAP)]; - tcg_debug_assert(gen != NULL); + if (gen) { + MemOpIdx oi = make_memop_idx(memop & ~MO_SIGN, idx); + TCGv_i64 a64 = maybe_extend_addr64(addr); + gen(ret, cpu_env, a64, val, tcg_constant_i32(oi)); + maybe_free_addr64(a64); + return; + } - oi = make_memop_idx(memop & ~MO_SIGN, idx); - a64 = maybe_extend_addr64(addr); - gen(ret, cpu_env, a64, val, tcg_constant_i32(oi)); - maybe_free_addr64(a64); -#else gen_helper_exit_atomic(cpu_env); /* Produce a result, so that we have a well-formed opcode stream with respect to uses of the result in the (dead) code following. */ tcg_gen_movi_i64(ret, 0); -#endif /* CONFIG_ATOMIC64 */ } else { TCGv_i32 v32 = tcg_temp_ebb_new_i32(); TCGv_i32 r32 = tcg_temp_ebb_new_i32(); @@ -1075,9 +1129,12 @@ static void * const table_##NAME[(MO_SIZE | MO_BSWAP) + 1] = { \ WITH_ATOMIC64([MO_64 | MO_LE] = gen_helper_atomic_##NAME##q_le) \ WITH_ATOMIC64([MO_64 | MO_BE] = gen_helper_atomic_##NAME##q_be) \ }; \ -void tcg_gen_atomic_##NAME##_i32 \ - (TCGv_i32 ret, TCGv addr, TCGv_i32 val, TCGArg idx, MemOp memop) \ +void tcg_gen_atomic_##NAME##_i32_chk(TCGv_i32 ret, TCGTemp *addr, \ + TCGv_i32 val, TCGArg idx, \ + MemOp memop, TCGType addr_type) \ { \ + tcg_debug_assert(addr_type == tcg_ctx->addr_type); \ + tcg_debug_assert((memop & MO_SIZE) <= MO_32); \ if (tcg_ctx->gen_tb->cflags & CF_PARALLEL) { \ do_atomic_op_i32(ret, addr, val, idx, memop, table_##NAME); \ } else { \ @@ -1085,9 +1142,12 @@ void tcg_gen_atomic_##NAME##_i32 \ tcg_gen_##OP##_i32); \ } \ } \ -void tcg_gen_atomic_##NAME##_i64 \ - (TCGv_i64 ret, TCGv addr, TCGv_i64 val, TCGArg idx, MemOp memop) \ +void tcg_gen_atomic_##NAME##_i64_chk(TCGv_i64 ret, TCGTemp *addr, \ + TCGv_i64 val, TCGArg idx, \ + MemOp memop, TCGType addr_type) \ { \ + tcg_debug_assert(addr_type == tcg_ctx->addr_type); \ + tcg_debug_assert((memop & MO_SIZE) <= MO_64); \ if (tcg_ctx->gen_tb->cflags & CF_PARALLEL) { \ do_atomic_op_i64(ret, addr, val, idx, memop, table_##NAME); \ } else { \ From patchwork Tue May 16 19:41:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682411 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp20530wrt; Tue, 16 May 2023 12:57:30 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7VZQy8FlCWg5mL+F2+onqKDJWe6mUJtw3OUtkw1bajxcoBbux7YWDZ5IF9/Xt34ksN2LRS X-Received: by 2002:a05:622a:110a:b0:3f5:269a:201f with SMTP id e10-20020a05622a110a00b003f5269a201fmr16409608qty.47.1684267050093; Tue, 16 May 2023 12:57:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267050; cv=none; d=google.com; s=arc-20160816; b=SqJSiABdm2ons6IENUPViBnrRNS6lZ2Yc7cD6uFsz0z3l2Wtp6Ndv2Eg2i8L89dJQ0 /9pI6YOwTWEWoci/aP+ohuBvMGJnrSJe7FxOQU/CtYx6CGev1mJWxNW7Zm3AZcp2qirz jfaRtTUGGFLXblz2YLJt59cM52YyF95uSbq/de2qDcjgIH8CSwT6YwD/oP/wMFioBa+G 9B7VM8wluTUgWHBawiOK++c+jQVJP73PZn4qTRhIPwcB+VOlLy6XNIYBCnDznZ1xfsRQ q2cPR93c1LTEQoGfENlKrmoud7EdtxpmTBsS10Eqjggq7m+CWQBVDA5G+S1woDnHXFN7 1fTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=lAxhKYsfUhN14zHRtzkICtce/Ie7f8eMZfNOpwXVrjE=; b=UN5ktTN+n7m+c/2O0aHJvbP6r1zfFfZFGqkk/R7ygv//NnWRZ8dfv+u0JHiAqyA122 4xS5iTBODFPDkZvfOHf7hu5MK0Ko8qTeHaYdTiPB/jri0GkoOx9xVN4RCic3Q3aLISju +XRNRC+o5YVEYEHVNVTHFxpmDJM+ivzmuhFrmRC4Axlrd4AWBgvFtYdYuXT1DrugWHP3 QqRojWvZlc3gxeodUgjZjjBI7Tfxv00tFs250cPg4FvXKF/QzAyO8MxsK/vJnWu9Dryh YN5DOrUZnCo6RJarkKni5DVGh4Rcgw8XtjPAUYxnJMp0GqpVDb7w52zkiZ4lupcFWK1v J4bA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=PHC1RDK6; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m21-20020ae9f215000000b007468a0431fasi247277qkg.77.2023.05.16.12.57.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:57:30 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=PHC1RDK6; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0eR-0002pA-2t; Tue, 16 May 2023 15:48:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cM-0000Tr-Jk for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:54 -0400 Received: from mail-pg1-x535.google.com ([2607:f8b0:4864:20::535]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cF-0003m9-D0 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:54 -0400 Received: by mail-pg1-x535.google.com with SMTP id 41be03b00d2f7-517ab9a4a13so13458209a12.1 for ; Tue, 16 May 2023 12:45:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266345; x=1686858345; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lAxhKYsfUhN14zHRtzkICtce/Ie7f8eMZfNOpwXVrjE=; b=PHC1RDK6KZW9AsGBZ7PpfUIPnh3uO1CapEcp+f4GRlRoAFaU7z4nKPJXqXaGEBM5uB EMu8C2dBNMUixGLxR7c9gMgy7iIXGQca83lwYqR7Lr20wrj3WSYQHC3XMmhgBr+/zAI3 Z8WksBeL0b+gJPpV8f1ls3JHeRHe3sHZQxOd0iDmFhdYWnloCw1rpANBHG8h5TIukVMn /DG/TBj1XD7aXlUK6pR9dVC/Pc+tCtcH+VcdTZ7fSWOx9QuGmwsFCSCaug8Yrevgf/Tc IMiewGJCLogV4T+IGUUb48y9vbil84VIt4Gffkd+I/4wE2x7NipI6QEI0yDFrLuQhw8W NR5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266345; x=1686858345; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lAxhKYsfUhN14zHRtzkICtce/Ie7f8eMZfNOpwXVrjE=; b=UlwdlK2v7uIn8DsB/v4mQ9NJEKYBQGd8pBLjilMMPiJ215OfOJeZ3Lha9Y8pDtUoxk hC57MqqDU6sqGH4ZnOwd/H7yqIcmnLQrPy3YY3n8sZEZFI0CnT91kVfoyvqD9g9QOGKC Zx+KxLp6gndlbR77yaiIkBAOpVcGPrsEV4wud9uSYLbQB/IOMrEFZz36/G9joQWxPPv2 N5qKE+kMsHYYcB5y2XmtKv0MJuRe3YkZ9xZOw+ADgzTMcHOiosmKZlLI9BnPN8JeQApw UQnETYJeFgsVvLDFMYMKhBvoazaIp5KpMNPAUxPXlOaXOF1dd8KVH2ctGaW2H85V4Nbt pL9Q== X-Gm-Message-State: AC+VfDydauRmBcdhUAkS2qgTLviNXUTpuoAzLxmiMkYd9YLBTHpULGvn 72t2Z36610p6IwQ7Binvlfddst3QEQJsnTIwfWk= X-Received: by 2002:a17:90b:354d:b0:253:2927:4a22 with SMTP id lt13-20020a17090b354d00b0025329274a22mr2156829pjb.48.1684266345171; Tue, 16 May 2023 12:45:45 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:44 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 66/80] tcg: Split INDEX_op_qemu_{ld, st}* for guest address size Date: Tue, 16 May 2023 12:41:31 -0700 Message-Id: <20230516194145.1749305-67-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::535; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x535.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org For 32-bit hosts, we cannot simply rely on TCGContext.addr_bits, as we need one or two host registers to represent the guest address. Create the new opcodes and update all users. Since we have not yet eliminated TARGET_LONG_BITS, only one of the two opcodes will ever be used, so we can get away with treating them the same in the backends. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- include/tcg/tcg-opc.h | 35 ++++++++---- tcg/optimize.c | 19 +++++-- tcg/tcg-op-ldst.c | 83 ++++++++++++++++++++++------- tcg/tcg.c | 42 ++++++++++----- tcg/tci.c | 32 +++++++---- tcg/aarch64/tcg-target.c.inc | 36 ++++++++----- tcg/arm/tcg-target.c.inc | 83 +++++++++++++++-------------- tcg/i386/tcg-target.c.inc | 91 ++++++++++++++++++++------------ tcg/loongarch64/tcg-target.c.inc | 24 ++++++--- tcg/mips/tcg-target.c.inc | 66 ++++++++++++++--------- tcg/ppc/tcg-target.c.inc | 91 +++++++++++++++++++------------- tcg/riscv/tcg-target.c.inc | 24 ++++++--- tcg/s390x/tcg-target.c.inc | 36 ++++++++----- tcg/sparc64/tcg-target.c.inc | 24 ++++++--- tcg/tci/tcg-target.c.inc | 44 ++++++++------- 15 files changed, 468 insertions(+), 262 deletions(-) diff --git a/include/tcg/tcg-opc.h b/include/tcg/tcg-opc.h index 29216366d2..21594c1590 100644 --- a/include/tcg/tcg-opc.h +++ b/include/tcg/tcg-opc.h @@ -186,7 +186,6 @@ DEF(muls2_i64, 2, 2, 0, IMPL64 | IMPL(TCG_TARGET_HAS_muls2_i64)) DEF(muluh_i64, 1, 2, 0, IMPL64 | IMPL(TCG_TARGET_HAS_muluh_i64)) DEF(mulsh_i64, 1, 2, 0, IMPL64 | IMPL(TCG_TARGET_HAS_mulsh_i64)) -#define TLADDR_ARGS (TARGET_LONG_BITS <= TCG_TARGET_REG_BITS ? 1 : 2) #define DATA64_ARGS (TCG_TARGET_REG_BITS == 64 ? 1 : 2) /* QEMU specific */ @@ -199,25 +198,44 @@ DEF(goto_ptr, 0, 1, 0, TCG_OPF_BB_EXIT | TCG_OPF_BB_END) DEF(plugin_cb_start, 0, 0, 3, TCG_OPF_NOT_PRESENT) DEF(plugin_cb_end, 0, 0, 0, TCG_OPF_NOT_PRESENT) -DEF(qemu_ld_i32, 1, TLADDR_ARGS, 1, +/* Replicate ld/st ops for 32 and 64-bit guest addresses. */ +DEF(qemu_ld_a32_i32, 1, 1, 1, TCG_OPF_CALL_CLOBBER | TCG_OPF_SIDE_EFFECTS) -DEF(qemu_st_i32, 0, TLADDR_ARGS + 1, 1, +DEF(qemu_st_a32_i32, 0, 1 + 1, 1, TCG_OPF_CALL_CLOBBER | TCG_OPF_SIDE_EFFECTS) -DEF(qemu_ld_i64, DATA64_ARGS, TLADDR_ARGS, 1, +DEF(qemu_ld_a32_i64, DATA64_ARGS, 1, 1, TCG_OPF_CALL_CLOBBER | TCG_OPF_SIDE_EFFECTS | TCG_OPF_64BIT) -DEF(qemu_st_i64, 0, TLADDR_ARGS + DATA64_ARGS, 1, +DEF(qemu_st_a32_i64, 0, DATA64_ARGS + 1, 1, + TCG_OPF_CALL_CLOBBER | TCG_OPF_SIDE_EFFECTS | TCG_OPF_64BIT) + +DEF(qemu_ld_a64_i32, 1, DATA64_ARGS, 1, + TCG_OPF_CALL_CLOBBER | TCG_OPF_SIDE_EFFECTS) +DEF(qemu_st_a64_i32, 0, 1 + DATA64_ARGS, 1, + TCG_OPF_CALL_CLOBBER | TCG_OPF_SIDE_EFFECTS) +DEF(qemu_ld_a64_i64, DATA64_ARGS, DATA64_ARGS, 1, + TCG_OPF_CALL_CLOBBER | TCG_OPF_SIDE_EFFECTS | TCG_OPF_64BIT) +DEF(qemu_st_a64_i64, 0, DATA64_ARGS + DATA64_ARGS, 1, TCG_OPF_CALL_CLOBBER | TCG_OPF_SIDE_EFFECTS | TCG_OPF_64BIT) /* Only used by i386 to cope with stupid register constraints. */ -DEF(qemu_st8_i32, 0, TLADDR_ARGS + 1, 1, +DEF(qemu_st8_a32_i32, 0, 1 + 1, 1, + TCG_OPF_CALL_CLOBBER | TCG_OPF_SIDE_EFFECTS | + IMPL(TCG_TARGET_HAS_qemu_st8_i32)) +DEF(qemu_st8_a64_i32, 0, 1 + DATA64_ARGS, 1, TCG_OPF_CALL_CLOBBER | TCG_OPF_SIDE_EFFECTS | IMPL(TCG_TARGET_HAS_qemu_st8_i32)) /* Only for 64-bit hosts at the moment. */ -DEF(qemu_ld_i128, 2, 1, 1, +DEF(qemu_ld_a32_i128, 2, 1, 1, TCG_OPF_CALL_CLOBBER | TCG_OPF_SIDE_EFFECTS | TCG_OPF_64BIT | IMPL(TCG_TARGET_HAS_qemu_ldst_i128)) -DEF(qemu_st_i128, 0, 3, 1, +DEF(qemu_ld_a64_i128, 2, 1, 1, + TCG_OPF_CALL_CLOBBER | TCG_OPF_SIDE_EFFECTS | TCG_OPF_64BIT | + IMPL(TCG_TARGET_HAS_qemu_ldst_i128)) +DEF(qemu_st_a32_i128, 0, 3, 1, + TCG_OPF_CALL_CLOBBER | TCG_OPF_SIDE_EFFECTS | TCG_OPF_64BIT | + IMPL(TCG_TARGET_HAS_qemu_ldst_i128)) +DEF(qemu_st_a64_i128, 0, 3, 1, TCG_OPF_CALL_CLOBBER | TCG_OPF_SIDE_EFFECTS | TCG_OPF_64BIT | IMPL(TCG_TARGET_HAS_qemu_ldst_i128)) @@ -291,7 +309,6 @@ DEF(tci_movi, 1, 0, 1, TCG_OPF_NOT_PRESENT) DEF(tci_movl, 1, 0, 1, TCG_OPF_NOT_PRESENT) #endif -#undef TLADDR_ARGS #undef DATA64_ARGS #undef IMPL #undef IMPL64 diff --git a/tcg/optimize.c b/tcg/optimize.c index 9614fa3638..bf975a3a6c 100644 --- a/tcg/optimize.c +++ b/tcg/optimize.c @@ -2184,13 +2184,22 @@ void tcg_optimize(TCGContext *s) CASE_OP_32_64_VEC(orc): done = fold_orc(&ctx, op); break; - case INDEX_op_qemu_ld_i32: - case INDEX_op_qemu_ld_i64: + case INDEX_op_qemu_ld_a32_i32: + case INDEX_op_qemu_ld_a64_i32: + case INDEX_op_qemu_ld_a32_i64: + case INDEX_op_qemu_ld_a64_i64: + case INDEX_op_qemu_ld_a32_i128: + case INDEX_op_qemu_ld_a64_i128: done = fold_qemu_ld(&ctx, op); break; - case INDEX_op_qemu_st_i32: - case INDEX_op_qemu_st8_i32: - case INDEX_op_qemu_st_i64: + case INDEX_op_qemu_st8_a32_i32: + case INDEX_op_qemu_st8_a64_i32: + case INDEX_op_qemu_st_a32_i32: + case INDEX_op_qemu_st_a64_i32: + case INDEX_op_qemu_st_a32_i64: + case INDEX_op_qemu_st_a64_i64: + case INDEX_op_qemu_st_a32_i128: + case INDEX_op_qemu_st_a64_i128: done = fold_qemu_st(&ctx, op); break; CASE_OP_32_64(rem): diff --git a/tcg/tcg-op-ldst.c b/tcg/tcg-op-ldst.c index 281a80db41..1ee21f68fc 100644 --- a/tcg/tcg-op-ldst.c +++ b/tcg/tcg-op-ldst.c @@ -164,6 +164,7 @@ static void tcg_gen_qemu_ld_i32_int(TCGv_i32 val, TCGTemp *addr, MemOp orig_memop; MemOpIdx orig_oi, oi; TCGv_i64 copy_addr; + TCGOpcode opc; tcg_gen_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); orig_memop = memop = tcg_canonicalize_memop(memop, 0, 0); @@ -179,7 +180,12 @@ static void tcg_gen_qemu_ld_i32_int(TCGv_i32 val, TCGTemp *addr, } copy_addr = plugin_maybe_preserve_addr(addr); - gen_ldst(INDEX_op_qemu_ld_i32, tcgv_i32_temp(val), NULL, addr, oi); + if (tcg_ctx->addr_type == TCG_TYPE_I32) { + opc = INDEX_op_qemu_ld_a32_i32; + } else { + opc = INDEX_op_qemu_ld_a64_i32; + } + gen_ldst(opc, tcgv_i32_temp(val), NULL, addr, oi); plugin_gen_mem_callbacks(copy_addr, addr, orig_oi, QEMU_PLUGIN_MEM_R); if ((orig_memop ^ memop) & MO_BSWAP) { @@ -235,9 +241,17 @@ static void tcg_gen_qemu_st_i32_int(TCGv_i32 val, TCGTemp *addr, } if (TCG_TARGET_HAS_qemu_st8_i32 && (memop & MO_SIZE) == MO_8) { - opc = INDEX_op_qemu_st8_i32; + if (tcg_ctx->addr_type == TCG_TYPE_I32) { + opc = INDEX_op_qemu_st8_a32_i32; + } else { + opc = INDEX_op_qemu_st8_a64_i32; + } } else { - opc = INDEX_op_qemu_st_i32; + if (tcg_ctx->addr_type == TCG_TYPE_I32) { + opc = INDEX_op_qemu_st_a32_i32; + } else { + opc = INDEX_op_qemu_st_a64_i32; + } } gen_ldst(opc, tcgv_i32_temp(val), NULL, addr, oi); plugin_gen_mem_callbacks(NULL, addr, orig_oi, QEMU_PLUGIN_MEM_W); @@ -261,6 +275,7 @@ static void tcg_gen_qemu_ld_i64_int(TCGv_i64 val, TCGTemp *addr, MemOp orig_memop; MemOpIdx orig_oi, oi; TCGv_i64 copy_addr; + TCGOpcode opc; if (TCG_TARGET_REG_BITS == 32 && (memop & MO_SIZE) < MO_64) { tcg_gen_qemu_ld_i32_int(TCGV_LOW(val), addr, idx, memop); @@ -286,7 +301,12 @@ static void tcg_gen_qemu_ld_i64_int(TCGv_i64 val, TCGTemp *addr, } copy_addr = plugin_maybe_preserve_addr(addr); - gen_ldst_i64(INDEX_op_qemu_ld_i64, val, addr, oi); + if (tcg_ctx->addr_type == TCG_TYPE_I32) { + opc = INDEX_op_qemu_ld_a32_i64; + } else { + opc = INDEX_op_qemu_ld_a64_i64; + } + gen_ldst_i64(opc, val, addr, oi); plugin_gen_mem_callbacks(copy_addr, addr, orig_oi, QEMU_PLUGIN_MEM_R); if ((orig_memop ^ memop) & MO_BSWAP) { @@ -322,6 +342,7 @@ static void tcg_gen_qemu_st_i64_int(TCGv_i64 val, TCGTemp *addr, { TCGv_i64 swap = NULL; MemOpIdx orig_oi, oi; + TCGOpcode opc; if (TCG_TARGET_REG_BITS == 32 && (memop & MO_SIZE) < MO_64) { tcg_gen_qemu_st_i32_int(TCGV_LOW(val), addr, idx, memop); @@ -352,7 +373,12 @@ static void tcg_gen_qemu_st_i64_int(TCGv_i64 val, TCGTemp *addr, oi = make_memop_idx(memop, idx); } - gen_ldst_i64(INDEX_op_qemu_st_i64, val, addr, oi); + if (tcg_ctx->addr_type == TCG_TYPE_I32) { + opc = INDEX_op_qemu_st_a32_i64; + } else { + opc = INDEX_op_qemu_st_a64_i64; + } + gen_ldst_i64(opc, val, addr, oi); plugin_gen_mem_callbacks(NULL, addr, orig_oi, QEMU_PLUGIN_MEM_W); if (swap) { @@ -465,6 +491,7 @@ static void tcg_gen_qemu_ld_i128_int(TCGv_i128 val, TCGTemp *addr, { const MemOpIdx orig_oi = make_memop_idx(memop, idx); TCGv_i64 ext_addr = NULL; + TCGOpcode opc; tcg_gen_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); @@ -484,8 +511,12 @@ static void tcg_gen_qemu_ld_i128_int(TCGv_i128 val, TCGTemp *addr, hi = TCGV128_HIGH(val); } - gen_ldst(INDEX_op_qemu_ld_i128, tcgv_i64_temp(lo), - tcgv_i64_temp(hi), addr, oi); + if (tcg_ctx->addr_type == TCG_TYPE_I32) { + opc = INDEX_op_qemu_ld_a32_i128; + } else { + opc = INDEX_op_qemu_ld_a64_i128; + } + gen_ldst(opc, tcgv_i64_temp(lo), tcgv_i64_temp(hi), addr, oi); if (need_bswap) { tcg_gen_bswap64_i64(lo, lo); @@ -501,6 +532,12 @@ static void tcg_gen_qemu_ld_i128_int(TCGv_i128 val, TCGTemp *addr, canonicalize_memop_i128_as_i64(mop, memop); need_bswap = (mop[0] ^ memop) & MO_BSWAP; + if (tcg_ctx->addr_type == TCG_TYPE_I32) { + opc = INDEX_op_qemu_ld_a32_i64; + } else { + opc = INDEX_op_qemu_ld_a64_i64; + } + /* * Since there are no global TCGv_i128, there is no visible state * changed if the second load faults. Load directly into the two @@ -515,7 +552,7 @@ static void tcg_gen_qemu_ld_i128_int(TCGv_i128 val, TCGTemp *addr, } oi = make_memop_idx(mop[0], idx); - gen_ldst_i64(INDEX_op_qemu_ld_i64, x, addr, oi); + gen_ldst_i64(opc, x, addr, oi); if (need_bswap) { tcg_gen_bswap64_i64(x, x); @@ -531,7 +568,7 @@ static void tcg_gen_qemu_ld_i128_int(TCGv_i128 val, TCGTemp *addr, addr_p8 = tcgv_i64_temp(t); } - gen_ldst_i64(INDEX_op_qemu_ld_i64, y, addr_p8, oi); + gen_ldst_i64(opc, y, addr_p8, oi); tcg_temp_free_internal(addr_p8); if (need_bswap) { @@ -564,6 +601,7 @@ static void tcg_gen_qemu_st_i128_int(TCGv_i128 val, TCGTemp *addr, { const MemOpIdx orig_oi = make_memop_idx(memop, idx); TCGv_i64 ext_addr = NULL; + TCGOpcode opc; tcg_gen_req_mo(TCG_MO_ST_LD | TCG_MO_ST_ST); @@ -586,8 +624,12 @@ static void tcg_gen_qemu_st_i128_int(TCGv_i128 val, TCGTemp *addr, hi = TCGV128_HIGH(val); } - gen_ldst(INDEX_op_qemu_st_i128, tcgv_i64_temp(lo), - tcgv_i64_temp(hi), addr, oi); + if (tcg_ctx->addr_type == TCG_TYPE_I32) { + opc = INDEX_op_qemu_st_a32_i128; + } else { + opc = INDEX_op_qemu_st_a64_i128; + } + gen_ldst(opc, tcgv_i64_temp(lo), tcgv_i64_temp(hi), addr, oi); if (need_bswap) { tcg_temp_free_i64(lo); @@ -600,6 +642,12 @@ static void tcg_gen_qemu_st_i128_int(TCGv_i128 val, TCGTemp *addr, canonicalize_memop_i128_as_i64(mop, memop); + if (tcg_ctx->addr_type == TCG_TYPE_I32) { + opc = INDEX_op_qemu_st_a32_i64; + } else { + opc = INDEX_op_qemu_st_a64_i64; + } + if ((memop & MO_BSWAP) == MO_LE) { x = TCGV128_LOW(val); y = TCGV128_HIGH(val); @@ -613,8 +661,8 @@ static void tcg_gen_qemu_st_i128_int(TCGv_i128 val, TCGTemp *addr, tcg_gen_bswap64_i64(b, x); x = b; } - gen_ldst_i64(INDEX_op_qemu_st_i64, x, addr, - make_memop_idx(mop[0], idx)); + + gen_ldst_i64(opc, x, addr, make_memop_idx(mop[0], idx)); if (tcg_ctx->addr_type == TCG_TYPE_I32) { TCGv_i32 t = tcg_temp_ebb_new_i32(); @@ -628,13 +676,10 @@ static void tcg_gen_qemu_st_i128_int(TCGv_i128 val, TCGTemp *addr, if (b) { tcg_gen_bswap64_i64(b, y); - y = b; - } - gen_ldst_i64(INDEX_op_qemu_st_i64, y, addr_p8, - make_memop_idx(mop[1], idx)); - - if (b) { + gen_ldst_i64(opc, b, addr_p8, make_memop_idx(mop[1], idx)); tcg_temp_free_i64(b); + } else { + gen_ldst_i64(opc, y, addr_p8, make_memop_idx(mop[1], idx)); } tcg_temp_free_internal(addr_p8); } else { diff --git a/tcg/tcg.c b/tcg/tcg.c index dfe5dde63d..5a2b2b1371 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -1839,17 +1839,24 @@ bool tcg_op_supported(TCGOpcode op) case INDEX_op_exit_tb: case INDEX_op_goto_tb: case INDEX_op_goto_ptr: - case INDEX_op_qemu_ld_i32: - case INDEX_op_qemu_st_i32: - case INDEX_op_qemu_ld_i64: - case INDEX_op_qemu_st_i64: + case INDEX_op_qemu_ld_a32_i32: + case INDEX_op_qemu_ld_a64_i32: + case INDEX_op_qemu_st_a32_i32: + case INDEX_op_qemu_st_a64_i32: + case INDEX_op_qemu_ld_a32_i64: + case INDEX_op_qemu_ld_a64_i64: + case INDEX_op_qemu_st_a32_i64: + case INDEX_op_qemu_st_a64_i64: return true; - case INDEX_op_qemu_st8_i32: + case INDEX_op_qemu_st8_a32_i32: + case INDEX_op_qemu_st8_a64_i32: return TCG_TARGET_HAS_qemu_st8_i32; - case INDEX_op_qemu_ld_i128: - case INDEX_op_qemu_st_i128: + case INDEX_op_qemu_ld_a32_i128: + case INDEX_op_qemu_ld_a64_i128: + case INDEX_op_qemu_st_a32_i128: + case INDEX_op_qemu_st_a64_i128: return TCG_TARGET_HAS_qemu_ldst_i128; case INDEX_op_mov_i32: @@ -2464,13 +2471,20 @@ static void tcg_dump_ops(TCGContext *s, FILE *f, bool have_prefs) } i = 1; break; - case INDEX_op_qemu_ld_i32: - case INDEX_op_qemu_st_i32: - case INDEX_op_qemu_st8_i32: - case INDEX_op_qemu_ld_i64: - case INDEX_op_qemu_st_i64: - case INDEX_op_qemu_ld_i128: - case INDEX_op_qemu_st_i128: + case INDEX_op_qemu_ld_a32_i32: + case INDEX_op_qemu_ld_a64_i32: + case INDEX_op_qemu_st_a32_i32: + case INDEX_op_qemu_st_a64_i32: + case INDEX_op_qemu_st8_a32_i32: + case INDEX_op_qemu_st8_a64_i32: + case INDEX_op_qemu_ld_a32_i64: + case INDEX_op_qemu_ld_a64_i64: + case INDEX_op_qemu_st_a32_i64: + case INDEX_op_qemu_st_a64_i64: + case INDEX_op_qemu_ld_a32_i128: + case INDEX_op_qemu_ld_a64_i128: + case INDEX_op_qemu_st_a32_i128: + case INDEX_op_qemu_st_a64_i128: { const char *s_al, *s_op, *s_at; MemOpIdx oi = op->args[k++]; diff --git a/tcg/tci.c b/tcg/tci.c index 15f2f8c463..742c791726 100644 --- a/tcg/tci.c +++ b/tcg/tci.c @@ -922,7 +922,8 @@ uintptr_t QEMU_DISABLE_CFI tcg_qemu_tb_exec(CPUArchState *env, tb_ptr = ptr; break; - case INDEX_op_qemu_ld_i32: + case INDEX_op_qemu_ld_a32_i32: + case INDEX_op_qemu_ld_a64_i32: if (TARGET_LONG_BITS <= TCG_TARGET_REG_BITS) { tci_args_rrm(insn, &r0, &r1, &oi); taddr = regs[r1]; @@ -934,7 +935,8 @@ uintptr_t QEMU_DISABLE_CFI tcg_qemu_tb_exec(CPUArchState *env, regs[r0] = tmp32; break; - case INDEX_op_qemu_ld_i64: + case INDEX_op_qemu_ld_a32_i64: + case INDEX_op_qemu_ld_a64_i64: if (TCG_TARGET_REG_BITS == 64) { tci_args_rrm(insn, &r0, &r1, &oi); taddr = regs[r1]; @@ -954,7 +956,8 @@ uintptr_t QEMU_DISABLE_CFI tcg_qemu_tb_exec(CPUArchState *env, } break; - case INDEX_op_qemu_st_i32: + case INDEX_op_qemu_st_a32_i32: + case INDEX_op_qemu_st_a64_i32: if (TARGET_LONG_BITS <= TCG_TARGET_REG_BITS) { tci_args_rrm(insn, &r0, &r1, &oi); taddr = regs[r1]; @@ -966,7 +969,8 @@ uintptr_t QEMU_DISABLE_CFI tcg_qemu_tb_exec(CPUArchState *env, tci_qemu_st(env, taddr, tmp32, oi, tb_ptr); break; - case INDEX_op_qemu_st_i64: + case INDEX_op_qemu_st_a32_i64: + case INDEX_op_qemu_st_a64_i64: if (TCG_TARGET_REG_BITS == 64) { tci_args_rrm(insn, &r0, &r1, &oi); taddr = regs[r1]; @@ -1251,15 +1255,21 @@ int print_insn_tci(bfd_vma addr, disassemble_info *info) str_r(r3), str_r(r4), str_r(r5)); break; - case INDEX_op_qemu_ld_i64: - case INDEX_op_qemu_st_i64: - len = DIV_ROUND_UP(64, TCG_TARGET_REG_BITS); + case INDEX_op_qemu_ld_a32_i32: + case INDEX_op_qemu_st_a32_i32: + len = 1 + 1; + goto do_qemu_ldst; + case INDEX_op_qemu_ld_a32_i64: + case INDEX_op_qemu_st_a32_i64: + case INDEX_op_qemu_ld_a64_i32: + case INDEX_op_qemu_st_a64_i32: + len = 1 + DIV_ROUND_UP(64, TCG_TARGET_REG_BITS); + goto do_qemu_ldst; + case INDEX_op_qemu_ld_a64_i64: + case INDEX_op_qemu_st_a64_i64: + len = 2 * DIV_ROUND_UP(64, TCG_TARGET_REG_BITS); goto do_qemu_ldst; - case INDEX_op_qemu_ld_i32: - case INDEX_op_qemu_st_i32: - len = 1; do_qemu_ldst: - len += DIV_ROUND_UP(TARGET_LONG_BITS, TCG_TARGET_REG_BITS); switch (len) { case 2: tci_args_rrm(insn, &r0, &r1, &oi); diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index 893b3514bb..cc0f55623b 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -2328,18 +2328,24 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, tcg_out_insn(s, 3506, CSEL, ext, a0, REG0(3), REG0(4), args[5]); break; - case INDEX_op_qemu_ld_i32: - case INDEX_op_qemu_ld_i64: + case INDEX_op_qemu_ld_a32_i32: + case INDEX_op_qemu_ld_a64_i32: + case INDEX_op_qemu_ld_a32_i64: + case INDEX_op_qemu_ld_a64_i64: tcg_out_qemu_ld(s, a0, a1, a2, ext); break; - case INDEX_op_qemu_st_i32: - case INDEX_op_qemu_st_i64: + case INDEX_op_qemu_st_a32_i32: + case INDEX_op_qemu_st_a64_i32: + case INDEX_op_qemu_st_a32_i64: + case INDEX_op_qemu_st_a64_i64: tcg_out_qemu_st(s, REG0(0), a1, a2, ext); break; - case INDEX_op_qemu_ld_i128: + case INDEX_op_qemu_ld_a32_i128: + case INDEX_op_qemu_ld_a64_i128: tcg_out_qemu_ld128(s, a0, a1, a2, args[3]); break; - case INDEX_op_qemu_st_i128: + case INDEX_op_qemu_st_a32_i128: + case INDEX_op_qemu_st_a64_i128: tcg_out_qemu_st128(s, REG0(0), REG0(1), a2, args[3]); break; @@ -2976,15 +2982,21 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_movcond_i64: return C_O1_I4(r, r, rA, rZ, rZ); - case INDEX_op_qemu_ld_i32: - case INDEX_op_qemu_ld_i64: + case INDEX_op_qemu_ld_a32_i32: + case INDEX_op_qemu_ld_a64_i32: + case INDEX_op_qemu_ld_a32_i64: + case INDEX_op_qemu_ld_a64_i64: return C_O1_I1(r, l); - case INDEX_op_qemu_ld_i128: + case INDEX_op_qemu_ld_a32_i128: + case INDEX_op_qemu_ld_a64_i128: return C_O2_I1(r, r, l); - case INDEX_op_qemu_st_i32: - case INDEX_op_qemu_st_i64: + case INDEX_op_qemu_st_a32_i32: + case INDEX_op_qemu_st_a64_i32: + case INDEX_op_qemu_st_a32_i64: + case INDEX_op_qemu_st_a64_i64: return C_O0_I2(lZ, l); - case INDEX_op_qemu_st_i128: + case INDEX_op_qemu_st_a32_i128: + case INDEX_op_qemu_st_a64_i128: return C_O0_I3(lZ, lZ, l); case INDEX_op_deposit_i32: diff --git a/tcg/arm/tcg-target.c.inc b/tcg/arm/tcg-target.c.inc index add8cc1fd5..47f3ff18fa 100644 --- a/tcg/arm/tcg-target.c.inc +++ b/tcg/arm/tcg-target.c.inc @@ -1985,41 +1985,36 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, ARITH_MOV, args[0], 0, 0); break; - case INDEX_op_qemu_ld_i32: - if (TARGET_LONG_BITS == 32) { - tcg_out_qemu_ld(s, args[0], -1, args[1], -1, - args[2], TCG_TYPE_I32); - } else { - tcg_out_qemu_ld(s, args[0], -1, args[1], args[2], - args[3], TCG_TYPE_I32); - } + case INDEX_op_qemu_ld_a32_i32: + tcg_out_qemu_ld(s, args[0], -1, args[1], -1, args[2], TCG_TYPE_I32); break; - case INDEX_op_qemu_ld_i64: - if (TARGET_LONG_BITS == 32) { - tcg_out_qemu_ld(s, args[0], args[1], args[2], -1, - args[3], TCG_TYPE_I64); - } else { - tcg_out_qemu_ld(s, args[0], args[1], args[2], args[3], - args[4], TCG_TYPE_I64); - } + case INDEX_op_qemu_ld_a64_i32: + tcg_out_qemu_ld(s, args[0], -1, args[1], args[2], + args[3], TCG_TYPE_I32); break; - case INDEX_op_qemu_st_i32: - if (TARGET_LONG_BITS == 32) { - tcg_out_qemu_st(s, args[0], -1, args[1], -1, - args[2], TCG_TYPE_I32); - } else { - tcg_out_qemu_st(s, args[0], -1, args[1], args[2], - args[3], TCG_TYPE_I32); - } + case INDEX_op_qemu_ld_a32_i64: + tcg_out_qemu_ld(s, args[0], args[1], args[2], -1, + args[3], TCG_TYPE_I64); break; - case INDEX_op_qemu_st_i64: - if (TARGET_LONG_BITS == 32) { - tcg_out_qemu_st(s, args[0], args[1], args[2], -1, - args[3], TCG_TYPE_I64); - } else { - tcg_out_qemu_st(s, args[0], args[1], args[2], args[3], - args[4], TCG_TYPE_I64); - } + case INDEX_op_qemu_ld_a64_i64: + tcg_out_qemu_ld(s, args[0], args[1], args[2], args[3], + args[4], TCG_TYPE_I64); + break; + + case INDEX_op_qemu_st_a32_i32: + tcg_out_qemu_st(s, args[0], -1, args[1], -1, args[2], TCG_TYPE_I32); + break; + case INDEX_op_qemu_st_a64_i32: + tcg_out_qemu_st(s, args[0], -1, args[1], args[2], + args[3], TCG_TYPE_I32); + break; + case INDEX_op_qemu_st_a32_i64: + tcg_out_qemu_st(s, args[0], args[1], args[2], -1, + args[3], TCG_TYPE_I64); + break; + case INDEX_op_qemu_st_a64_i64: + tcg_out_qemu_st(s, args[0], args[1], args[2], args[3], + args[4], TCG_TYPE_I64); break; case INDEX_op_bswap16_i32: @@ -2160,14 +2155,22 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_setcond2_i32: return C_O1_I4(r, r, r, rI, rI); - case INDEX_op_qemu_ld_i32: - return TARGET_LONG_BITS == 32 ? C_O1_I1(r, q) : C_O1_I2(r, q, q); - case INDEX_op_qemu_ld_i64: - return TARGET_LONG_BITS == 32 ? C_O2_I1(e, p, q) : C_O2_I2(e, p, q, q); - case INDEX_op_qemu_st_i32: - return TARGET_LONG_BITS == 32 ? C_O0_I2(q, q) : C_O0_I3(q, q, q); - case INDEX_op_qemu_st_i64: - return TARGET_LONG_BITS == 32 ? C_O0_I3(Q, p, q) : C_O0_I4(Q, p, q, q); + case INDEX_op_qemu_ld_a32_i32: + return C_O1_I1(r, q); + case INDEX_op_qemu_ld_a64_i32: + return C_O1_I2(r, q, q); + case INDEX_op_qemu_ld_a32_i64: + return C_O2_I1(e, p, q); + case INDEX_op_qemu_ld_a64_i64: + return C_O2_I2(e, p, q, q); + case INDEX_op_qemu_st_a32_i32: + return C_O0_I2(q, q); + case INDEX_op_qemu_st_a64_i32: + return C_O0_I3(q, q, q); + case INDEX_op_qemu_st_a32_i64: + return C_O0_I3(Q, p, q); + case INDEX_op_qemu_st_a64_i64: + return C_O0_I4(Q, p, q, q); case INDEX_op_st_vec: return C_O0_I2(w, r); diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index b66769952e..647c31fa23 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -2666,44 +2666,62 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, tcg_out_modrm(s, OPC_GRP3_Ev + rexw, EXT3_NOT, a0); break; - case INDEX_op_qemu_ld_i32: - if (TCG_TARGET_REG_BITS >= TARGET_LONG_BITS) { - tcg_out_qemu_ld(s, a0, -1, a1, -1, a2, TCG_TYPE_I32); - } else { + case INDEX_op_qemu_ld_a64_i32: + if (TCG_TARGET_REG_BITS == 32) { tcg_out_qemu_ld(s, a0, -1, a1, a2, args[3], TCG_TYPE_I32); + break; } + /* fall through */ + case INDEX_op_qemu_ld_a32_i32: + tcg_out_qemu_ld(s, a0, -1, a1, -1, a2, TCG_TYPE_I32); break; - case INDEX_op_qemu_ld_i64: + case INDEX_op_qemu_ld_a32_i64: if (TCG_TARGET_REG_BITS == 64) { tcg_out_qemu_ld(s, a0, -1, a1, -1, a2, TCG_TYPE_I64); - } else if (TARGET_LONG_BITS == 32) { + } else { tcg_out_qemu_ld(s, a0, a1, a2, -1, args[3], TCG_TYPE_I64); + } + break; + case INDEX_op_qemu_ld_a64_i64: + if (TCG_TARGET_REG_BITS == 64) { + tcg_out_qemu_ld(s, a0, -1, a1, -1, a2, TCG_TYPE_I64); } else { tcg_out_qemu_ld(s, a0, a1, a2, args[3], args[4], TCG_TYPE_I64); } break; - case INDEX_op_qemu_ld_i128: + case INDEX_op_qemu_ld_a32_i128: + case INDEX_op_qemu_ld_a64_i128: tcg_debug_assert(TCG_TARGET_REG_BITS == 64); tcg_out_qemu_ld(s, a0, a1, a2, -1, args[3], TCG_TYPE_I128); break; - case INDEX_op_qemu_st_i32: - case INDEX_op_qemu_st8_i32: - if (TCG_TARGET_REG_BITS >= TARGET_LONG_BITS) { - tcg_out_qemu_st(s, a0, -1, a1, -1, a2, TCG_TYPE_I32); - } else { + + case INDEX_op_qemu_st_a64_i32: + case INDEX_op_qemu_st8_a64_i32: + if (TCG_TARGET_REG_BITS == 32) { tcg_out_qemu_st(s, a0, -1, a1, a2, args[3], TCG_TYPE_I32); + break; } + /* fall through */ + case INDEX_op_qemu_st_a32_i32: + case INDEX_op_qemu_st8_a32_i32: + tcg_out_qemu_st(s, a0, -1, a1, -1, a2, TCG_TYPE_I32); break; - case INDEX_op_qemu_st_i64: + case INDEX_op_qemu_st_a32_i64: if (TCG_TARGET_REG_BITS == 64) { tcg_out_qemu_st(s, a0, -1, a1, -1, a2, TCG_TYPE_I64); - } else if (TARGET_LONG_BITS == 32) { + } else { tcg_out_qemu_st(s, a0, a1, a2, -1, args[3], TCG_TYPE_I64); + } + break; + case INDEX_op_qemu_st_a64_i64: + if (TCG_TARGET_REG_BITS == 64) { + tcg_out_qemu_st(s, a0, -1, a1, -1, a2, TCG_TYPE_I64); } else { tcg_out_qemu_st(s, a0, a1, a2, args[3], args[4], TCG_TYPE_I64); } break; - case INDEX_op_qemu_st_i128: + case INDEX_op_qemu_st_a32_i128: + case INDEX_op_qemu_st_a64_i128: tcg_debug_assert(TCG_TARGET_REG_BITS == 64); tcg_out_qemu_st(s, a0, a1, a2, -1, args[3], TCG_TYPE_I128); break; @@ -3380,31 +3398,36 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_clz_i64: return have_lzcnt ? C_N1_I2(r, r, rW) : C_N1_I2(r, r, r); - case INDEX_op_qemu_ld_i32: - return (TARGET_LONG_BITS <= TCG_TARGET_REG_BITS - ? C_O1_I1(r, L) : C_O1_I2(r, L, L)); + case INDEX_op_qemu_ld_a32_i32: + return C_O1_I1(r, L); + case INDEX_op_qemu_ld_a64_i32: + return TCG_TARGET_REG_BITS == 64 ? C_O1_I1(r, L) : C_O1_I2(r, L, L); - case INDEX_op_qemu_st_i32: - return (TARGET_LONG_BITS <= TCG_TARGET_REG_BITS - ? C_O0_I2(L, L) : C_O0_I3(L, L, L)); - case INDEX_op_qemu_st8_i32: - return (TARGET_LONG_BITS <= TCG_TARGET_REG_BITS - ? C_O0_I2(s, L) : C_O0_I3(s, L, L)); + case INDEX_op_qemu_st_a32_i32: + return C_O0_I2(L, L); + case INDEX_op_qemu_st_a64_i32: + return TCG_TARGET_REG_BITS == 64 ? C_O0_I2(L, L) : C_O0_I3(L, L, L); + case INDEX_op_qemu_st8_a32_i32: + return C_O0_I2(s, L); + case INDEX_op_qemu_st8_a64_i32: + return TCG_TARGET_REG_BITS == 64 ? C_O0_I2(s, L) : C_O0_I3(s, L, L); - case INDEX_op_qemu_ld_i64: - return (TCG_TARGET_REG_BITS == 64 ? C_O1_I1(r, L) - : TARGET_LONG_BITS <= TCG_TARGET_REG_BITS ? C_O2_I1(r, r, L) - : C_O2_I2(r, r, L, L)); + case INDEX_op_qemu_ld_a32_i64: + return TCG_TARGET_REG_BITS == 64 ? C_O1_I1(r, L) : C_O2_I1(r, r, L); + case INDEX_op_qemu_ld_a64_i64: + return TCG_TARGET_REG_BITS == 64 ? C_O1_I1(r, L) : C_O2_I2(r, r, L, L); - case INDEX_op_qemu_st_i64: - return (TCG_TARGET_REG_BITS == 64 ? C_O0_I2(L, L) - : TARGET_LONG_BITS <= TCG_TARGET_REG_BITS ? C_O0_I3(L, L, L) - : C_O0_I4(L, L, L, L)); + case INDEX_op_qemu_st_a32_i64: + return TCG_TARGET_REG_BITS == 64 ? C_O0_I2(L, L) : C_O0_I3(L, L, L); + case INDEX_op_qemu_st_a64_i64: + return TCG_TARGET_REG_BITS == 64 ? C_O0_I2(L, L) : C_O0_I4(L, L, L, L); - case INDEX_op_qemu_ld_i128: + case INDEX_op_qemu_ld_a32_i128: + case INDEX_op_qemu_ld_a64_i128: tcg_debug_assert(TCG_TARGET_REG_BITS == 64); return C_O2_I1(r, r, L); - case INDEX_op_qemu_st_i128: + case INDEX_op_qemu_st_a32_i128: + case INDEX_op_qemu_st_a64_i128: tcg_debug_assert(TCG_TARGET_REG_BITS == 64); return C_O0_I3(L, L, L); diff --git a/tcg/loongarch64/tcg-target.c.inc b/tcg/loongarch64/tcg-target.c.inc index 07d35f92fa..0c4ef72d6f 100644 --- a/tcg/loongarch64/tcg-target.c.inc +++ b/tcg/loongarch64/tcg-target.c.inc @@ -1443,16 +1443,20 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, tcg_out_ldst(s, OPC_ST_D, a0, a1, a2); break; - case INDEX_op_qemu_ld_i32: + case INDEX_op_qemu_ld_a32_i32: + case INDEX_op_qemu_ld_a64_i32: tcg_out_qemu_ld(s, a0, a1, a2, TCG_TYPE_I32); break; - case INDEX_op_qemu_ld_i64: + case INDEX_op_qemu_ld_a32_i64: + case INDEX_op_qemu_ld_a64_i64: tcg_out_qemu_ld(s, a0, a1, a2, TCG_TYPE_I64); break; - case INDEX_op_qemu_st_i32: + case INDEX_op_qemu_st_a32_i32: + case INDEX_op_qemu_st_a64_i32: tcg_out_qemu_st(s, a0, a1, a2, TCG_TYPE_I32); break; - case INDEX_op_qemu_st_i64: + case INDEX_op_qemu_st_a32_i64: + case INDEX_op_qemu_st_a64_i64: tcg_out_qemu_st(s, a0, a1, a2, TCG_TYPE_I64); break; @@ -1492,8 +1496,10 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_st32_i64: case INDEX_op_st_i32: case INDEX_op_st_i64: - case INDEX_op_qemu_st_i32: - case INDEX_op_qemu_st_i64: + case INDEX_op_qemu_st_a32_i32: + case INDEX_op_qemu_st_a64_i32: + case INDEX_op_qemu_st_a32_i64: + case INDEX_op_qemu_st_a64_i64: return C_O0_I2(rZ, r); case INDEX_op_brcond_i32: @@ -1535,8 +1541,10 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_ld32u_i64: case INDEX_op_ld_i32: case INDEX_op_ld_i64: - case INDEX_op_qemu_ld_i32: - case INDEX_op_qemu_ld_i64: + case INDEX_op_qemu_ld_a32_i32: + case INDEX_op_qemu_ld_a64_i32: + case INDEX_op_qemu_ld_a32_i64: + case INDEX_op_qemu_ld_a64_i64: return C_O1_I1(r, r); case INDEX_op_andc_i32: diff --git a/tcg/mips/tcg-target.c.inc b/tcg/mips/tcg-target.c.inc index 3f3fe5b991..7ff4e2ff71 100644 --- a/tcg/mips/tcg-target.c.inc +++ b/tcg/mips/tcg-target.c.inc @@ -1954,34 +1954,49 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, tcg_out_setcond2(s, args[5], a0, a1, a2, args[3], args[4]); break; - case INDEX_op_qemu_ld_i32: - if (TCG_TARGET_REG_BITS >= TARGET_LONG_BITS) { - tcg_out_qemu_ld(s, a0, 0, a1, 0, a2, TCG_TYPE_I32); - } else { + case INDEX_op_qemu_ld_a64_i32: + if (TCG_TARGET_REG_BITS == 32) { tcg_out_qemu_ld(s, a0, 0, a1, a2, args[3], TCG_TYPE_I32); + break; } + /* fall through */ + case INDEX_op_qemu_ld_a32_i32: + tcg_out_qemu_ld(s, a0, 0, a1, 0, a2, TCG_TYPE_I32); break; - case INDEX_op_qemu_ld_i64: + case INDEX_op_qemu_ld_a32_i64: if (TCG_TARGET_REG_BITS == 64) { tcg_out_qemu_ld(s, a0, 0, a1, 0, a2, TCG_TYPE_I64); - } else if (TARGET_LONG_BITS == 32) { + } else { tcg_out_qemu_ld(s, a0, a1, a2, 0, args[3], TCG_TYPE_I64); + } + break; + case INDEX_op_qemu_ld_a64_i64: + if (TCG_TARGET_REG_BITS == 64) { + tcg_out_qemu_ld(s, a0, 0, a1, 0, a2, TCG_TYPE_I64); } else { tcg_out_qemu_ld(s, a0, a1, a2, args[3], args[4], TCG_TYPE_I64); } break; - case INDEX_op_qemu_st_i32: - if (TCG_TARGET_REG_BITS >= TARGET_LONG_BITS) { - tcg_out_qemu_st(s, a0, 0, a1, 0, a2, TCG_TYPE_I32); - } else { + + case INDEX_op_qemu_st_a64_i32: + if (TCG_TARGET_REG_BITS == 32) { tcg_out_qemu_st(s, a0, 0, a1, a2, args[3], TCG_TYPE_I32); + break; } + /* fall through */ + case INDEX_op_qemu_st_a32_i32: + tcg_out_qemu_st(s, a0, 0, a1, 0, a2, TCG_TYPE_I32); break; - case INDEX_op_qemu_st_i64: + case INDEX_op_qemu_st_a32_i64: if (TCG_TARGET_REG_BITS == 64) { tcg_out_qemu_st(s, a0, 0, a1, 0, a2, TCG_TYPE_I64); - } else if (TARGET_LONG_BITS == 32) { + } else { tcg_out_qemu_st(s, a0, a1, a2, 0, args[3], TCG_TYPE_I64); + } + break; + case INDEX_op_qemu_st_a64_i64: + if (TCG_TARGET_REG_BITS == 64) { + tcg_out_qemu_st(s, a0, 0, a1, 0, a2, TCG_TYPE_I64); } else { tcg_out_qemu_st(s, a0, a1, a2, args[3], args[4], TCG_TYPE_I64); } @@ -2140,19 +2155,22 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_brcond2_i32: return C_O0_I4(rZ, rZ, rZ, rZ); - case INDEX_op_qemu_ld_i32: - return (TCG_TARGET_REG_BITS == 64 || TARGET_LONG_BITS == 32 - ? C_O1_I1(r, r) : C_O1_I2(r, r, r)); - case INDEX_op_qemu_st_i32: - return (TCG_TARGET_REG_BITS == 64 || TARGET_LONG_BITS == 32 - ? C_O0_I2(rZ, r) : C_O0_I3(rZ, r, r)); - case INDEX_op_qemu_ld_i64: - return (TCG_TARGET_REG_BITS == 64 ? C_O1_I1(r, r) - : TARGET_LONG_BITS == 32 ? C_O2_I1(r, r, r) - : C_O2_I2(r, r, r, r)); - case INDEX_op_qemu_st_i64: + case INDEX_op_qemu_ld_a32_i32: + return C_O1_I1(r, r); + case INDEX_op_qemu_ld_a64_i32: + return TCG_TARGET_REG_BITS == 64 ? C_O1_I1(r, r) : C_O1_I2(r, r, r); + case INDEX_op_qemu_st_a32_i32: + return C_O0_I2(rZ, r); + case INDEX_op_qemu_st_a64_i32: + return TCG_TARGET_REG_BITS == 64 ? C_O0_I2(rZ, r) : C_O0_I3(rZ, r, r); + case INDEX_op_qemu_ld_a32_i64: + return TCG_TARGET_REG_BITS == 64 ? C_O1_I1(r, r) : C_O2_I1(r, r, r); + case INDEX_op_qemu_ld_a64_i64: + return TCG_TARGET_REG_BITS == 64 ? C_O1_I1(r, r) : C_O2_I2(r, r, r, r); + case INDEX_op_qemu_st_a32_i64: + return TCG_TARGET_REG_BITS == 64 ? C_O0_I2(rZ, r) : C_O0_I3(rZ, rZ, r); + case INDEX_op_qemu_st_a64_i64: return (TCG_TARGET_REG_BITS == 64 ? C_O0_I2(rZ, r) - : TARGET_LONG_BITS == 32 ? C_O0_I3(rZ, rZ, r) : C_O0_I4(rZ, rZ, r, r)); default: diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc index c3a1527856..f74218b13f 100644 --- a/tcg/ppc/tcg-target.c.inc +++ b/tcg/ppc/tcg-target.c.inc @@ -2909,54 +2909,70 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, tcg_out32(s, MODUD | TAB(args[0], args[1], args[2])); break; - case INDEX_op_qemu_ld_i32: - if (TCG_TARGET_REG_BITS >= TARGET_LONG_BITS) { - tcg_out_qemu_ld(s, args[0], -1, args[1], -1, - args[2], TCG_TYPE_I32); - } else { + case INDEX_op_qemu_ld_a64_i32: + if (TCG_TARGET_REG_BITS == 32) { tcg_out_qemu_ld(s, args[0], -1, args[1], args[2], args[3], TCG_TYPE_I32); + break; } + /* fall through */ + case INDEX_op_qemu_ld_a32_i32: + tcg_out_qemu_ld(s, args[0], -1, args[1], -1, args[2], TCG_TYPE_I32); break; - case INDEX_op_qemu_ld_i64: + case INDEX_op_qemu_ld_a32_i64: if (TCG_TARGET_REG_BITS == 64) { tcg_out_qemu_ld(s, args[0], -1, args[1], -1, args[2], TCG_TYPE_I64); - } else if (TARGET_LONG_BITS == 32) { + } else { tcg_out_qemu_ld(s, args[0], args[1], args[2], -1, args[3], TCG_TYPE_I64); + } + break; + case INDEX_op_qemu_ld_a64_i64: + if (TCG_TARGET_REG_BITS == 64) { + tcg_out_qemu_ld(s, args[0], -1, args[1], -1, + args[2], TCG_TYPE_I64); } else { tcg_out_qemu_ld(s, args[0], args[1], args[2], args[3], args[4], TCG_TYPE_I64); } break; - case INDEX_op_qemu_ld_i128: + case INDEX_op_qemu_ld_a32_i128: + case INDEX_op_qemu_ld_a64_i128: tcg_debug_assert(TCG_TARGET_REG_BITS == 64); tcg_out_qemu_ldst_i128(s, args[0], args[1], args[2], args[3], true); break; - case INDEX_op_qemu_st_i32: - if (TCG_TARGET_REG_BITS >= TARGET_LONG_BITS) { - tcg_out_qemu_st(s, args[0], -1, args[1], -1, - args[2], TCG_TYPE_I32); - } else { + case INDEX_op_qemu_st_a64_i32: + if (TCG_TARGET_REG_BITS == 32) { tcg_out_qemu_st(s, args[0], -1, args[1], args[2], args[3], TCG_TYPE_I32); + break; } + /* fall through */ + case INDEX_op_qemu_st_a32_i32: + tcg_out_qemu_st(s, args[0], -1, args[1], -1, args[2], TCG_TYPE_I32); break; - case INDEX_op_qemu_st_i64: + case INDEX_op_qemu_st_a32_i64: if (TCG_TARGET_REG_BITS == 64) { tcg_out_qemu_st(s, args[0], -1, args[1], -1, args[2], TCG_TYPE_I64); - } else if (TARGET_LONG_BITS == 32) { + } else { tcg_out_qemu_st(s, args[0], args[1], args[2], -1, args[3], TCG_TYPE_I64); + } + break; + case INDEX_op_qemu_st_a64_i64: + if (TCG_TARGET_REG_BITS == 64) { + tcg_out_qemu_st(s, args[0], -1, args[1], -1, + args[2], TCG_TYPE_I64); } else { tcg_out_qemu_st(s, args[0], args[1], args[2], args[3], args[4], TCG_TYPE_I64); } break; - case INDEX_op_qemu_st_i128: + case INDEX_op_qemu_st_a32_i128: + case INDEX_op_qemu_st_a64_i128: tcg_debug_assert(TCG_TARGET_REG_BITS == 64); tcg_out_qemu_ldst_i128(s, args[0], args[1], args[2], args[3], false); break; @@ -3775,29 +3791,28 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_sub2_i32: return C_O2_I4(r, r, rI, rZM, r, r); - case INDEX_op_qemu_ld_i32: - return (TCG_TARGET_REG_BITS == 64 || TARGET_LONG_BITS == 32 - ? C_O1_I1(r, r) - : C_O1_I2(r, r, r)); - - case INDEX_op_qemu_st_i32: - return (TCG_TARGET_REG_BITS == 64 || TARGET_LONG_BITS == 32 - ? C_O0_I2(r, r) - : C_O0_I3(r, r, r)); - - case INDEX_op_qemu_ld_i64: - return (TCG_TARGET_REG_BITS == 64 ? C_O1_I1(r, r) - : TARGET_LONG_BITS == 32 ? C_O2_I1(r, r, r) - : C_O2_I2(r, r, r, r)); - - case INDEX_op_qemu_st_i64: - return (TCG_TARGET_REG_BITS == 64 ? C_O0_I2(r, r) - : TARGET_LONG_BITS == 32 ? C_O0_I3(r, r, r) - : C_O0_I4(r, r, r, r)); - - case INDEX_op_qemu_ld_i128: + case INDEX_op_qemu_ld_a32_i32: + return C_O1_I1(r, r); + case INDEX_op_qemu_ld_a64_i32: + return TCG_TARGET_REG_BITS == 64 ? C_O1_I1(r, r) : C_O1_I2(r, r, r); + case INDEX_op_qemu_ld_a32_i64: + return TCG_TARGET_REG_BITS == 64 ? C_O1_I1(r, r) : C_O2_I1(r, r, r); + case INDEX_op_qemu_ld_a64_i64: + return TCG_TARGET_REG_BITS == 64 ? C_O1_I1(r, r) : C_O2_I2(r, r, r, r); + case INDEX_op_qemu_ld_a32_i128: + case INDEX_op_qemu_ld_a64_i128: return C_O2_I1(o, m, r); - case INDEX_op_qemu_st_i128: + + case INDEX_op_qemu_st_a32_i32: + return C_O0_I2(r, r); + case INDEX_op_qemu_st_a64_i32: + return TCG_TARGET_REG_BITS == 64 ? C_O0_I2(r, r) : C_O0_I3(r, r, r); + case INDEX_op_qemu_st_a32_i64: + return TCG_TARGET_REG_BITS == 64 ? C_O0_I2(r, r) : C_O0_I3(r, r, r); + case INDEX_op_qemu_st_a64_i64: + return TCG_TARGET_REG_BITS == 64 ? C_O0_I2(r, r) : C_O0_I4(r, r, r, r); + case INDEX_op_qemu_st_a32_i128: + case INDEX_op_qemu_st_a64_i128: return C_O0_I3(o, m, r); case INDEX_op_add_vec: diff --git a/tcg/riscv/tcg-target.c.inc b/tcg/riscv/tcg-target.c.inc index 1fc1a9199b..de61edb5df 100644 --- a/tcg/riscv/tcg-target.c.inc +++ b/tcg/riscv/tcg-target.c.inc @@ -1382,16 +1382,20 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, tcg_out_setcond(s, args[3], a0, a1, a2); break; - case INDEX_op_qemu_ld_i32: + case INDEX_op_qemu_ld_a32_i32: + case INDEX_op_qemu_ld_a64_i32: tcg_out_qemu_ld(s, a0, a1, a2, TCG_TYPE_I32); break; - case INDEX_op_qemu_ld_i64: + case INDEX_op_qemu_ld_a32_i64: + case INDEX_op_qemu_ld_a64_i64: tcg_out_qemu_ld(s, a0, a1, a2, TCG_TYPE_I64); break; - case INDEX_op_qemu_st_i32: + case INDEX_op_qemu_st_a32_i32: + case INDEX_op_qemu_st_a64_i32: tcg_out_qemu_st(s, a0, a1, a2, TCG_TYPE_I32); break; - case INDEX_op_qemu_st_i64: + case INDEX_op_qemu_st_a32_i64: + case INDEX_op_qemu_st_a64_i64: tcg_out_qemu_st(s, a0, a1, a2, TCG_TYPE_I64); break; @@ -1533,11 +1537,15 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_sub2_i64: return C_O2_I4(r, r, rZ, rZ, rM, rM); - case INDEX_op_qemu_ld_i32: - case INDEX_op_qemu_ld_i64: + case INDEX_op_qemu_ld_a32_i32: + case INDEX_op_qemu_ld_a64_i32: + case INDEX_op_qemu_ld_a32_i64: + case INDEX_op_qemu_ld_a64_i64: return C_O1_I1(r, r); - case INDEX_op_qemu_st_i32: - case INDEX_op_qemu_st_i64: + case INDEX_op_qemu_st_a32_i32: + case INDEX_op_qemu_st_a64_i32: + case INDEX_op_qemu_st_a32_i64: + case INDEX_op_qemu_st_a64_i64: return C_O0_I2(rZ, r); default: diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc index 835daa51fa..7606073c81 100644 --- a/tcg/s390x/tcg-target.c.inc +++ b/tcg/s390x/tcg-target.c.inc @@ -2297,22 +2297,28 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, args[2], const_args[2], args[3], const_args[3], args[4]); break; - case INDEX_op_qemu_ld_i32: + case INDEX_op_qemu_ld_a32_i32: + case INDEX_op_qemu_ld_a64_i32: tcg_out_qemu_ld(s, args[0], args[1], args[2], TCG_TYPE_I32); break; - case INDEX_op_qemu_ld_i64: + case INDEX_op_qemu_ld_a32_i64: + case INDEX_op_qemu_ld_a64_i64: tcg_out_qemu_ld(s, args[0], args[1], args[2], TCG_TYPE_I64); break; - case INDEX_op_qemu_st_i32: + case INDEX_op_qemu_st_a32_i32: + case INDEX_op_qemu_st_a64_i32: tcg_out_qemu_st(s, args[0], args[1], args[2], TCG_TYPE_I32); break; - case INDEX_op_qemu_st_i64: + case INDEX_op_qemu_st_a32_i64: + case INDEX_op_qemu_st_a64_i64: tcg_out_qemu_st(s, args[0], args[1], args[2], TCG_TYPE_I64); break; - case INDEX_op_qemu_ld_i128: + case INDEX_op_qemu_ld_a32_i128: + case INDEX_op_qemu_ld_a64_i128: tcg_out_qemu_ldst_i128(s, args[0], args[1], args[2], args[3], true); break; - case INDEX_op_qemu_st_i128: + case INDEX_op_qemu_st_a32_i128: + case INDEX_op_qemu_st_a64_i128: tcg_out_qemu_ldst_i128(s, args[0], args[1], args[2], args[3], false); break; @@ -3186,15 +3192,21 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_ctpop_i64: return C_O1_I1(r, r); - case INDEX_op_qemu_ld_i32: - case INDEX_op_qemu_ld_i64: + case INDEX_op_qemu_ld_a32_i32: + case INDEX_op_qemu_ld_a64_i32: + case INDEX_op_qemu_ld_a32_i64: + case INDEX_op_qemu_ld_a64_i64: return C_O1_I1(r, r); - case INDEX_op_qemu_st_i64: - case INDEX_op_qemu_st_i32: + case INDEX_op_qemu_st_a32_i64: + case INDEX_op_qemu_st_a64_i64: + case INDEX_op_qemu_st_a32_i32: + case INDEX_op_qemu_st_a64_i32: return C_O0_I2(r, r); - case INDEX_op_qemu_ld_i128: + case INDEX_op_qemu_ld_a32_i128: + case INDEX_op_qemu_ld_a64_i128: return C_O2_I1(o, m, r); - case INDEX_op_qemu_st_i128: + case INDEX_op_qemu_st_a32_i128: + case INDEX_op_qemu_st_a64_i128: return C_O0_I3(o, m, r); case INDEX_op_deposit_i32: diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index 9676b745a2..6e6c26d470 100644 --- a/tcg/sparc64/tcg-target.c.inc +++ b/tcg/sparc64/tcg-target.c.inc @@ -1376,16 +1376,20 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, tcg_out_arithi(s, a1, a0, 32, SHIFT_SRLX); break; - case INDEX_op_qemu_ld_i32: + case INDEX_op_qemu_ld_a32_i32: + case INDEX_op_qemu_ld_a64_i32: tcg_out_qemu_ld(s, a0, a1, a2, TCG_TYPE_I32); break; - case INDEX_op_qemu_ld_i64: + case INDEX_op_qemu_ld_a32_i64: + case INDEX_op_qemu_ld_a64_i64: tcg_out_qemu_ld(s, a0, a1, a2, TCG_TYPE_I64); break; - case INDEX_op_qemu_st_i32: + case INDEX_op_qemu_st_a32_i32: + case INDEX_op_qemu_st_a64_i32: tcg_out_qemu_st(s, a0, a1, a2, TCG_TYPE_I32); break; - case INDEX_op_qemu_st_i64: + case INDEX_op_qemu_st_a32_i64: + case INDEX_op_qemu_st_a64_i64: tcg_out_qemu_st(s, a0, a1, a2, TCG_TYPE_I64); break; @@ -1507,8 +1511,10 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_extu_i32_i64: case INDEX_op_extrl_i64_i32: case INDEX_op_extrh_i64_i32: - case INDEX_op_qemu_ld_i32: - case INDEX_op_qemu_ld_i64: + case INDEX_op_qemu_ld_a32_i32: + case INDEX_op_qemu_ld_a64_i32: + case INDEX_op_qemu_ld_a32_i64: + case INDEX_op_qemu_ld_a64_i64: return C_O1_I1(r, r); case INDEX_op_st8_i32: @@ -1518,8 +1524,10 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_st_i32: case INDEX_op_st32_i64: case INDEX_op_st_i64: - case INDEX_op_qemu_st_i32: - case INDEX_op_qemu_st_i64: + case INDEX_op_qemu_st_a32_i32: + case INDEX_op_qemu_st_a64_i32: + case INDEX_op_qemu_st_a32_i64: + case INDEX_op_qemu_st_a64_i64: return C_O0_I2(rZ, r); case INDEX_op_add_i32: diff --git a/tcg/tci/tcg-target.c.inc b/tcg/tci/tcg-target.c.inc index 41fbf042da..586b2e6a08 100644 --- a/tcg/tci/tcg-target.c.inc +++ b/tcg/tci/tcg-target.c.inc @@ -156,22 +156,22 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_setcond2_i32: return C_O1_I4(r, r, r, r, r); - case INDEX_op_qemu_ld_i32: - return (TARGET_LONG_BITS <= TCG_TARGET_REG_BITS - ? C_O1_I1(r, r) - : C_O1_I2(r, r, r)); - case INDEX_op_qemu_ld_i64: - return (TCG_TARGET_REG_BITS == 64 ? C_O1_I1(r, r) - : TARGET_LONG_BITS <= TCG_TARGET_REG_BITS ? C_O2_I1(r, r, r) - : C_O2_I2(r, r, r, r)); - case INDEX_op_qemu_st_i32: - return (TARGET_LONG_BITS <= TCG_TARGET_REG_BITS - ? C_O0_I2(r, r) - : C_O0_I3(r, r, r)); - case INDEX_op_qemu_st_i64: - return (TCG_TARGET_REG_BITS == 64 ? C_O0_I2(r, r) - : TARGET_LONG_BITS <= TCG_TARGET_REG_BITS ? C_O0_I3(r, r, r) - : C_O0_I4(r, r, r, r)); + case INDEX_op_qemu_ld_a32_i32: + return C_O1_I1(r, r); + case INDEX_op_qemu_ld_a64_i32: + return TCG_TARGET_REG_BITS == 64 ? C_O1_I1(r, r) : C_O1_I2(r, r, r); + case INDEX_op_qemu_ld_a32_i64: + return TCG_TARGET_REG_BITS == 64 ? C_O1_I1(r, r) : C_O2_I1(r, r, r); + case INDEX_op_qemu_ld_a64_i64: + return TCG_TARGET_REG_BITS == 64 ? C_O1_I1(r, r) : C_O2_I2(r, r, r, r); + case INDEX_op_qemu_st_a32_i32: + return C_O0_I2(r, r); + case INDEX_op_qemu_st_a64_i32: + return TCG_TARGET_REG_BITS == 64 ? C_O0_I2(r, r) : C_O0_I3(r, r, r); + case INDEX_op_qemu_st_a32_i64: + return TCG_TARGET_REG_BITS == 64 ? C_O0_I2(r, r) : C_O0_I3(r, r, r); + case INDEX_op_qemu_st_a64_i64: + return TCG_TARGET_REG_BITS == 64 ? C_O0_I2(r, r) : C_O0_I4(r, r, r, r); default: g_assert_not_reached(); @@ -849,8 +849,10 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, tcg_out_op_rrrr(s, opc, args[0], args[1], args[2], args[3]); break; - case INDEX_op_qemu_ld_i32: - case INDEX_op_qemu_st_i32: + case INDEX_op_qemu_ld_a32_i32: + case INDEX_op_qemu_ld_a64_i32: + case INDEX_op_qemu_st_a32_i32: + case INDEX_op_qemu_st_a64_i32: if (TARGET_LONG_BITS <= TCG_TARGET_REG_BITS) { tcg_out_op_rrm(s, opc, args[0], args[1], args[2]); } else { @@ -858,8 +860,10 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, } break; - case INDEX_op_qemu_ld_i64: - case INDEX_op_qemu_st_i64: + case INDEX_op_qemu_ld_a32_i64: + case INDEX_op_qemu_ld_a64_i64: + case INDEX_op_qemu_st_a32_i64: + case INDEX_op_qemu_st_a64_i64: if (TCG_TARGET_REG_BITS == 64) { tcg_out_op_rrm(s, opc, args[0], args[1], args[2]); } else if (TARGET_LONG_BITS <= TCG_TARGET_REG_BITS) { From patchwork Tue May 16 19:41:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682417 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp20995wrt; Tue, 16 May 2023 12:59:15 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ53plWZUeSb2M4EokTheWjSSGNDCfx+R+iqqdtJaIiZ38smIeoHxrijTQZ8do8nAqd06G8g X-Received: by 2002:a1c:cc14:0:b0:3f4:20bd:ba46 with SMTP id h20-20020a1ccc14000000b003f420bdba46mr24621936wmb.5.1684267155732; Tue, 16 May 2023 12:59:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267155; cv=none; d=google.com; s=arc-20160816; b=m9hFtODW7SprVNfGR19jIuNJuRVKkKUwsUqm+bVH9pbuk0j9IAlUKyrku0dLUXSIwM SiqrwqizCdDfJxJx7kaZnP8tzKMNx7k6s1xROb9HCOQBuvcVzMs8eXloTDrrYbMM8cPk RI210U2Ap2wu9FKZSCQX/xTZXppo8Gw71vDwWmfvDjouVbOAOhhFPZKuI+NA4xN7kO4o QUBjqIW0Aiz7B6gMOz1lxGVAZd6c9O4ni57ibxBKRp29V+c0OuM915RtEe8zSX4j7sDl nVGd149OHcw/2CCpnHYVID9IPV+LbQ6B9rpNv5W/0/DJ9CJmKwTFAJG/4VnXjvxBhaHj LSiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=MS9Et0j5wf1gzEHwKSqAAQ7tPBGo83o+Uozzg1/3C6w=; b=wLxmsDJpya8px1jKkL5YKCmmQUe+nOYOsNlwBL2nNIXTiKC/mCy0PeJrrJiQSXTwWi io4rC6fqkmCyPX8g2VTEbFpP2Ql13AOYWyZozH/i62Q96fWYszdj2RWPIC0J3ADhP3YN Xm5mbyny5Ayc4ZkMvowt7F8ode1qX0TQj5Byyd6yM/fU13PYzxm2fTEHz5s8ohfEDyss ml+IKIAWlfFeTk2g1g62BM/vRcuG4Len957EoZLh+t82ZH+EccbhFhSxXQ2mNeFtgW4U pRB3O1jpBGP55ebo+4UXwIRiOzFXcomlq0Up+cRKoyzprBrm9O9/nfQojoYo4fV6fzQP 40Pg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qcTOn1hh; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m10-20020a05600c460a00b003f42c3c2416si78220wmo.232.2023.05.16.12.59.15 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:59:15 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qcTOn1hh; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0f3-0003ww-V0; Tue, 16 May 2023 15:48:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cU-0000i1-Bh for qemu-devel@nongnu.org; Tue, 16 May 2023 15:46:02 -0400 Received: from mail-pj1-x1034.google.com ([2607:f8b0:4864:20::1034]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cI-0003mS-3p for qemu-devel@nongnu.org; Tue, 16 May 2023 15:46:01 -0400 Received: by mail-pj1-x1034.google.com with SMTP id 98e67ed59e1d1-24e14a24c9dso58913a91.0 for ; Tue, 16 May 2023 12:45:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266346; x=1686858346; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MS9Et0j5wf1gzEHwKSqAAQ7tPBGo83o+Uozzg1/3C6w=; b=qcTOn1hh5/jVMPt2984DyQLjMmkP925dWNmrZZOXllUukAzc935hhBHvQnKAnCQ8Lm PwdjRJtAs3ENBwEqAfhaAU7l+gJd559OhkF1jjMR2wjU3i2Iq2JmxPbkQJvd5D/JbPlI XXszjQlbI/t6VeMBoURpNdJVlK12dnBVcBtfxcQ9pN7luc6E5ncVbHX3GtOga8fOA1Xf lWOUgApctAC3HwrZSTCZbOzPt5mZgO2GxZpcUT48oJuRJDEhidge2xQWYRldYNg1Ckgl q5TmuSSiiN+TP6N4aK1cKrGW41hVRTIONW99R85xGFQBGVHWPvL4LjOg3LX5+gMaC1EG npLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266346; x=1686858346; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MS9Et0j5wf1gzEHwKSqAAQ7tPBGo83o+Uozzg1/3C6w=; b=GcnlyBfb+t4RMSOnr1E90z1TBk1r+JDssb0ZvpRaLs+aRhlNSFLfmoh6oQp2bJ1ETu JNL59scnZLDBlUiV8VPBM1lkXqkjayLdMj50mO4u322poRaOC5kVoMCngoowwb6IhTvB aky1+5ZgHv9ahlQ8p1FH+gNVzQK3siYsq0Yp6fZNoS5xPIPqSrHddFVPcpt3tYUc3Sdk eEga1N28Nr8FAhQGdZDS1+ktemdxULkHWS2U0aptW5CgXR1QYBPSNqMZPoDAGymLABLv S7bWtfjHHJ5tvaINyLSjvGYc5zMvAFE5nF3vq9befii76+kqZT/4UMSgdYFnDGrqlvzo TVLg== X-Gm-Message-State: AC+VfDzLbl8dVQIR28OSDcCH6/BlKmYNa0uhHnjMkl/nfqcUy3R6TdrH XOpK7XhJeolL1Y4JSE0GcCXmMDoPHRtPKsxStlc= X-Received: by 2002:a17:90b:2289:b0:247:1233:9b28 with SMTP id kx9-20020a17090b228900b0024712339b28mr37631768pjb.17.1684266346015; Tue, 16 May 2023 12:45:46 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:45 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 67/80] tcg/tci: Elimnate TARGET_LONG_BITS, target_ulong Date: Tue, 16 May 2023 12:41:32 -0700 Message-Id: <20230516194145.1749305-68-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1034; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1034.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org We now have the address size as part of the opcode, so we no longer need to test TARGET_LONG_BITS. We can use uint64_t for target_ulong, as passed into load/store helpers. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- tcg/tci.c | 61 +++++++++++++++++++++++++--------------- tcg/tci/tcg-target.c.inc | 15 +++++----- 2 files changed, 46 insertions(+), 30 deletions(-) diff --git a/tcg/tci.c b/tcg/tci.c index 742c791726..bab4397bc5 100644 --- a/tcg/tci.c +++ b/tcg/tci.c @@ -286,7 +286,7 @@ static bool tci_compare64(uint64_t u0, uint64_t u1, TCGCond condition) return result; } -static uint64_t tci_qemu_ld(CPUArchState *env, target_ulong taddr, +static uint64_t tci_qemu_ld(CPUArchState *env, uint64_t taddr, MemOpIdx oi, const void *tb_ptr) { MemOp mop = get_memop(oi); @@ -312,7 +312,7 @@ static uint64_t tci_qemu_ld(CPUArchState *env, target_ulong taddr, } } -static void tci_qemu_st(CPUArchState *env, target_ulong taddr, uint64_t val, +static void tci_qemu_st(CPUArchState *env, uint64_t taddr, uint64_t val, MemOpIdx oi, const void *tb_ptr) { MemOp mop = get_memop(oi); @@ -372,10 +372,9 @@ uintptr_t QEMU_DISABLE_CFI tcg_qemu_tb_exec(CPUArchState *env, TCGReg r0, r1, r2, r3, r4, r5; tcg_target_ulong t1; TCGCond condition; - target_ulong taddr; uint8_t pos, len; uint32_t tmp32; - uint64_t tmp64; + uint64_t tmp64, taddr; uint64_t T1, T2; MemOpIdx oi; int32_t ofs; @@ -923,31 +922,40 @@ uintptr_t QEMU_DISABLE_CFI tcg_qemu_tb_exec(CPUArchState *env, break; case INDEX_op_qemu_ld_a32_i32: + tci_args_rrm(insn, &r0, &r1, &oi); + taddr = (uint32_t)regs[r1]; + goto do_ld_i32; case INDEX_op_qemu_ld_a64_i32: - if (TARGET_LONG_BITS <= TCG_TARGET_REG_BITS) { + if (TCG_TARGET_REG_BITS == 64) { tci_args_rrm(insn, &r0, &r1, &oi); taddr = regs[r1]; } else { tci_args_rrrm(insn, &r0, &r1, &r2, &oi); taddr = tci_uint64(regs[r2], regs[r1]); } - tmp32 = tci_qemu_ld(env, taddr, oi, tb_ptr); - regs[r0] = tmp32; + do_ld_i32: + regs[r0] = tci_qemu_ld(env, taddr, oi, tb_ptr); break; case INDEX_op_qemu_ld_a32_i64: + if (TCG_TARGET_REG_BITS == 64) { + tci_args_rrm(insn, &r0, &r1, &oi); + taddr = (uint32_t)regs[r1]; + } else { + tci_args_rrrm(insn, &r0, &r1, &r2, &oi); + taddr = (uint32_t)regs[r2]; + } + goto do_ld_i64; case INDEX_op_qemu_ld_a64_i64: if (TCG_TARGET_REG_BITS == 64) { tci_args_rrm(insn, &r0, &r1, &oi); taddr = regs[r1]; - } else if (TARGET_LONG_BITS <= TCG_TARGET_REG_BITS) { - tci_args_rrrm(insn, &r0, &r1, &r2, &oi); - taddr = regs[r2]; } else { tci_args_rrrrr(insn, &r0, &r1, &r2, &r3, &r4); taddr = tci_uint64(regs[r3], regs[r2]); oi = regs[r4]; } + do_ld_i64: tmp64 = tci_qemu_ld(env, taddr, oi, tb_ptr); if (TCG_TARGET_REG_BITS == 32) { tci_write_reg64(regs, r1, r0, tmp64); @@ -957,35 +965,44 @@ uintptr_t QEMU_DISABLE_CFI tcg_qemu_tb_exec(CPUArchState *env, break; case INDEX_op_qemu_st_a32_i32: + tci_args_rrm(insn, &r0, &r1, &oi); + taddr = (uint32_t)regs[r1]; + goto do_st_i32; case INDEX_op_qemu_st_a64_i32: - if (TARGET_LONG_BITS <= TCG_TARGET_REG_BITS) { + if (TCG_TARGET_REG_BITS == 64) { tci_args_rrm(insn, &r0, &r1, &oi); taddr = regs[r1]; } else { tci_args_rrrm(insn, &r0, &r1, &r2, &oi); taddr = tci_uint64(regs[r2], regs[r1]); } - tmp32 = regs[r0]; - tci_qemu_st(env, taddr, tmp32, oi, tb_ptr); + do_st_i32: + tci_qemu_st(env, taddr, regs[r0], oi, tb_ptr); break; case INDEX_op_qemu_st_a32_i64: + if (TCG_TARGET_REG_BITS == 64) { + tci_args_rrm(insn, &r0, &r1, &oi); + tmp64 = regs[r0]; + taddr = (uint32_t)regs[r1]; + } else { + tci_args_rrrm(insn, &r0, &r1, &r2, &oi); + tmp64 = tci_uint64(regs[r1], regs[r0]); + taddr = (uint32_t)regs[r2]; + } + goto do_st_i64; case INDEX_op_qemu_st_a64_i64: if (TCG_TARGET_REG_BITS == 64) { tci_args_rrm(insn, &r0, &r1, &oi); - taddr = regs[r1]; tmp64 = regs[r0]; + taddr = regs[r1]; } else { - if (TARGET_LONG_BITS <= TCG_TARGET_REG_BITS) { - tci_args_rrrm(insn, &r0, &r1, &r2, &oi); - taddr = regs[r2]; - } else { - tci_args_rrrrr(insn, &r0, &r1, &r2, &r3, &r4); - taddr = tci_uint64(regs[r3], regs[r2]); - oi = regs[r4]; - } + tci_args_rrrrr(insn, &r0, &r1, &r2, &r3, &r4); tmp64 = tci_uint64(regs[r1], regs[r0]); + taddr = tci_uint64(regs[r3], regs[r2]); + oi = regs[r4]; } + do_st_i64: tci_qemu_st(env, taddr, tmp64, oi, tb_ptr); break; diff --git a/tcg/tci/tcg-target.c.inc b/tcg/tci/tcg-target.c.inc index 586b2e6a08..c9516a5e8b 100644 --- a/tcg/tci/tcg-target.c.inc +++ b/tcg/tci/tcg-target.c.inc @@ -243,7 +243,7 @@ static bool patch_reloc(tcg_insn_unit *code_ptr, int type, return false; } -static void stack_bounds_check(TCGReg base, target_long offset) +static void stack_bounds_check(TCGReg base, intptr_t offset) { if (base == TCG_REG_CALL_STACK) { tcg_debug_assert(offset >= 0); @@ -850,24 +850,23 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, break; case INDEX_op_qemu_ld_a32_i32: - case INDEX_op_qemu_ld_a64_i32: case INDEX_op_qemu_st_a32_i32: + tcg_out_op_rrm(s, opc, args[0], args[1], args[2]); + break; + case INDEX_op_qemu_ld_a64_i32: case INDEX_op_qemu_st_a64_i32: - if (TARGET_LONG_BITS <= TCG_TARGET_REG_BITS) { + case INDEX_op_qemu_ld_a32_i64: + case INDEX_op_qemu_st_a32_i64: + if (TCG_TARGET_REG_BITS == 64) { tcg_out_op_rrm(s, opc, args[0], args[1], args[2]); } else { tcg_out_op_rrrm(s, opc, args[0], args[1], args[2], args[3]); } break; - - case INDEX_op_qemu_ld_a32_i64: case INDEX_op_qemu_ld_a64_i64: - case INDEX_op_qemu_st_a32_i64: case INDEX_op_qemu_st_a64_i64: if (TCG_TARGET_REG_BITS == 64) { tcg_out_op_rrm(s, opc, args[0], args[1], args[2]); - } else if (TARGET_LONG_BITS <= TCG_TARGET_REG_BITS) { - tcg_out_op_rrrm(s, opc, args[0], args[1], args[2], args[3]); } else { tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_TMP, args[4]); tcg_out_op_rrrrr(s, opc, args[0], args[1], From patchwork Tue May 16 19:41:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682424 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp21108wrt; Tue, 16 May 2023 12:59:37 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6GOMeHq6umTXvoJNCwEVsR52zGBnwW8+/SbzZ+bDF2BnJytdyI3Mp17PSFq4CQQ6ekF9eT X-Received: by 2002:a5d:560a:0:b0:304:6d32:d589 with SMTP id l10-20020a5d560a000000b003046d32d589mr26045016wrv.18.1684267177771; Tue, 16 May 2023 12:59:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267177; cv=none; d=google.com; s=arc-20160816; b=nliHsxvpgrzFWo8MuwlCWgk+Ip/kgp3C+KCCef8EGIVfHBcGpKFqAYRFY0pZODmLY+ 5W35KV0joqxvTyUcib5NYw/X6sr1mn0zcPO8z3FCj2I9X3HJvQUQRYpnavj2xvnfhBxS 0EgHhttvPsPrFwyinDpK3iP0Zfd5QTnOEO5U7RzglWWwbHkQDP/FIXNqGm/9NSrz14vK rehbkIIQru1lo9YHaBl6d1PauoxuBsrk25ZtVvRMKXWMMs3laNFZ0mHSVLaQoz3FVu0b Byvwv7tdPw/ibG+3UyYlyhpJ7ZQu6dXEks7a+NSR4YKxfbTt8p/X6Bk6xZZYBzIqHOVB h0tA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=BEmipnomtWF/XOvFu0hq64ZuWqb9TMH4BLWDViQ9M4Q=; b=rQNx419UUU4pr9jx0/G+xCeX3SUWXRzk0AMBxTCAi58lA/BvYASjVcsooS5FtCK/Gx oVwocdsPh+w0yVzd54Nvu4VebNdIUCFCAn2fsRjTgXlv0PdWwI5TtwtbysdVuPc37uoL TmQY4YygDELeNFIf2BtFF0rjhd4VF2sNrfC5dxCCgLzHYtSwMrE3+fU5w7BtdrdjsSPr sLq28lrLIWTMlumbuca3gqDQ/D8Ha0X0SgRs5Z/cy2HE2cBr1gPbXAoz0z5K3HAlIrGk k9hmFqa5zyLBUuE6n2VqHA67Bnx0vXprXzEwsv34V1gRR4Jw5d1M/ulJbfH88cqcI90v 2ghg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=g+w1sMes; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h9-20020adffa89000000b002f02dd5b8edsi162625wrr.920.2023.05.16.12.59.37 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:59:37 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=g+w1sMes; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0eH-0002kN-Fq; Tue, 16 May 2023 15:47:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cK-0000OX-K8 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:53 -0400 Received: from mail-pg1-x52c.google.com ([2607:f8b0:4864:20::52c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cI-0003mf-1j for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:52 -0400 Received: by mail-pg1-x52c.google.com with SMTP id 41be03b00d2f7-52c759b7d45so13442521a12.3 for ; Tue, 16 May 2023 12:45:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266347; x=1686858347; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BEmipnomtWF/XOvFu0hq64ZuWqb9TMH4BLWDViQ9M4Q=; b=g+w1sMesltXUG7tXcGv+EGEIQRhSNOQCeVSiFrCB6ZgiRX1iZcAZoh7VZdR8aF0ZF5 Ynuf2898tUuF6NAqL6YNne2NhBY/C+2/F2QzE11BQTRKKhoNvmNPWwe1sZMT8tH851XR fxxM6vKy09+cDNLqqRlk72Qm0Hp+3btOg0IVdXza1iBNTPQMJvzHrDDiNfJ7l7OGnrd6 /nqP2D/zPGlwDz9oWKiIDPYI59UstoRS4P4nAqW6y19jZyg3uA2vRlqTOOk+0x9Dl6mG fBU5P1zSZx7vpFPQXEphtcTHWe5F5LkVB4BzY0x0vJdWR2dZSardl9dMeQeNauTP4P5D qlTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266347; x=1686858347; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BEmipnomtWF/XOvFu0hq64ZuWqb9TMH4BLWDViQ9M4Q=; b=WZAeTq59w3fdJoCSmNgpau5pjwm5BsD2zLezgJ/e7XBPRPa9GZa3xkPDdd09RkInk8 lyG1nsLU5hf3oy0kvKZ8vGpxseWrNSVf5cwvx5htmQA7Uf2xGF65/YkFFOGzdprS4TEY U4/20gbUHvSVHHOwGhI2lpJUM94zS92ycbn4x1F+X1uBxXRQudiWR6ruBCBla5newtIm RZtDXbQFSLewiR0nFAI333v73usNdxjYTFbKUNS6Bn+gCCbQLgNSloT8E8CirFPYWFPk 0+i50Hvw3Fxv/uZVQcNnva1qnt+jEpvNO5D5Ma5qPey24UmWDb9t6cL/8RngLiDWevcA qmWQ== X-Gm-Message-State: AC+VfDydVmVeAIF2HTqdvvsk8E6vurKO1kMCFI+DJiLA1Fpdvowohkam sSFhUq9VqcnIO+65kOZv2X2pEJSjoEqs9ToDMFU= X-Received: by 2002:a17:90b:4b8d:b0:253:27b4:cd4d with SMTP id lr13-20020a17090b4b8d00b0025327b4cd4dmr2392078pjb.27.1684266347030; Tue, 16 May 2023 12:45:47 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:46 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 68/80] tcg/i386: Always enable TCG_TARGET_HAS_extr[lh]_i64_i32 Date: Tue, 16 May 2023 12:41:33 -0700 Message-Id: <20230516194145.1749305-69-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52c; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Keep all 32-bit values zero extended in the register, not solely when addresses are 32 bits. This eliminates a dependency on TARGET_LONG_BITS. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index 7f69997e30..7ebd6e5135 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -154,9 +154,9 @@ extern bool have_atomic16; #define TCG_TARGET_HAS_mulsh_i32 0 #if TCG_TARGET_REG_BITS == 64 -/* Keep target addresses zero-extended in a register. */ -#define TCG_TARGET_HAS_extrl_i64_i32 (TARGET_LONG_BITS == 32) -#define TCG_TARGET_HAS_extrh_i64_i32 (TARGET_LONG_BITS == 32) +/* Keep 32-bit values zero-extended in a register. */ +#define TCG_TARGET_HAS_extrl_i64_i32 1 +#define TCG_TARGET_HAS_extrh_i64_i32 1 #define TCG_TARGET_HAS_div2_i64 1 #define TCG_TARGET_HAS_rot_i64 1 #define TCG_TARGET_HAS_ext8s_i64 1 From patchwork Tue May 16 19:41:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682391 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp18974wrt; Tue, 16 May 2023 12:51:59 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4jF6pbrW/2zj8PYVdHem/eWw/O4hs95qjzTGlFLiThZGyVHVIbR89i8Sh4jnHzwgqbLQ+B X-Received: by 2002:a05:6214:2a83:b0:61b:68e1:1211 with SMTP id jr3-20020a0562142a8300b0061b68e11211mr43938461qvb.10.1684266719255; Tue, 16 May 2023 12:51:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266719; cv=none; d=google.com; s=arc-20160816; b=bkWbO4hReQVNty0gX4CO6DFLutJd8YtJqhRjROHJopBzjw8lt92Hi1PtCbgn6/9N+3 6KAawHR1xk02x5h+SxFGgeZCAltYfVLJsZRQijEDHnYwhA2KD41nqDblAafjwmbXAsYc PSsC0oktlmUl0fjP1atWaEbZmWeSRDXS3Xk6S0Sco0ZZJ8QnzRRKBzphs/LA2g9cIKcs JzFntecxBQQv+7KdwHSBAgJE1IvCChQLA/Xcs/q3y7qgAcitKmAMzuB+A4zrDGNwwiFz 9zzzIU8djycjc5A5skOOWArK7bL8T2vP2iS8uNKUqneD/dT9PnUtO1XTrty6OkJyLaSD N7Lg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=k62oH3YiaxNUbwFSA39EGQkVtGdW8K0k9LWPenWZTMA=; b=rKUsb4p70mwOiC8W8ap/9i9uheXosIF+bCa6osr5x+T7e634UOklVw3dSneHerVzRk mrft/bPzhfSpY+AUGM0iRIOcptIZnvtTvnyQhslgtlCd+0XCp7YldmR3CbO0DfP6sCZ3 B3nCDKFuFymmHWWb/8h8MLwthpORfU9BVBBQwxfLI5+Mo2SbsKCFWJHPqSzYALZ4eGyg OT3sit5pJUlGHPs0EO3t4yVa2vWkxdfnNtgVl/ViWfO3ERs+ZNyHaGSnUqJsCVRoHIiU Y8jDEx8Im2hl26YCDP/8d0JNXv1TemyG943uCoeDMufXIg297UmULQOuCyPe5pMimYNN X3Og== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RKAXd0wY; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 2-20020a0562140cc200b00623851987b4si25073qvx.153.2023.05.16.12.51.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:51:59 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RKAXd0wY; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0eu-0003On-QN; Tue, 16 May 2023 15:48:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cT-0000gz-J7 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:46:02 -0400 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cI-0003mp-3j for qemu-devel@nongnu.org; Tue, 16 May 2023 15:46:01 -0400 Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-1ae4f28454bso401915ad.3 for ; Tue, 16 May 2023 12:45:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266348; x=1686858348; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=k62oH3YiaxNUbwFSA39EGQkVtGdW8K0k9LWPenWZTMA=; b=RKAXd0wYtIZON71lvjZHeleJ5sGh12TC4jaw9ldIAH5wCrxUuNqnvrpHhN8UzpS2Zp h1PLPI08Eozcy41JUc41iCLo+hnmK23+aLDl+guXpkgvI/HheFfnoGYxwVA/ztoxqHtB hoS6theMXCpFYZukFLRxOovTpby291DpZSQhq8X1ch9VjTAf0goKNwGA+vLHXgP01yrO lc5yF+U8wscwHV1E/GyLOraBSysJmsCtqBKjevCkbjLImdYvRlRlv1/ai8YcozbyrdlS ckNrbrYwVf6BH0HQDgcw8+TmbtPp4vG0HLdIc3RqulC1qD3WG7KcFX8nqj3XhbOZBFg7 /VJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266348; x=1686858348; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=k62oH3YiaxNUbwFSA39EGQkVtGdW8K0k9LWPenWZTMA=; b=hCZbh4E+r5HxrksDR8cLIV42+N3Gx5rot+urZUsbou86jTIJcX0yNtleuhuxlCGJO9 e6g1kJiYTYwFCeTx72D1gGaY4eHHAH5bRL6QolP1APAisZoptzBn9MbZTGpHMeiP2w7E MuoHErOqAtiAV1d7m/tMwgLsMZsS2QfgGxZ8Ype4kBmuboIoBzg5k32LpY9wQhVGpZP0 qxmdVSWCGMQJQthT4mTc7RU9Kz3t6w8bqfPSONrCyJvqG48ZO3HaO0LXSN8F1EGdiU3N Cm0gqgXQXh2UOh7sEaH8uDCw/8vXSpQBKb9u5XTKGVBhM3Xsx5O37YCb5y4MNOrK+kNO ku2w== X-Gm-Message-State: AC+VfDy7Rc056vFlM70tDlYff928mAG2bDOFUXYSh8D3sb3X8G0CtUsr qYSSP24LNbAZW9+IiclOWXN+0hwdizdpX2tl+ug= X-Received: by 2002:a17:902:c102:b0:1ae:436c:b064 with SMTP id 2-20020a170902c10200b001ae436cb064mr2166470pli.8.1684266347859; Tue, 16 May 2023 12:45:47 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:47 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= , =?utf-8?q?Philipp?= =?utf-8?q?e_Mathieu-Daud=C3=A9?= Subject: [PULL 69/80] tcg/i386: Conditionalize tcg_out_extu_i32_i64 Date: Tue, 16 May 2023 12:41:34 -0700 Message-Id: <20230516194145.1749305-70-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62d; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Since TCG_TYPE_I32 values are kept zero-extended in registers, via omission of the REXW bit, we need not extend if the register matches. This is already relied upon by qemu_{ld,st}. Reviewed-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.c.inc | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index 647c31fa23..aed5bbd94c 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -1323,7 +1323,9 @@ static void tcg_out_exts_i32_i64(TCGContext *s, TCGReg dest, TCGReg src) static void tcg_out_extu_i32_i64(TCGContext *s, TCGReg dest, TCGReg src) { - tcg_out_ext32u(s, dest, src); + if (dest != src) { + tcg_out_ext32u(s, dest, src); + } } static void tcg_out_extrl_i64_i32(TCGContext *s, TCGReg dest, TCGReg src) From patchwork Tue May 16 19:41:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682413 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp20822wrt; Tue, 16 May 2023 12:58:37 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5nA/e3dSTekPqF3n4IUjNzQlFh1BSEwAI06aOGGqfWIuk/jftsuzFWz4z2XZfmo8ZHVOhV X-Received: by 2002:a1c:7212:0:b0:3f4:e4f5:1f63 with SMTP id n18-20020a1c7212000000b003f4e4f51f63mr11785818wmc.41.1684267117833; Tue, 16 May 2023 12:58:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267117; cv=none; d=google.com; s=arc-20160816; b=HfYQ4idPrRwH/xTNEuQ2G0r9dE0PgOuaJg3NmgulfdwJKj8AySMl/YV0m5OkDvbuIw J4amSWpwMQe6nCyDh8TPVSiIaGKe3CtfOjNoQOVNONkw8JRH2z/eWNiNomxg11St9257 ovL2EnDJQPGVQFnOuUbguCvXT/8m/0O1wz6Xj7Fc3IaPhe9XtPmSO33XK7pX2j1/cWda gyPEvArAzWvQaJEgb5SexKYTswlRAMHBnCASd/tCCIpZecohDXZdM02K36EXAqWyki7R h96ZDYCn7aLZVf62j8A9lRAm42fmHzz+xgj/NUF8jBLFKiGxkTh7oRGzWbk7SnxUeNk4 j8Pw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=xRTzpWzeSnL8ktDRpx0X4qNz7VhUVCXzx0ex45e8cKE=; b=XZXXulZZkngb0uWagSIUKqOidghbrEpUYy5xhQXWhVannVQodnign9c+LMl6J+kTJM +pAkLK7tC3bD62Wn6QLvtiQgCxldn8v1fPDNzvndo5T/TW5zXDn/Jfw/m9uRkpDWdHyk AxQpdCE8T1T4IABDzBiSK3Y6vQsrbXdeqUYv77T3F/j2izmdhRi9c2dQndQNPb78NwT7 OiXc7tifC6/Fv6GQ/AYzun8/wcMmjvFILrMplZa7WNnsxL/BojKRy1xegL9WEKT3pLHk RBB4uuwyZEE/REHZwHGO23vkDPuxOLeqD9lwrzqRKBJ9P3myLA4bwTmIF8CTMbsXst3e t2Yg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=SDkq4dUW; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x6-20020a05600c420600b003f427cd41e3si112740wmh.74.2023.05.16.12.58.37 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:58:37 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=SDkq4dUW; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0ey-0003bh-Ms; Tue, 16 May 2023 15:48:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cM-0000Rn-0z for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:54 -0400 Received: from mail-pj1-x102e.google.com ([2607:f8b0:4864:20::102e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cI-0003my-4R for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:53 -0400 Received: by mail-pj1-x102e.google.com with SMTP id 98e67ed59e1d1-24e25e2808fso49738a91.0 for ; Tue, 16 May 2023 12:45:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266348; x=1686858348; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xRTzpWzeSnL8ktDRpx0X4qNz7VhUVCXzx0ex45e8cKE=; b=SDkq4dUW7vtH3/3iMpVZktEYyMPDRV1/bk+jr9cP+xiKLEpiUFk8H4wtTl3BOBWCfc ErOZ8cxJ0+Xz4ww+MOTtR21GxEBN6i7kgRIa+Y5BiKO9AHvPtFcp8zXX7oke8kYq+ijw +k/t2z7IpTaC6EjieoSk/rRImj1kWAbtUha1CkvltldX04WmkDa1lcFSpf9GZZriP9Vd rkE6R/h55MPoCP91yJOVv6nzF1a2svn2SYWVlcriMCsb2oFRMxlKybqEJPhvcjtJKC38 ZHZHj7HlbdNsyMZmvMucZSmgSU1OkrOzltGGeMBBVdVacyVtcMUycaWuIG/mriXg6XgM 7/QA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266348; x=1686858348; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xRTzpWzeSnL8ktDRpx0X4qNz7VhUVCXzx0ex45e8cKE=; b=Id1Ics9WUmOzWIr5bIZyxtkvtJVcyIQ1x+V/DzACMkt5pFSp9nkK/LbgXWa9b1OEtP q+ZF9Hs09Iy3Bw+JNeTM6gKu8E8c+X1nhEq1kbM6eQTRyTYN3YtZLjN+VyeyO0XrBgDI FZl/ulsUNQ670JeFAu3gSWhOCstZp2KbkybGdjVbNs5lulA2e26hVzAWnjNKmXGdrI8Q DdlFITeinyxNGaHoSBtnM2P2Zr+BgKZhz7E7UZR092MyAgjPyb72lQkE9g+7FcJH3tmV s0uDq0jCwpSInMnRCYILp9+UlvozofiXVtOyk3NkzzL9CEIh90W0FaBc7VzkmD8IktIR rPyA== X-Gm-Message-State: AC+VfDz0BV7rXcSpwt77IdCAX7W5PGxYYeAynXgjlUF+z20sFXCNI1NJ y4LOZ5CZg6AqLBjpYnXNkXFduxpspK+W9tQnSgQ= X-Received: by 2002:a17:90a:9502:b0:250:7ece:4461 with SMTP id t2-20020a17090a950200b002507ece4461mr30231397pjo.34.1684266348632; Tue, 16 May 2023 12:45:48 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:48 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 70/80] tcg/i386: Adjust type of tlb_mask Date: Tue, 16 May 2023 12:41:35 -0700 Message-Id: <20230516194145.1749305-71-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102e; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Because of its use on tgen_arithi, this value must be a signed 32-bit quantity, as that is what may be encoded in the insn. The truncation of the value to unsigned for 32-bit guests is done via the REX bit via 'trexw'. Removes the only uses of target_ulong from this tcg backend. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.c.inc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index aed5bbd94c..653e3e10a8 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -1966,7 +1966,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, int trexw = 0, hrexw = 0, tlbrexw = 0; unsigned mem_index = get_mmuidx(oi); unsigned s_mask = (1 << s_bits) - 1; - target_ulong tlb_mask; + int tlb_mask; ldst = new_ldst_label(s); ldst->is_ld = is_ld; @@ -2011,7 +2011,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, tcg_out_modrm_offset(s, OPC_LEA + trexw, TCG_REG_L1, addrlo, s_mask - a_mask); } - tlb_mask = (target_ulong)TARGET_PAGE_MASK | a_mask; + tlb_mask = TARGET_PAGE_MASK | a_mask; tgen_arithi(s, ARITH_AND + trexw, TCG_REG_L1, tlb_mask, 0); /* cmp 0(TCG_REG_L0), TCG_REG_L1 */ From patchwork Tue May 16 19:41:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682430 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp21483wrt; Tue, 16 May 2023 13:00:33 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7IEuU75D4KUSyGQLhv0TkMEVkjesDvlnDqncKzcOSi+iYy/gx3j0n8eUZKGsG9aiB+ebvI X-Received: by 2002:a5d:674f:0:b0:309:3698:8011 with SMTP id l15-20020a5d674f000000b0030936988011mr2065806wrw.57.1684267233241; Tue, 16 May 2023 13:00:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267233; cv=none; d=google.com; s=arc-20160816; b=tj2Oy4vkT2SkBwxA1TBIreAYhXILU6ZIrwGA7xQyQduEE3ZMMVtznjJlXPsKSLMJtM 9e3rt75+/4LnsVq/KTUxFKQh5GzP6U+4UlNHv1IGMxA/pMyms62WVgQC7ylIYnhV1Qyn 4LatewgjMdAahFLppG/r0f4zKHPcG29BrOeqvOeKWqLyvIj2UFBxZza+lGqEAjBZUItd wUvaVIyAT9hWFzYlo8g2gU/fmwmFyuBY/UTd1wD2wgbUFoQD4VSSS3K15mQ7W1GwM27l n5fbXqQmtAG2oVE4+afZRNEWCTAgBcUiW84O2BlMGTXIfKa7YiLk2rLME+7DE/lyR0N0 Xsqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=8gd23/Zs47HSFuLNvqC9U/DMCSd+WEyq0LmTxbksagg=; b=gG/uW3tCDOMZCKPvhyRW6RSzWY+okiz+/n7kbI+UyJ3M6cfVACDtUeUkRz4kgcULvf 5627fh6u/JZ/xo2Gwp08ym25eCkXYmtJosov3RXsrAkFoTyaxBpkK12o4fpibyOKGgS9 j95OusoGnd9j7dU3Z07M3YU1iLOqItd/8AYmhQO9exu5x7FZ5WwYWXRt9bwvZmAOF7IC ohPpap3e4rgrDN+mjYe7oW+VUikCYg125/ztgPzmmR7549EB3hRqWM4scSrkIVcMYjRq 6JUC96hAgHKahOlmkqkomf9sSjmgzK7xIyuzLi51/NiQTzwdHwQVggh0rgBwnpWAKjJK vkKg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=F2Wlvsb1; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y1-20020adffa41000000b00306443884ccsi259504wrr.438.2023.05.16.13.00.33 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 13:00:33 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=F2Wlvsb1; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0eP-0002nh-97; Tue, 16 May 2023 15:48:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cM-0000SW-9V for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:54 -0400 Received: from mail-pg1-x529.google.com ([2607:f8b0:4864:20::529]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cJ-0003n5-GM for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:53 -0400 Received: by mail-pg1-x529.google.com with SMTP id 41be03b00d2f7-53033a0b473so7584050a12.0 for ; Tue, 16 May 2023 12:45:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266349; x=1686858349; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8gd23/Zs47HSFuLNvqC9U/DMCSd+WEyq0LmTxbksagg=; b=F2Wlvsb1w8oRpkyCeogTBk4vdxj7xiWr/UpmWDUOLe5PIKSF/VWeeQHt7bmafQgbeC zJtRCcX2abZnSzVYYOT16QP6pi4LMbRl9l87OEa+4XB9rAnpkkZlsYWnKArw1BcaL9dz J3tVdKBCHdr3Axhin4D8uRomoHIXNIRVL/PuhHVDRVNkGMXVynve5oj8oNJj8BA5V0nD 0mB33nWRpTE/DmCeKZtbUZ/MZDmslH5lttat5b0k/ETwhDr0xWiFyM7v9JawrYl8HFmt P+nDtqAaPkQeTd5XYl7kxngetUe2a4xBuZV6YwRhmYJyWhhV/4feJ2QBF9RcGQL4hCvv SMGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266349; x=1686858349; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8gd23/Zs47HSFuLNvqC9U/DMCSd+WEyq0LmTxbksagg=; b=Cxagq7eh938SdvAwgUXBVj9j8m/KIn4SuoWfBKZ45lvsAjWvae16noXOXrissONZge IxuQc4tHuXxZ969XWO21E6QvND83i3naacSsCRoLIQ5xF7DpLwerRhDpqclfFrXOopFC OVL8WQZAdTlNjnJHFhfZgszQulNVlh5BUIYB5Sjcy9voRGavl+estiYQnkupUIY34P7f VFU7uvQ24dVCQtEgUdaiBtWf+sxbkqfKBM/+EWMLgxZT9FkAMdeHeh8pqPAwd41I9wpa SDgbzPS2motvj5jjp6kfpI4FQlEsmG0YmD8N5wrpYDv2Pk7+fMFWw3R/Ic5+OhyJ3WMU l3Gg== X-Gm-Message-State: AC+VfDygBSQp83BHmwn/J/x8WAvkJnKjotr7oh0JMU/s7gCFA76kK1Au /JqcNwmfsgV2kT06+XLAh7X8zU5r0k9UxN/sxyg= X-Received: by 2002:a17:90a:8d16:b0:250:faff:e201 with SMTP id c22-20020a17090a8d1600b00250faffe201mr22140072pjo.36.1684266349357; Tue, 16 May 2023 12:45:49 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:49 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 71/80] tcg/i386: Remove TARGET_LONG_BITS, TCG_TYPE_TL Date: Tue, 16 May 2023 12:41:36 -0700 Message-Id: <20230516194145.1749305-72-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::529; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x529.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org All uses can be infered from the INDEX_op_qemu_*_a{32,64}_* opcode being used. Add a field into TCGLabelQemuLdst to record the usage. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.c.inc | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index 653e3e10a8..e173853dc4 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -1975,10 +1975,8 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, ldst->addrhi_reg = addrhi; if (TCG_TARGET_REG_BITS == 64) { - if (TARGET_LONG_BITS == 64) { - ttype = TCG_TYPE_I64; - trexw = P_REXW; - } + ttype = s->addr_type; + trexw = (ttype == TCG_TYPE_I32 ? 0 : P_REXW); if (TCG_TYPE_PTR == TCG_TYPE_I64) { hrexw = P_REXW; if (TARGET_PAGE_BITS + CPU_TLB_DYN_MAX_BITS > 32) { @@ -2023,7 +2021,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, ldst->label_ptr[0] = s->code_ptr; s->code_ptr += 4; - if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { + if (TCG_TARGET_REG_BITS == 32 && s->addr_type == TCG_TYPE_I64) { /* cmp 4(TCG_REG_L0), addrhi */ tcg_out_modrm_offset(s, OPC_CMP_GvEv, addrhi, TCG_REG_L0, cmp_ofs + 4); From patchwork Tue May 16 19:41:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682392 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp19059wrt; Tue, 16 May 2023 12:52:18 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6lzA6JSxzuPyXMtrPeTp7Sk2zTbYMvbEYR2QtUbufkVMkaeX/3nVT2emO8sbpDo/Zx1grU X-Received: by 2002:ac8:598f:0:b0:3ef:31c4:c8da with SMTP id e15-20020ac8598f000000b003ef31c4c8damr63941860qte.50.1684266737771; Tue, 16 May 2023 12:52:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266737; cv=none; d=google.com; s=arc-20160816; b=HerHHlmsG1uIn66bNm7qo+094vTNx7voW/LK3oh6BIbBvrbT/Rc7WW/RBGCazzcT4P V8Dq04H4bsCKCuHXkt82tc/d21J8w0G0oYTg/OBUTAjBRXZpHDzi6Fz3OhEYA3LfNxUj arbgRr8S0NFpwlOpG3GcTXgvjaCJbpAyvP0O2WmG60Lfa8TmuxCvY43mLFSwaQSqSXp9 nqjpRbTVvVbKSGH+Tpfc30oGBuX0db+aTzrXs/lzXs3f7Jg+Nn0IO/nRXJ2U92gtc8Sh pHuzL1yRH978aecCVRlHDDhdhLkiNRtcQNNQBarjoXiFKpbA71lcRYn9g5a2dJ7ptHWm KBfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=x0m1o00zlToFNXbMeHftTRJjM9JUo6XAGwMqXPxpsNk=; b=ZzVtMjlJhJ/CX7VvQJgTPcuwrrttJDAPdQFCl29SGuZnqaXRhufE4Mpt3CaLXtXk+E +ac8RSYHAjTqbzqh9uR9276Hfq6cXuzCcnfCyIOnswoNjNx1+05HLhrHKIxUxuKFhc0P 8P+iWxJkiunkwCwBhZ2O3fCYlQPFoBymRVAuOrBJsYURbQgh7lHI53jORf+X1a2sICx3 FtDxtMe4eI6co0xoRXG/hd0Ty8l1JlOD4+ZVvztlzzv+0ANOtE/Jid9BMjGv+uFCDBwA DwN10wVYA2T8u0x+hyovcINoq6Pm7jY3oXFRofoFCmjz39B273DcToVdvuXqu0W5VdwX QDlA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xh8JkQ8s; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v8-20020ac85788000000b003b644e69427si9809240qta.504.2023.05.16.12.52.17 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:52:17 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xh8JkQ8s; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0el-00032r-5W; Tue, 16 May 2023 15:48:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cN-0000Vz-Oc for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:56 -0400 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cK-0003iw-6z for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:55 -0400 Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-1ae4be0b1f3so545275ad.0 for ; Tue, 16 May 2023 12:45:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266350; x=1686858350; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=x0m1o00zlToFNXbMeHftTRJjM9JUo6XAGwMqXPxpsNk=; b=xh8JkQ8sexYe60SmPHI76wR/QADRDBQ3fs8bCoV4k/6wa6Za0Vy/RaSLjbI16MyRqQ 4FmF89yy1ad2yD/G9B+iOkQeLr65AwOYDXVwfKisxX3JOvGrwUWpOKHRw/9NJc00lf7T dbIZzYSiXgbxRES4ahV7bpHVgO/uV3osIpmQ3ma/G2q1IPDkP1sY2pW60dmD+Hfo2itO u+oKJRsYpGyTo7lZ6e/zNl71mM2DZYKyOS/lNP7V198S7NWTj9rgN4MRYLsxuk9BxZGj AocXAhVcl71uN3n9n5RdxbSW0zd0jB1gnnIczOHCpGu4yVJHuj/0GaE45E/ct+WMmAC+ hWGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266350; x=1686858350; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=x0m1o00zlToFNXbMeHftTRJjM9JUo6XAGwMqXPxpsNk=; b=cmTazjfkDVBAuCmAyGcLWssFuFeT9xMRSftXaMJeSlzZ5z7l5VsqlIIdexlHvOdLar uTvz/w7ipEPFnJ2o5+8RcyGzlJdK/XUwbQvYivHno7+ZOWEqofXGDq0YArV72wAgrYfT CQR5jNsbIGK7HZYGQkPg7zMidcTW3eJ39iKdMEcO9F6qYXKuAyG4D+ddvrzYs4/Ewghs W1HxqV5dIImDtq/63nfMiT7RkCv61EWQDZLM12/2VFC6cbqoMcq+EWZ2/n/SVt7bDi2M tbnyNH/D9Ctqtld8tq2KsrvAGRph9nvg1rWXAS9w3ASXcWhrIupYpIWvQWNakN7nFcYg ZAsQ== X-Gm-Message-State: AC+VfDzw6N6CG3e0XgiYsny5FBGe6YvD4DGaRzcgD+AYb6PMwsuBkb8y wGFxwonNLCn35rwVAoJJ4J2tTA8LAyj+bbncMtY= X-Received: by 2002:a17:90a:7388:b0:253:3d7c:949a with SMTP id j8-20020a17090a738800b002533d7c949amr520361pjg.33.1684266350191; Tue, 16 May 2023 12:45:50 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:49 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 72/80] tcg/arm: Remove TARGET_LONG_BITS Date: Tue, 16 May 2023 12:41:37 -0700 Message-Id: <20230516194145.1749305-73-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62d; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org All uses can be infered from the INDEX_op_qemu_*_a{32,64}_* opcode being used. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- tcg/arm/tcg-target.c.inc | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/tcg/arm/tcg-target.c.inc b/tcg/arm/tcg-target.c.inc index 47f3ff18fa..3c38e868e2 100644 --- a/tcg/arm/tcg-target.c.inc +++ b/tcg/arm/tcg-target.c.inc @@ -1431,18 +1431,18 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, * Load the tlb comparator into R2/R3 and the fast path addend into R1. */ if (cmp_off == 0) { - if (TARGET_LONG_BITS == 64) { - tcg_out_ldrd_rwb(s, COND_AL, TCG_REG_R2, TCG_REG_R1, TCG_REG_R0); - } else { + if (s->addr_type == TCG_TYPE_I32) { tcg_out_ld32_rwb(s, COND_AL, TCG_REG_R2, TCG_REG_R1, TCG_REG_R0); + } else { + tcg_out_ldrd_rwb(s, COND_AL, TCG_REG_R2, TCG_REG_R1, TCG_REG_R0); } } else { tcg_out_dat_reg(s, COND_AL, ARITH_ADD, TCG_REG_R1, TCG_REG_R1, TCG_REG_R0, 0); - if (TARGET_LONG_BITS == 64) { - tcg_out_ldrd_8(s, COND_AL, TCG_REG_R2, TCG_REG_R1, cmp_off); - } else { + if (s->addr_type == TCG_TYPE_I32) { tcg_out_ld32_12(s, COND_AL, TCG_REG_R2, TCG_REG_R1, cmp_off); + } else { + tcg_out_ldrd_8(s, COND_AL, TCG_REG_R2, TCG_REG_R1, cmp_off); } } @@ -1485,7 +1485,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, SHIFT_IMM_LSL(TARGET_PAGE_BITS)); } - if (TARGET_LONG_BITS == 64) { + if (s->addr_type != TCG_TYPE_I32) { tcg_out_dat_reg(s, COND_EQ, ARITH_CMP, 0, TCG_REG_R3, addrhi, 0); } #else From patchwork Tue May 16 19:41:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682419 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp21050wrt; Tue, 16 May 2023 12:59:27 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5a6MdEX5HcdPslZn6WtVbIdRGOuIwD6DGYn48KqZ7s+jeFCUq9kRxR3f6rV6pGd9fV1+yO X-Received: by 2002:a7b:c40f:0:b0:3f4:2300:e015 with SMTP id k15-20020a7bc40f000000b003f42300e015mr23474842wmi.5.1684267167401; Tue, 16 May 2023 12:59:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267167; cv=none; d=google.com; s=arc-20160816; b=VkYWybuVLis53Qxc9kNJSRmUDTpfuGvyPuBfHIfi4sKqujJBPTCaCUiQlyW3n3n4iG XPd/YPXjbp6wHU4YgGh4aqjGI3C83q+678fkPOKz2A1pl4u8ZcQwlRA8UCpa5lgmIMPs qqRP20gESqayVyyU6BXzDLw+xOWWsbCaeXNTHbzQMIf5K45tqqtYu3I6xMwjuKLIWiL+ 7Y7OnuiKttBNAh6mKPAp0Kru66rCcYaTHxOkHKce9aGMHtb7qulpwPNtdnmW9Ge/L8AM 3M8wFAN9x9yk3IIgWUDJKOoeEBWJ582/lMEVWwNgcMP9vSe6Bqp6sMD6EH7lL7x1V/tw q5MA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=v6AghR2GxNGQxhvr7PKKi9iS59oHnnhrCkJuSzSdVhA=; b=w7yJpoNvHIf50aaPwj8kTGqVmMkglI0aP3CLYl1xjVbDwTGX2KdupiWCVsXSZ3siUu UKb84nPFhzL1KVklDRY305m5sIMp08TIqeo22jvrYyNWAKYlSlU2ygxdjwlyeCrrswWF uXJvg5e4dKk8K6v/g2e6Y/6LGCl4KnzukXbig7Eu1spbIhw2NkzWmEIMzwYy67XV5taa Uao0wtU6evIwpCWcd0uhEL4EjdPS5jX1Sh1O3I9gBBE0KzD0CUH6tBItKaY8ZdgXciYx r0vxw9aNPaIfoOtafVMmBsSV+8ZOJFuLwEkedYErlcx71Pn07ni4eN/lbvoy7KRmddFH Up8g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cj7LyJZd; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r15-20020a05600c458f00b003f424323c3fsi120967wmo.154.2023.05.16.12.59.27 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:59:27 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cj7LyJZd; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0f0-0003ot-V3; Tue, 16 May 2023 15:48:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cO-0000WH-1s for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:56 -0400 Received: from mail-pf1-x430.google.com ([2607:f8b0:4864:20::430]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cK-0003o1-CC for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:55 -0400 Received: by mail-pf1-x430.google.com with SMTP id d2e1a72fcca58-64a9335a8e7so8239168b3a.0 for ; Tue, 16 May 2023 12:45:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266351; x=1686858351; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=v6AghR2GxNGQxhvr7PKKi9iS59oHnnhrCkJuSzSdVhA=; b=cj7LyJZde5gUxByrIlnDcfzwYmHhUPBa3vDl3PjvcrQO08IpXYA17ZUZ3TO+jh1pFh u94bIwmgTVK0hFAIoWZY5wLgNo9dSDMKmZU2MKqjX37lQqj8MXRw8fmj9w7MObttycNu OUPrI54RdOoRAOZnDrvdPQ3YFUevZj/t6oRvBD+/erGwH8lcikWE+oyC+zXkGc+Ww4ES ZVbkf2rx2dqFkV1Wx5Kwx2/npIfckddNw8K6R5iRuBuDXVG2J0WzN66x/ZnIHaK1lqwW pcgVESIv6anT+R8jghkKie5af78j0KyPzxJtFkvDr1LD0ILHfEUsdGykF3Xy8BIfM2aq BgSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266351; x=1686858351; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=v6AghR2GxNGQxhvr7PKKi9iS59oHnnhrCkJuSzSdVhA=; b=jEDBlzW6oRb4sD7pkHkQve+nf6TjWsdrSJrsLwvnUqaZHx/aYRJi6Gdd1R/qMWiWUG nhcxPhrNvoz75qZqDoF/VRpKIpnPlDzDugLv7ewyRCxuku+OyNLlV1x//o3+V0IMsKK8 3W6thT1So5FOqMWguBL7cPr9Y/ziv7g4Cjjx4bcIwf2X5i7a8SjKSb+Egtx79MkY/ICG kKnNkOQc2h2XecPS6igKXzHHxHq81iVu6UYjmLE1m1O+jUutjrMzIvcibxix5TmqkT68 QM8IvEXfgXNJ8EZy14xhHWxGNM+uT6z/nKBxWl6oMM1tDFWbXU9aqDbQEIBjbaO1kCtJ 0RLw== X-Gm-Message-State: AC+VfDx0jDIFz2L6hx6pfNPXJBqHiD7dkOI1Ts3KUBzxVE1l0a7m3mQL MAIYnbhk46dyj1+uOKD5xr1xNl2xk/8wkNEEU00= X-Received: by 2002:a17:90a:d241:b0:252:b345:7953 with SMTP id o1-20020a17090ad24100b00252b3457953mr16550130pjw.24.1684266350973; Tue, 16 May 2023 12:45:50 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:50 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 73/80] tcg/aarch64: Remove USE_GUEST_BASE Date: Tue, 16 May 2023 12:41:38 -0700 Message-Id: <20230516194145.1749305-74-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::430; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x430.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Eliminate the test vs TARGET_LONG_BITS by considering this predicate to be always true, and simplify accordingly. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.c.inc | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index cc0f55623b..8d78838796 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -85,11 +85,6 @@ bool have_lse2; #define TCG_VEC_TMP0 TCG_REG_V31 #ifndef CONFIG_SOFTMMU -/* Note that XZR cannot be encoded in the address base register slot, - as that actaully encodes SP. So if we need to zero-extend the guest - address, via the address index register slot, we need to load even - a zero guest base into a register. */ -#define USE_GUEST_BASE (guest_base != 0 || TARGET_LONG_BITS == 32) #define TCG_REG_GUEST_BASE TCG_REG_X28 #endif @@ -1760,7 +1755,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, tcg_out_insn(s, 3202, B_C, TCG_COND_NE, 0); } - if (USE_GUEST_BASE) { + if (guest_base || addr_type == TCG_TYPE_I32) { h->base = TCG_REG_GUEST_BASE; h->index = addr_reg; h->index_ext = addr_type; @@ -3172,10 +3167,14 @@ static void tcg_target_qemu_prologue(TCGContext *s) CPU_TEMP_BUF_NLONGS * sizeof(long)); #if !defined(CONFIG_SOFTMMU) - if (USE_GUEST_BASE) { - tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_GUEST_BASE, guest_base); - tcg_regset_set_reg(s->reserved_regs, TCG_REG_GUEST_BASE); - } + /* + * Note that XZR cannot be encoded in the address base register slot, + * as that actaully encodes SP. Depending on the guest, we may need + * to zero-extend the guest address via the address index register slot, + * therefore we need to load even a zero guest base into a register. + */ + tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_GUEST_BASE, guest_base); + tcg_regset_set_reg(s->reserved_regs, TCG_REG_GUEST_BASE); #endif tcg_out_mov(s, TCG_TYPE_PTR, TCG_AREG0, tcg_target_call_iarg_regs[0]); From patchwork Tue May 16 19:41:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682400 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp19803wrt; Tue, 16 May 2023 12:54:50 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6d/PGvB4rPH5gHu3czqK4x8mtfh/2lgLbeU1yW6amqoqPXiLHP7byrSB5WJQIDqoDqmDn4 X-Received: by 2002:ac8:7d46:0:b0:3f4:e020:63dd with SMTP id h6-20020ac87d46000000b003f4e02063ddmr36701939qtb.7.1684266890661; Tue, 16 May 2023 12:54:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266890; cv=none; d=google.com; s=arc-20160816; b=bBzJoVOSERVeNDI8uy3GyojaSApD58AlAwJt/QQ1aoJOG4f+kVnD9uytgBiw0HIm2k sAdhGGrb94h1xqaLR2wRDDSYDvOw+OTCBlo9ZN+17cXXEMr2gHlTlrlsbiYFkPSr+m+5 UmJ1dCHai13WacZqtYjL/gVRgztqmGGpZUjlJR+c86BaM+a8vVppHjs619ZbjPbonTla 5EqfbPNV6VcION1ZoT0Cbi7lfBMEJIZ7rh6lq6kQP+dcScrNja2WR56OGLIFiADksAZe oi7TGK/ew1xTg0edcSwsppLLFL4JV7NUsxsqD4L+ypQrEcJ52y1Oh851bbYCPHXcJPkf 0FsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=x+EtsvBcKcdpaXvTeCONOLhtSa///+bxez6Y/ftYRpI=; b=Hjb6QSRyhtuPuUyZ3IUpG0VdlVTLY+GzSeAY+P4A6yVwRUr0DOgGM2fuFpDGqR/m8E KcEClPq8LGZn8os/dav7OteRZBswanGc9OI9cWwU/KvRJ6ZTX7VojhhvVsIKC4j6cGja IzGHH40tLxKFcY/gCzasIppeqFrAlP7oJF1n9qZTnRWAFoKiZiBHgQofkCz34XN6VvA2 KLd2zIOc8Jjm0+EK8PRZsLg8LJShflKdAn8qo+tOTdO+wMe7IWVSF82Kcg6b0v5SkvBk 7+rVTy3Cf+mROi+1+Gjp9l9czUbdqinmvcB+yIixBJR/NJFOFW7SA/mPhwely8wv9AC5 tgsA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=FJWpNgHk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f2-20020a05622a104200b003f1f18be2dbsi11738355qte.415.2023.05.16.12.54.50 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:54:50 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=FJWpNgHk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0cp-0001TE-Ro; Tue, 16 May 2023 15:46:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cS-0000eb-An for qemu-devel@nongnu.org; Tue, 16 May 2023 15:46:00 -0400 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cL-0003oD-30 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:59 -0400 Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-1ae408f4d1aso537285ad.0 for ; Tue, 16 May 2023 12:45:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266352; x=1686858352; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=x+EtsvBcKcdpaXvTeCONOLhtSa///+bxez6Y/ftYRpI=; b=FJWpNgHkoWISN7pRa9ZCsVVMwhUx2VBqy6yoyr69K+WPWPzlHCK24wIBo61rrEGw35 AfSXktII/mP9yRqZIOyggUx+6hZM98hKW1JH0t6FGUAmLm5ZcWIDCzMAC1ptnuN0vSTO 8c9vvtmypCr6pV8DFY5trbbdbaZBscEAi37iDTORtHUgJXEuhYBQIyLGtsur9+1a7QDQ liIgvNQX8uJF9cRyCT89G/E8ur1McU6LW7P8h8boX42s7mFyVSkKYgIag1cnr9bJrjNQ soph7ecTIXtqLrNgON78F+uGmX5EGJBBxBukDyefNLgreK1WHejqeqvhhcXR5tofyXWR UwuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266352; x=1686858352; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=x+EtsvBcKcdpaXvTeCONOLhtSa///+bxez6Y/ftYRpI=; b=JXEBe/0TIt0UvpZoR9tV9yi/zanIwuyc0DFKF9RE6B7MZKn10cRx8GRAtrPsJXWoB2 VxVf4QspZZRqspTMfF2at2+E4a/YIxrBFp0V+5CW63Z9gfQqNgAcy+bxKLH8HU3pcM0Y DjUKuTZ/ka+3PWAMa6vnqjyix/VJzQDIFvW+e18+wns3BF8h5GsmIIAe7pdbQNyC/Khi 1u+j+jX+6WfGurbQGekewZ1w3uRqU30ee2FzNiJR4/0M91Qdx40lISbPIhEFb7ErzIki 2boUS+VLSDZt7cUptd7oB41RmX6LI0w7xtaoNyUdf2G5cjta25jlMWuE8O/hnLyyLnWD Qp8w== X-Gm-Message-State: AC+VfDzDjXp572vHw6oQPtng7rXDLDURGfDHjjyFz3XJ2AnZL8URLXit PgOiGUeryUhx/xNDNgxjVAMmFayXjfmfK6aZEK0= X-Received: by 2002:a17:902:ba86:b0:1a6:bd5c:649d with SMTP id k6-20020a170902ba8600b001a6bd5c649dmr35650024pls.56.1684266351738; Tue, 16 May 2023 12:45:51 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:51 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 74/80] tcg/aarch64: Remove TARGET_LONG_BITS, TCG_TYPE_TL Date: Tue, 16 May 2023 12:41:39 -0700 Message-Id: <20230516194145.1749305-75-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::634; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x634.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org All uses replaced with TCGContext.addr_type. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.c.inc | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index 8d78838796..41838f8170 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -1661,7 +1661,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, TCGReg addr_reg, MemOpIdx oi, bool is_ld) { - TCGType addr_type = TARGET_LONG_BITS == 64 ? TCG_TYPE_I64 : TCG_TYPE_I32; + TCGType addr_type = s->addr_type; TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); MemOp s_bits = opc & MO_SIZE; @@ -1705,7 +1705,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, tcg_out_insn(s, 3502, ADD, 1, TCG_REG_X1, TCG_REG_X1, TCG_REG_X0); /* Load the tlb comparator into X0, and the fast path addend into X1. */ - tcg_out_ld(s, TCG_TYPE_TL, TCG_REG_X0, TCG_REG_X1, + tcg_out_ld(s, addr_type, TCG_REG_X0, TCG_REG_X1, is_ld ? offsetof(CPUTLBEntry, addr_read) : offsetof(CPUTLBEntry, addr_write)); tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_X1, TCG_REG_X1, @@ -1719,18 +1719,17 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, if (a_mask >= s_mask) { x3 = addr_reg; } else { - tcg_out_insn(s, 3401, ADDI, TARGET_LONG_BITS == 64, + tcg_out_insn(s, 3401, ADDI, addr_type, TCG_REG_X3, addr_reg, s_mask - a_mask); x3 = TCG_REG_X3; } compare_mask = (uint64_t)TARGET_PAGE_MASK | a_mask; /* Store the page mask part of the address into X3. */ - tcg_out_logicali(s, I3404_ANDI, TARGET_LONG_BITS == 64, - TCG_REG_X3, x3, compare_mask); + tcg_out_logicali(s, I3404_ANDI, addr_type, TCG_REG_X3, x3, compare_mask); /* Perform the address comparison. */ - tcg_out_cmp(s, TARGET_LONG_BITS == 64, TCG_REG_X0, TCG_REG_X3, 0); + tcg_out_cmp(s, addr_type, TCG_REG_X0, TCG_REG_X3, 0); /* If not equal, we jump to the slow path. */ ldst->label_ptr[0] = s->code_ptr; From patchwork Tue May 16 19:41:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682390 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp18964wrt; Tue, 16 May 2023 12:51:58 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6ZO98/h7Udy5YcSdljIfm9yStgZtA6Ta+57X8uJThCaQJUNcV1DEwQijPbxQmGfaO3YMYd X-Received: by 2002:a05:6214:c48:b0:623:47a9:941d with SMTP id r8-20020a0562140c4800b0062347a9941dmr16620861qvj.37.1684266718172; Tue, 16 May 2023 12:51:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266718; cv=none; d=google.com; s=arc-20160816; b=B43EL/YwcI3ntGRugUHwZ19vS6zSsWKaWS9vItyYgITWEssUiGOxyV9eXGWR2KtpFn nKdbBywBsLtNxoeMNyNIaGNO2ixKRmepn2IVKSX+cLKJ3zX0pBUamWpTDbf1lD5/+l0y 1hVA3KQH6V3mFLCJxPest3dhOX+8UbeViBMd5naYlLlIltH21LCCv+71+7icW3qTBTnk ridFVgCkV49dp30cxrf9+PwIxHS4y2r//1xVKyTCpoWSMpDrdNmctbXfBMmWdisaO01Z PZgnccqpbR++WMXnCfxqYnZfcFqbf7VsMGuoQLfxQgqCGYPr1KMRbVdqn9DubiQk6oFz IKLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=82nL5lFZVOS2kXTw0gpjyfDlKAnjIx0Wv02gro+5+RE=; b=R9B/uNASnSk6gyzuHYYeCd5LuPtdyHsrZ5NK2duyihLT75jIBfN+DQG91D3D1eJ3TO Fb0a0ZUqV7idMIIuv7YdpOTjac308/RyThSLkeiaYJLNIJ6MwZCsVAc95NC4bTTwg7yO t/kDkws5XM8PqFdxC8d2GcH7SNC6yQbJtZaJp6NyhR9PUF+nCwPJvGUJx/fEy+9+3l1D /tJS/yItYJuGVFbR9K/yo6XlB2M+EdYLghnUKe6wL1PzBon64pd9be6bvVMfRC/viBLq 7f3TsSj34lsEYtR5bw1/uTmSB2ynlh6mMbncHbCNsGEXFvnpMWzBTkq/8rWozN0GV66W yVpg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=eeqtOo+8; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id g15-20020a0562140acf00b005ef6e5c03a0si12367037qvi.224.2023.05.16.12.51.57 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:51:58 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=eeqtOo+8; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0et-00036S-KV; Tue, 16 May 2023 15:48:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cW-0000ky-A7 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:46:04 -0400 Received: from mail-pj1-x1032.google.com ([2607:f8b0:4864:20::1032]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cL-0003oX-UN for qemu-devel@nongnu.org; Tue, 16 May 2023 15:46:03 -0400 Received: by mail-pj1-x1032.google.com with SMTP id 98e67ed59e1d1-24de9c66559so55532a91.0 for ; Tue, 16 May 2023 12:45:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266352; x=1686858352; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=82nL5lFZVOS2kXTw0gpjyfDlKAnjIx0Wv02gro+5+RE=; b=eeqtOo+82OyjSkBdnkkqRDvd383Csib7Z6ZCA8sPYYgpeOyhf2cazljsP5uwd6cd0h RlruN8EXodMfvM8eytDVpx4jb1U4RRQ928DJgtnAW46zeNE6DAY4KaTLuWR9Ft5OlZMw 8GKaOoooU42E10BeXM6ogTVSL3tvlNPZ94g7sf1Uyyc48CxpgHL8oZzoioT7cpWQIUew 44F65ErzK71ZEPetQYIMCTWc1wUjpKfCGCPO6TjOQZAP+/LYrkDOg2loeaa/1aj2UFlh LiphLzIopc5+cOzF+/Yy+25nM46SU107Md2v8jtdukKX4SSwq6GX1fnyRzZXZXIeRsGE imwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266352; x=1686858352; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=82nL5lFZVOS2kXTw0gpjyfDlKAnjIx0Wv02gro+5+RE=; b=lfiV9a3rN0fqCbjU1Y5jyPdADT7b0pcYzNsiEHlJIufNzN0F+uMmWa/Atxq40fTa18 y1SKUiqv/UG+LEzO4mgfxxm+BHnTjE8VgKtFL0e78vN22hWhnEpWQVsO5WFfDDNcWnUJ wWvDYhia+z85N+Pi16a5PAp/m9tqf1OfYyIaZX5bySbtTQnb4yTmpdIBswnzptAfzpx4 AD/I1D/vnwfCgam8mVX4VhDIfVsgJ6FLu2/xIWehlXREoM5abRQZjt9cBny1PedRfaXS ekN8ZBjFUiLU9DafkqN2VowqcE3E5E0FqnWiJNY7lN4scklA0g8byl0iBnx/W7l/JS4Z 1G1A== X-Gm-Message-State: AC+VfDzJ57Et+SGLlGBmtAPnSbhr+Q+Zmyy9DvgxuITGeJtrUJbGjFC0 PEbjSxZ7HzFXDY2+i2bQBmlUhwD0l49GUkIQn2k= X-Received: by 2002:a17:90b:17d0:b0:23d:15d8:1bc3 with SMTP id me16-20020a17090b17d000b0023d15d81bc3mr37770884pjb.39.1684266352470; Tue, 16 May 2023 12:45:52 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:52 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 75/80] tcg/loongarch64: Remove TARGET_LONG_BITS, TCG_TYPE_TL Date: Tue, 16 May 2023 12:41:40 -0700 Message-Id: <20230516194145.1749305-76-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1032; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1032.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org All uses replaced with TCGContext.addr_type. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- tcg/loongarch64/tcg-target.c.inc | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/tcg/loongarch64/tcg-target.c.inc b/tcg/loongarch64/tcg-target.c.inc index 0c4ef72d6f..e5f98845a0 100644 --- a/tcg/loongarch64/tcg-target.c.inc +++ b/tcg/loongarch64/tcg-target.c.inc @@ -844,6 +844,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, TCGReg addr_reg, MemOpIdx oi, bool is_ld) { + TCGType addr_type = s->addr_type; TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); MemOp a_bits; @@ -874,7 +875,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, tcg_out_opc_add_d(s, TCG_REG_TMP2, TCG_REG_TMP2, TCG_REG_TMP1); /* Load the tlb comparator and the addend. */ - tcg_out_ld(s, TCG_TYPE_TL, TCG_REG_TMP0, TCG_REG_TMP2, + tcg_out_ld(s, addr_type, TCG_REG_TMP0, TCG_REG_TMP2, is_ld ? offsetof(CPUTLBEntry, addr_read) : offsetof(CPUTLBEntry, addr_write)); tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP2, TCG_REG_TMP2, @@ -888,9 +889,9 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, if (a_bits < s_bits) { unsigned a_mask = (1u << a_bits) - 1; unsigned s_mask = (1u << s_bits) - 1; - tcg_out_addi(s, TCG_TYPE_TL, TCG_REG_TMP1, addr_reg, s_mask - a_mask); + tcg_out_addi(s, addr_type, TCG_REG_TMP1, addr_reg, s_mask - a_mask); } else { - tcg_out_mov(s, TCG_TYPE_TL, TCG_REG_TMP1, addr_reg); + tcg_out_mov(s, addr_type, TCG_REG_TMP1, addr_reg); } tcg_out_opc_bstrins_d(s, TCG_REG_TMP1, TCG_REG_ZERO, a_bits, TARGET_PAGE_BITS - 1); @@ -923,7 +924,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, h->index = USE_GUEST_BASE ? TCG_GUEST_BASE_REG : TCG_REG_ZERO; #endif - if (TARGET_LONG_BITS == 32) { + if (addr_type == TCG_TYPE_I32) { h->base = TCG_REG_TMP0; tcg_out_ext32u(s, h->base, addr_reg); } else { From patchwork Tue May 16 19:41:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682382 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp18003wrt; Tue, 16 May 2023 12:48:51 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7nWbvvH60V5UAoDG2La80XIE/prIwRrRMB395+62jDKvg2DeZ3peOoo71fL0Ax3LiHuJQH X-Received: by 2002:a05:622a:1811:b0:3f5:434c:3504 with SMTP id t17-20020a05622a181100b003f5434c3504mr5518265qtc.25.1684266531110; Tue, 16 May 2023 12:48:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266531; cv=none; d=google.com; s=arc-20160816; b=F2cH1NmsZ9FMVvv7HbQoPQHyPvQE0w84fGTqOUOTYaHFunT3+jIxljFIDga3fIpo1j MfqtC8YbegPi6q5wUz3PA6q5YMFf1+FMUhQnYPNp5meBYQjOAzhnZA+UJ/1M+SHlu+Cm 2NJ+hOxjzI8J6svORfuOCQ3U5bJo4gFGfE2GBkWnOCsjlRdRlcbRoAezrb1zEse8hFgr 2QzYSMh4zztIocE4ENIA+DYelSZcKp+wJKxbOf7vgpFL0rIZtpQAivbHT+21clScggnX E0wtQ6rrd6Xw1T24jf6XcUGQhk2KLvEh4SWvasrk6JDvqjLfr42+UBmyexFvp0KkvpHQ 6FzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=Yv7aBG6T/iJWBmCYMVKJpJUKFqeiGo5/e+UXaZWl6U0=; b=x6ShMh6rMXeRA9suhRpspGKDQKlIPVgSWLu9ORQWgwbFPopXCwy+dHnYQBeWZK6pvm yZ+dFNDYn9CwO+nkTZF0KIZVAYxaRLl7lEr9nc98PL3zj0EP8aDl6bfnoTbKagjOj1mz 37FsPiOz+mOgiHz6JIRPV7cZkZeAlKFEJdVi7kHrXcJx5dGneahTN7JLgCiWswLITzVM sOITCvBICC1pkEtBzn5CJe/uqO8WhDAJ96EzGTYf4V8bSkYBo3S80t7gHDAEGAjujjUP S0M7xMzwi+ivbBbMwVGBdxtt3no6wCZmlj2cw5kRLrA/UMxbaM6yuh5fE6dWiNpJuxl5 OKmg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tKK0GAoY; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id c17-20020ac87dd1000000b003e4e228cfd2si11732050qte.761.2023.05.16.12.48.50 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:48:51 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tKK0GAoY; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0eO-0002n5-GA; Tue, 16 May 2023 15:48:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cR-0000c3-34 for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:59 -0400 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cM-0003ow-OC for qemu-devel@nongnu.org; Tue, 16 May 2023 15:45:58 -0400 Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-1ae3fe67980so563475ad.3 for ; Tue, 16 May 2023 12:45:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266353; x=1686858353; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Yv7aBG6T/iJWBmCYMVKJpJUKFqeiGo5/e+UXaZWl6U0=; b=tKK0GAoYr52eB7yUdHDrZ31V21SVwutf1PdSBBz/6ITg9FgCuN754YODfhKj0yl6vv Sf1/lu7ZmrfHM7vc19zcPDgopWHnt0zQL1dgXzMAUt82ARQxF57m3ntjqi50X5rmLe83 RYIC9K2oYLkcVJFssmKKccbWXLF1jqgCHuv0NiaaLgWBCLQDsHSeZfEEq5j1haTXzGmz JM2DSIsXV9p7vtDIyh4GkHOPhA23fqiAy9Ep0NtwDJGCIDpBEO7Yg/yfg9/rD+zkhpOS hMW3ZXcn2kvjZXY+8Yj5r3X7dI5U83wE/dk31h1hX7HLrHGAtGoKxAYBUIami9Dia4Cr Pbig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266353; x=1686858353; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Yv7aBG6T/iJWBmCYMVKJpJUKFqeiGo5/e+UXaZWl6U0=; b=dJVX5oa+BIeRwnndG7u+mDiFHR1pGmGi9qxToaOD1bCrjRBrv0HeubOwCaik4eC8eB U9TlS7kD4sC83O2QNdzo4reaRdkFBUi+9YNzCo5Gl8vISqX0SM94iBLr/b6ta9/AuXwj WitltBJqN8BY/W1ysBBbYqrlkMS4iURiowpV6FLVIdrcBS0JfsbYHZBbEochwuBXuQL5 Rn3Tyn+74tPHZ1nrpES4zwQrTADI7sLnw329862D4k9hAo/ZTbCZlGlvzVlRwZrf03iG 7DAY/jD2JdNkfFeTJN23poiZMYSzfyXT5tq7s6sR52LoK+P+jmC5yUMD56cXSqZtBE/Q W5oA== X-Gm-Message-State: AC+VfDyP+Kscjitiq0msDglumOps/PPAMgPWeQaoRkLzDZEAN/uQotda kdCePjDLnoUgynXQKGebxR8fg1Gs3+AxYC6wBKI= X-Received: by 2002:a17:902:ce91:b0:1ad:c1c2:7d1a with SMTP id f17-20020a170902ce9100b001adc1c27d1amr27070824plg.63.1684266353337; Tue, 16 May 2023 12:45:53 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:52 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 76/80] tcg/mips: Remove TARGET_LONG_BITS, TCG_TYPE_TL Date: Tue, 16 May 2023 12:41:41 -0700 Message-Id: <20230516194145.1749305-77-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62d; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org All uses replaced with TCGContext.addr_type. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- tcg/mips/tcg-target.c.inc | 42 +++++++++++++++++++++------------------ 1 file changed, 23 insertions(+), 19 deletions(-) diff --git a/tcg/mips/tcg-target.c.inc b/tcg/mips/tcg-target.c.inc index 7ff4e2ff71..209d95992e 100644 --- a/tcg/mips/tcg-target.c.inc +++ b/tcg/mips/tcg-target.c.inc @@ -354,10 +354,6 @@ typedef enum { /* Aliases for convenience. */ ALIAS_PADD = sizeof(void *) == 4 ? OPC_ADDU : OPC_DADDU, ALIAS_PADDI = sizeof(void *) == 4 ? OPC_ADDIU : OPC_DADDIU, - ALIAS_TSRL = TARGET_LONG_BITS == 32 || TCG_TARGET_REG_BITS == 32 - ? OPC_SRL : OPC_DSRL, - ALIAS_TADDI = TARGET_LONG_BITS == 32 || TCG_TARGET_REG_BITS == 32 - ? OPC_ADDIU : OPC_DADDIU, } MIPSInsn; /* @@ -1156,6 +1152,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, TCGReg addrlo, TCGReg addrhi, MemOpIdx oi, bool is_ld) { + TCGType addr_type = s->addr_type; TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); MemOp a_bits; @@ -1190,23 +1187,26 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, tcg_out_ld(s, TCG_TYPE_PTR, TCG_TMP1, TCG_AREG0, table_off); /* Extract the TLB index from the address into TMP3. */ - tcg_out_opc_sa(s, ALIAS_TSRL, TCG_TMP3, addrlo, - TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); + if (TCG_TARGET_REG_BITS == 32 || addr_type == TCG_TYPE_I32) { + tcg_out_opc_sa(s, OPC_SRL, TCG_TMP3, addrlo, + TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); + } else { + tcg_out_dsrl(s, TCG_TMP3, addrlo, + TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); + } tcg_out_opc_reg(s, OPC_AND, TCG_TMP3, TCG_TMP3, TCG_TMP0); /* Add the tlb_table pointer, creating the CPUTLBEntry address in TMP3. */ tcg_out_opc_reg(s, ALIAS_PADD, TCG_TMP3, TCG_TMP3, TCG_TMP1); - /* Load the (low-half) tlb comparator. */ - if (TCG_TARGET_REG_BITS < TARGET_LONG_BITS) { - tcg_out_ldst(s, OPC_LW, TCG_TMP0, TCG_TMP3, cmp_off + LO_OFF); - } else { - tcg_out_ld(s, TCG_TYPE_TL, TCG_TMP0, TCG_TMP3, cmp_off); - } - - if (TCG_TARGET_REG_BITS >= TARGET_LONG_BITS) { + if (TCG_TARGET_REG_BITS == 64 || addr_type == TCG_TYPE_I32) { + /* Load the tlb comparator. */ + tcg_out_ld(s, addr_type, TCG_TMP0, TCG_TMP3, cmp_off); /* Load the tlb addend for the fast path. */ tcg_out_ld(s, TCG_TYPE_PTR, TCG_TMP3, TCG_TMP3, add_off); + } else { + /* Load the low half of the tlb comparator. */ + tcg_out_ldst(s, OPC_LW, TCG_TMP0, TCG_TMP3, cmp_off + LO_OFF); } /* @@ -1214,16 +1214,20 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, * For unaligned accesses, compare against the end of the access to * verify that it does not cross a page boundary. */ - tcg_out_movi(s, TCG_TYPE_TL, TCG_TMP1, TARGET_PAGE_MASK | a_mask); + tcg_out_movi(s, addr_type, TCG_TMP1, TARGET_PAGE_MASK | a_mask); if (a_mask < s_mask) { - tcg_out_opc_imm(s, ALIAS_TADDI, TCG_TMP2, addrlo, s_mask - a_mask); + if (TCG_TARGET_REG_BITS == 32 || addr_type == TCG_TYPE_I32) { + tcg_out_opc_imm(s, OPC_ADDIU, TCG_TMP2, addrlo, s_mask - a_mask); + } else { + tcg_out_opc_imm(s, OPC_DADDIU, TCG_TMP2, addrlo, s_mask - a_mask); + } tcg_out_opc_reg(s, OPC_AND, TCG_TMP1, TCG_TMP1, TCG_TMP2); } else { tcg_out_opc_reg(s, OPC_AND, TCG_TMP1, TCG_TMP1, addrlo); } /* Zero extend a 32-bit guest address for a 64-bit host. */ - if (TCG_TARGET_REG_BITS > TARGET_LONG_BITS) { + if (TCG_TARGET_REG_BITS == 64 && addr_type == TCG_TYPE_I32) { tcg_out_ext32u(s, TCG_TMP2, addrlo); addrlo = TCG_TMP2; } @@ -1232,7 +1236,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, tcg_out_opc_br(s, OPC_BNE, TCG_TMP1, TCG_TMP0); /* Load and test the high half tlb comparator. */ - if (TCG_TARGET_REG_BITS < TARGET_LONG_BITS) { + if (TCG_TARGET_REG_BITS == 32 && addr_type != TCG_TYPE_I32) { /* delay slot */ tcg_out_ldst(s, OPC_LW, TCG_TMP0, TCG_TMP3, cmp_off + HI_OFF); @@ -1269,7 +1273,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, } base = addrlo; - if (TCG_TARGET_REG_BITS > TARGET_LONG_BITS) { + if (TCG_TARGET_REG_BITS == 64 && addr_type == TCG_TYPE_I32) { tcg_out_ext32u(s, TCG_REG_A0, base); base = TCG_REG_A0; } From patchwork Tue May 16 19:41:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682433 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp23363wrt; Tue, 16 May 2023 13:04:35 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ43fxbuTnVtsdgkqT+9TIrGrP2+9QaVKEd6pkB7//bzaRrF/9ewebVgvG645aWlltmV1iIa X-Received: by 2002:adf:f507:0:b0:309:5d9:bc86 with SMTP id q7-20020adff507000000b0030905d9bc86mr9914774wro.63.1684267475000; Tue, 16 May 2023 13:04:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267474; cv=none; d=google.com; s=arc-20160816; b=f0AxlK10f4153hBsHAqwFGVt2XV061Pq4ouuKcxY6tZajN/Y4iKnyxTxG6gYEJQRwX QczUqN4GrbrZ1LWhcDAs2vaYexvaddd0B9zl/pBESMKdQhEBewFt92RN77cC8YjjJRoo 3QOqPEukeXfiMcHad4cVlJLZilLQ8Q3LC0WsgcMSrr/doyCJl0oWk7ZxXrWxhDoIM8wl aa4hNuv60NgSe7ShmOoycJH6ZYGoXOucEzfS8OnQyNxHk9kop6IVDxOyNgcBwFoAYG+P IyT5wtoMzdmf3+GmqhGmfWWXg4kq2i76zTRNuOeFJZIb2Bx/uysCcRZr+eFPUqAGuN8K 4RoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=e2t4/KP+oyhaRd7avsPSAhu3hAhPGFiLJYL76pKfu08=; b=uie1xcRfDnfw8uRNIDZVa3VHeEFJxmgo+iaefv/t1F3sHanMDnEuZiM7iG9pOawgjW Xf1Dx5VF9kTRdJNFfylzOVVeB3LUA1VQ08RGzp8zrTZth/iqrH2guTmz1IrlSsTGKUzO gbZMPvcpyDT4wSFNNSSw+NhR+kglruKdEbR2ZnXC4I/WrEhXKs2XvOdldnPZc4hLj9H3 Ce9ff0dDG9Zu/9MxxICLnAljqBDqNrBiPwERnBEkE4XILJTKTKzt+YKQRhXZJjYkp9cw xFAF/EK+wtS00F1CHmovWln9P85NNEtVzoPd+kQX2AZGcPn/Z2etKvM/7I1f9UH+7AKH s5nA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=gHxnzoSv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x9-20020a5d6509000000b00307a431f7cbsi185731wru.372.2023.05.16.13.04.34 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 13:04:34 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=gHxnzoSv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0en-00035F-FY; Tue, 16 May 2023 15:48:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cS-0000fT-Hq for qemu-devel@nongnu.org; Tue, 16 May 2023 15:46:00 -0400 Received: from mail-pj1-x1032.google.com ([2607:f8b0:4864:20::1032]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cN-0003pi-Hn for qemu-devel@nongnu.org; Tue, 16 May 2023 15:46:00 -0400 Received: by mail-pj1-x1032.google.com with SMTP id 98e67ed59e1d1-2533d3acd5fso52234a91.2 for ; Tue, 16 May 2023 12:45:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266354; x=1686858354; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=e2t4/KP+oyhaRd7avsPSAhu3hAhPGFiLJYL76pKfu08=; b=gHxnzoSv2+J5gKvHcyEiJjhqkX2IdGDyxlxkIAMs7CDLnGGOanxofW6ZQa4swQjcY1 OpPxuNdjFz/Zlb+7SgtrIeP5Z/yHQ5vsPGZXiIVADxpbHDNpU0zNRO3OsrUKv6XFkpa3 NMXg9fN+zrp1Rdt+FiGzCzB3F51p2ltajtxeWln7ueYCo7omLHnDOB3q2U8xRKPld5wm Qv6iNPQ9SDCwDXI5QDZiTVvZFGUkNDC4d6p/ptjee/7Fm7ylSk64YAkU1hqw1k+rjp6c buRgOdRuSeRLIuXp7hDLEjQ4YmZ5T1ISVk36Xv8Y7URb4WQMqPpdKt1t610Bwat7h1gw ctJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266354; x=1686858354; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=e2t4/KP+oyhaRd7avsPSAhu3hAhPGFiLJYL76pKfu08=; b=iFlQNpr2ZnHobfFUYa+GXswUnV7MkcI10ZMCu0rdvtrq4gGmiTU2fYY6YNc5+Myx2K wrJvc3Tn4NJgXKDNZx3/nShWVLTGL1nBZjckYTWqwPR6wjvIg/4H5m9C82PaLkPnqy+J XAuVC6WDBI/emcqf8lg3nWAq7pHoa/hDHas60Qsy3Et0bxnBl+SJSEOHQRCNkBJvxNAd 8QqJd+9eUxW817jMRT26i9bo4wmaFDBofuZ42QLiCB1of5LlrstVrSI3ETacfwZF3xlh plvX9Da1AueTivSysu+1vun0JqWLA4EUQ/A9fPvIc5Ad8eq56DZ1bylh3RnjHFqMUDl8 2n4A== X-Gm-Message-State: AC+VfDxd6sVLJjQXO80xGxzXZcbtyhxdDR7MNnWOv3pQFuPL6XfiYyhV 4gdXjyIFMeRrU2F3cAAPpAB7yVKSyq1ZuIKhUXU= X-Received: by 2002:a17:90a:cb02:b0:249:6098:b068 with SMTP id z2-20020a17090acb0200b002496098b068mr38362907pjt.45.1684266354040; Tue, 16 May 2023 12:45:54 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:53 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PULL 77/80] tcg: Remove TARGET_LONG_BITS, TCG_TYPE_TL Date: Tue, 16 May 2023 12:41:42 -0700 Message-Id: <20230516194145.1749305-78-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1032; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1032.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org All uses replaced with TCGContext.addr_type. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- tcg/tcg.c | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/tcg/tcg.c b/tcg/tcg.c index 5a2b2b1371..4bd598c18b 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -5612,12 +5612,7 @@ static void tcg_out_ld_helper_args(TCGContext *s, const TCGLabelQemuLdst *ldst, next_arg = 1; loc = &info->in[next_arg]; - if (TCG_TARGET_REG_BITS == 64 || TARGET_LONG_BITS == 64) { - nmov = tcg_out_helper_add_mov(mov, loc, TCG_TYPE_I64, TCG_TYPE_TL, - ldst->addrlo_reg, ldst->addrhi_reg); - tcg_out_helper_load_slots(s, nmov, mov, parm); - next_arg += nmov; - } else { + if (TCG_TARGET_REG_BITS == 32 && s->addr_type == TCG_TYPE_I32) { /* * 32-bit host with 32-bit guest: zero-extend the guest address * to 64-bits for the helper by storing the low part, then @@ -5631,6 +5626,11 @@ static void tcg_out_ld_helper_args(TCGContext *s, const TCGLabelQemuLdst *ldst, tcg_out_helper_load_imm(s, loc[!HOST_BIG_ENDIAN].arg_slot, TCG_TYPE_I32, 0, parm); next_arg += 2; + } else { + nmov = tcg_out_helper_add_mov(mov, loc, TCG_TYPE_I64, s->addr_type, + ldst->addrlo_reg, ldst->addrhi_reg); + tcg_out_helper_load_slots(s, nmov, mov, parm); + next_arg += nmov; } switch (info->out_kind) { @@ -5785,12 +5785,7 @@ static void tcg_out_st_helper_args(TCGContext *s, const TCGLabelQemuLdst *ldst, /* Handle addr argument. */ loc = &info->in[next_arg]; - if (TCG_TARGET_REG_BITS == 64 || TARGET_LONG_BITS == 64) { - n = tcg_out_helper_add_mov(mov, loc, TCG_TYPE_I64, TCG_TYPE_TL, - ldst->addrlo_reg, ldst->addrhi_reg); - next_arg += n; - nmov += n; - } else { + if (TCG_TARGET_REG_BITS == 32 && s->addr_type == TCG_TYPE_I32) { /* * 32-bit host with 32-bit guest: zero-extend the guest address * to 64-bits for the helper by storing the low part. Later, @@ -5802,6 +5797,11 @@ static void tcg_out_st_helper_args(TCGContext *s, const TCGLabelQemuLdst *ldst, ldst->addrlo_reg, -1); next_arg += 2; nmov += 1; + } else { + n = tcg_out_helper_add_mov(mov, loc, TCG_TYPE_I64, s->addr_type, + ldst->addrlo_reg, ldst->addrhi_reg); + next_arg += n; + nmov += n; } /* Handle data argument. */ @@ -5847,7 +5847,8 @@ static void tcg_out_st_helper_args(TCGContext *s, const TCGLabelQemuLdst *ldst, g_assert_not_reached(); } - if (TCG_TARGET_REG_BITS == 32 && TARGET_LONG_BITS == 32) { + if (TCG_TARGET_REG_BITS == 32 && s->addr_type == TCG_TYPE_I32) { + /* Zero extend the address by loading a zero for the high part. */ loc = &info->in[1 + !HOST_BIG_ENDIAN]; tcg_out_helper_load_imm(s, loc->arg_slot, TCG_TYPE_I32, 0, parm); } From patchwork Tue May 16 19:41:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682393 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp19379wrt; Tue, 16 May 2023 12:53:24 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4qRXHaZS5AE9q9jwEcfa2Gq6kOHsyaOY+kPSsbV1Mqq75IR5T2OmTxYsgFqQQEFOZZjRt0 X-Received: by 2002:a05:6214:d05:b0:56b:f28a:ee2d with SMTP id 5-20020a0562140d0500b0056bf28aee2dmr70589549qvh.5.1684266803811; Tue, 16 May 2023 12:53:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266803; cv=none; d=google.com; s=arc-20160816; b=hPphEgWgfSSio4AqkP9LNynba6S8jF7i4h6X6NkfqIFjtfXgD7Cqyg+CM4rEMjZAE4 Sbx8lWVXMlwyAaSV0Rt0ZLEVO6TEuYhSt0lP7z+O6P9DxE1Ri8b9XOkA5he3qdLdJoKc 3XkNJgoHPh7vef1GG2v8zjm/SjKnDtv4Ons6yqfObwnCXHJHUKZGSxNNRWMVT7ikeR+p LWXBDJsvU1vAhCWaBPiyujQzPs+ZeleiPFwTB8KPOXshdyWiyrr07F1B8EWQ1A8eRTze Vu8wKqF4F3O5i22NNSWqRHJ127OYEq/4K0rYxRJlNKOd9WsZLPfZh/9W7PPso/Dh9Ao3 ccAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=m8mhdjpDzRobguZAE7O6O+FOFP4jRA9PZjO71HHlvs0=; b=jFiUHDhzb42NXH6MaJyigsJZv/mhdVefr1yYqqltkGZtG/5bGfnHJ0NbyfuP/9Cx1i Eljro7kXbUIE3YwPtKaUvzH3gtzBggXbzOKA4TD8cXklPw2+45VYug0f882pxY/zYZ4h 4XH1qi3juQuZ9tURRLZtoADGaM/uE/R45RW7AbIYg1ueLNogkUY8cjZUaHS+eqelfULb fxUyMkH3dVBtGNA3TTqwCqvVOKJxAebEEDnbRLqAIUO6A15P6A4Df5RsA033ZRtfo9a7 V5H3qCh8pE+fQ8UphPZFSuk0ISlA01TflV3J6qhbucwXFRSKQmII5QTsmINEofjBxPTW 5fgQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fMrHdoTl; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id iu12-20020ad45ccc000000b006237c5646c6si1235335qvb.592.2023.05.16.12.53.23 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:53:23 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fMrHdoTl; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0f5-00042N-J2; Tue, 16 May 2023 15:48:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cU-0000iN-NT for qemu-devel@nongnu.org; Tue, 16 May 2023 15:46:03 -0400 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cO-0003qG-MD for qemu-devel@nongnu.org; Tue, 16 May 2023 15:46:02 -0400 Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-64a9335a8e7so8239521b3a.0 for ; Tue, 16 May 2023 12:45:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266355; x=1686858355; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=m8mhdjpDzRobguZAE7O6O+FOFP4jRA9PZjO71HHlvs0=; b=fMrHdoTloprN/u9EVdHAZvckWKQrDAabuZt1NjrBqaPjJhKznMLclXPVz+njhTG6q5 hmsi9uYCtMTDZcVywYFVo7ZtQRJ5o2RABrf3PXXFfvJEPZwmyWScQPoXBhJIbdTcKIC0 Igoddwd+g0QD7/iHxYG9zZCIvuG7h8mHOpRuHUoqSoJ2fWqs1J+Nh4SIpWA0drFLOB0D Tq5y6nh7hm880ndMWWtA2/R75vfYIrJNf6IPoi47gRP8HjWHqrjJQz21hd/NS4EJE0Zi sR9HAT4Q8W2NGy7LcCAxMjAVVxse3/aIL9xe7VbZADtpD73n9U7a/67Bu6Z7P9TDvYQZ gyuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266355; x=1686858355; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=m8mhdjpDzRobguZAE7O6O+FOFP4jRA9PZjO71HHlvs0=; b=LxLDcwPwcjnuy8a1OYuN/YBCqVXfh+oE3oYEkQ2mo/mn5EhlMRrtH8gl7rxl//KgUG vVBI3APC4UK4DfqcrTZ2FOPbglv+UMYY09ZvEId39i9g/BBmJpPZPGq5np6sgs8KyZW9 5vINNaiqiu0pgz8MTkBwdDxsvTDZENA1AEeXbPu8sC5+eZU9fGEqgLl0P1hvTuNSEwFV n5myRpUrkzh7xeRjamyE9aCrFhRRsIv5feYO8qmNRtgn+6gYtk+9xw2f9N9sk3PCG1K+ VYxWBputexmXs3/lKspz2SdR29eXuhhRkJ5vUiEJ1gfrRvz6TfmHsWrE0VqxzcWEn9DH WLEQ== X-Gm-Message-State: AC+VfDxJV4cwhVu4Ujuf5a1PPnOupvscw2QDLk87Q4SbGeXidScfzIBV XUxJ6x5NKLh/4C9KmW3AGGEN/hXbcjXsymYAkoU= X-Received: by 2002:a17:90a:d78f:b0:23f:962e:825d with SMTP id z15-20020a17090ad78f00b0023f962e825dmr45269329pju.1.1684266355022; Tue, 16 May 2023 12:45:55 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:54 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Anton Johansson Subject: [PULL 78/80] tcg: Add page_bits and page_mask to TCGContext Date: Tue, 16 May 2023 12:41:43 -0700 Message-Id: <20230516194145.1749305-79-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::432; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x432.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Disconnect guest page size from TCG compilation. While this could be done via exec/target_page.h, we want to cache the value across multiple memory access operations, so we might as well initialize this early. The changes within tcg/ are entirely mechanical: sed -i s/TARGET_PAGE_BITS/s->page_bits/g sed -i s/TARGET_PAGE_MASK/s->page_mask/g Reviewed-by: Anton Johansson Signed-off-by: Richard Henderson --- include/tcg/tcg.h | 5 +++++ accel/tcg/translate-all.c | 4 ++++ tcg/aarch64/tcg-target.c.inc | 6 +++--- tcg/arm/tcg-target.c.inc | 10 +++++----- tcg/i386/tcg-target.c.inc | 6 +++--- tcg/loongarch64/tcg-target.c.inc | 4 ++-- tcg/mips/tcg-target.c.inc | 6 +++--- tcg/ppc/tcg-target.c.inc | 14 +++++++------- tcg/riscv/tcg-target.c.inc | 4 ++-- tcg/s390x/tcg-target.c.inc | 4 ++-- tcg/sparc64/tcg-target.c.inc | 4 ++-- 11 files changed, 38 insertions(+), 29 deletions(-) diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h index b9748fd0c5..db57c4d492 100644 --- a/include/tcg/tcg.h +++ b/include/tcg/tcg.h @@ -560,6 +560,11 @@ struct TCGContext { int nb_ops; TCGType addr_type; /* TCG_TYPE_I32 or TCG_TYPE_I64 */ +#ifdef CONFIG_SOFTMMU + int page_mask; + uint8_t page_bits; +#endif + TCGRegSet reserved_regs; intptr_t current_frame_offset; intptr_t frame_start; diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c index 99a9d0e34f..ca306f67da 100644 --- a/accel/tcg/translate-all.c +++ b/accel/tcg/translate-all.c @@ -357,6 +357,10 @@ TranslationBlock *tb_gen_code(CPUState *cpu, tb_set_page_addr1(tb, -1); tcg_ctx->gen_tb = tb; tcg_ctx->addr_type = TCG_TYPE_TL; +#ifdef CONFIG_SOFTMMU + tcg_ctx->page_bits = TARGET_PAGE_BITS; + tcg_ctx->page_mask = TARGET_PAGE_MASK; +#endif tb_overflow: diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index 41838f8170..8b7c679349 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -1685,7 +1685,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, ldst->oi = oi; ldst->addrlo_reg = addr_reg; - mask_type = (TARGET_PAGE_BITS + CPU_TLB_DYN_MAX_BITS > 32 + mask_type = (s->page_bits + CPU_TLB_DYN_MAX_BITS > 32 ? TCG_TYPE_I64 : TCG_TYPE_I32); /* Load env_tlb(env)->f[mmu_idx].{mask,table} into {x0,x1}. */ @@ -1699,7 +1699,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, /* Extract the TLB index from the address into X0. */ tcg_out_insn(s, 3502S, AND_LSR, mask_type == TCG_TYPE_I64, TCG_REG_X0, TCG_REG_X0, addr_reg, - TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); + s->page_bits - CPU_TLB_ENTRY_BITS); /* Add the tlb_table pointer, creating the CPUTLBEntry address into X1. */ tcg_out_insn(s, 3502, ADD, 1, TCG_REG_X1, TCG_REG_X1, TCG_REG_X0); @@ -1723,7 +1723,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, TCG_REG_X3, addr_reg, s_mask - a_mask); x3 = TCG_REG_X3; } - compare_mask = (uint64_t)TARGET_PAGE_MASK | a_mask; + compare_mask = (uint64_t)s->page_mask | a_mask; /* Store the page mask part of the address into X3. */ tcg_out_logicali(s, I3404_ANDI, addr_type, TCG_REG_X3, x3, compare_mask); diff --git a/tcg/arm/tcg-target.c.inc b/tcg/arm/tcg-target.c.inc index 3c38e868e2..20cc1cc477 100644 --- a/tcg/arm/tcg-target.c.inc +++ b/tcg/arm/tcg-target.c.inc @@ -1424,7 +1424,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, /* Extract the tlb index from the address into R0. */ tcg_out_dat_reg(s, COND_AL, ARITH_AND, TCG_REG_R0, TCG_REG_R0, addrlo, - SHIFT_IMM_LSR(TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS)); + SHIFT_IMM_LSR(s->page_bits - CPU_TLB_ENTRY_BITS)); /* * Add the tlb_table pointer, creating the CPUTLBEntry address in R1. @@ -1468,8 +1468,8 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, tcg_out_dat_imm(s, COND_AL, ARITH_ADD, t_addr, addrlo, s_mask - a_mask); } - if (use_armv7_instructions && TARGET_PAGE_BITS <= 16) { - tcg_out_movi32(s, COND_AL, TCG_REG_TMP, ~(TARGET_PAGE_MASK | a_mask)); + if (use_armv7_instructions && s->page_bits <= 16) { + tcg_out_movi32(s, COND_AL, TCG_REG_TMP, ~(s->page_mask | a_mask)); tcg_out_dat_reg(s, COND_AL, ARITH_BIC, TCG_REG_TMP, t_addr, TCG_REG_TMP, 0); tcg_out_dat_reg(s, COND_AL, ARITH_CMP, 0, TCG_REG_R2, TCG_REG_TMP, 0); @@ -1479,10 +1479,10 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, tcg_out_dat_imm(s, COND_AL, ARITH_TST, 0, addrlo, a_mask); } tcg_out_dat_reg(s, COND_AL, ARITH_MOV, TCG_REG_TMP, 0, t_addr, - SHIFT_IMM_LSR(TARGET_PAGE_BITS)); + SHIFT_IMM_LSR(s->page_bits)); tcg_out_dat_reg(s, (a_mask ? COND_EQ : COND_AL), ARITH_CMP, 0, TCG_REG_R2, TCG_REG_TMP, - SHIFT_IMM_LSL(TARGET_PAGE_BITS)); + SHIFT_IMM_LSL(s->page_bits)); } if (s->addr_type != TCG_TYPE_I32) { diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index e173853dc4..d766e8652c 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -1979,7 +1979,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, trexw = (ttype == TCG_TYPE_I32 ? 0 : P_REXW); if (TCG_TYPE_PTR == TCG_TYPE_I64) { hrexw = P_REXW; - if (TARGET_PAGE_BITS + CPU_TLB_DYN_MAX_BITS > 32) { + if (s->page_bits + CPU_TLB_DYN_MAX_BITS > 32) { tlbtype = TCG_TYPE_I64; tlbrexw = P_REXW; } @@ -1988,7 +1988,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, tcg_out_mov(s, tlbtype, TCG_REG_L0, addrlo); tcg_out_shifti(s, SHIFT_SHR + tlbrexw, TCG_REG_L0, - TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); + s->page_bits - CPU_TLB_ENTRY_BITS); tcg_out_modrm_offset(s, OPC_AND_GvEv + trexw, TCG_REG_L0, TCG_AREG0, TLB_MASK_TABLE_OFS(mem_index) + @@ -2009,7 +2009,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, tcg_out_modrm_offset(s, OPC_LEA + trexw, TCG_REG_L1, addrlo, s_mask - a_mask); } - tlb_mask = TARGET_PAGE_MASK | a_mask; + tlb_mask = s->page_mask | a_mask; tgen_arithi(s, ARITH_AND + trexw, TCG_REG_L1, tlb_mask, 0); /* cmp 0(TCG_REG_L0), TCG_REG_L1 */ diff --git a/tcg/loongarch64/tcg-target.c.inc b/tcg/loongarch64/tcg-target.c.inc index e5f98845a0..0bae922982 100644 --- a/tcg/loongarch64/tcg-target.c.inc +++ b/tcg/loongarch64/tcg-target.c.inc @@ -870,7 +870,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP1, TCG_AREG0, table_ofs); tcg_out_opc_srli_d(s, TCG_REG_TMP2, addr_reg, - TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); + s->page_bits - CPU_TLB_ENTRY_BITS); tcg_out_opc_and(s, TCG_REG_TMP2, TCG_REG_TMP2, TCG_REG_TMP0); tcg_out_opc_add_d(s, TCG_REG_TMP2, TCG_REG_TMP2, TCG_REG_TMP1); @@ -894,7 +894,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, tcg_out_mov(s, addr_type, TCG_REG_TMP1, addr_reg); } tcg_out_opc_bstrins_d(s, TCG_REG_TMP1, TCG_REG_ZERO, - a_bits, TARGET_PAGE_BITS - 1); + a_bits, s->page_bits - 1); /* Compare masked address with the TLB entry. */ ldst->label_ptr[0] = s->code_ptr; diff --git a/tcg/mips/tcg-target.c.inc b/tcg/mips/tcg-target.c.inc index 209d95992e..ef146b193c 100644 --- a/tcg/mips/tcg-target.c.inc +++ b/tcg/mips/tcg-target.c.inc @@ -1189,10 +1189,10 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, /* Extract the TLB index from the address into TMP3. */ if (TCG_TARGET_REG_BITS == 32 || addr_type == TCG_TYPE_I32) { tcg_out_opc_sa(s, OPC_SRL, TCG_TMP3, addrlo, - TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); + s->page_bits - CPU_TLB_ENTRY_BITS); } else { tcg_out_dsrl(s, TCG_TMP3, addrlo, - TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); + s->page_bits - CPU_TLB_ENTRY_BITS); } tcg_out_opc_reg(s, OPC_AND, TCG_TMP3, TCG_TMP3, TCG_TMP0); @@ -1214,7 +1214,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, * For unaligned accesses, compare against the end of the access to * verify that it does not cross a page boundary. */ - tcg_out_movi(s, addr_type, TCG_TMP1, TARGET_PAGE_MASK | a_mask); + tcg_out_movi(s, addr_type, TCG_TMP1, s->page_mask | a_mask); if (a_mask < s_mask) { if (TCG_TARGET_REG_BITS == 32 || addr_type == TCG_TYPE_I32) { tcg_out_opc_imm(s, OPC_ADDIU, TCG_TMP2, addrlo, s_mask - a_mask); diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc index f74218b13f..7e2c0b26ab 100644 --- a/tcg/ppc/tcg-target.c.inc +++ b/tcg/ppc/tcg-target.c.inc @@ -2090,10 +2090,10 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, /* Extract the page index, shifted into place for tlb index. */ if (TCG_TARGET_REG_BITS == 32) { tcg_out_shri32(s, TCG_REG_R0, addrlo, - TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); + s->page_bits - CPU_TLB_ENTRY_BITS); } else { tcg_out_shri64(s, TCG_REG_R0, addrlo, - TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); + s->page_bits - CPU_TLB_ENTRY_BITS); } tcg_out32(s, AND | SAB(TCG_REG_TMP1, TCG_REG_TMP1, TCG_REG_R0)); @@ -2132,7 +2132,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, a_bits = s_bits; } tcg_out_rlw(s, RLWINM, TCG_REG_R0, addrlo, 0, - (32 - a_bits) & 31, 31 - TARGET_PAGE_BITS); + (32 - a_bits) & 31, 31 - s->page_bits); } else { TCGReg t = addrlo; @@ -2153,13 +2153,13 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, /* Mask the address for the requested alignment. */ if (TARGET_LONG_BITS == 32) { tcg_out_rlw(s, RLWINM, TCG_REG_R0, t, 0, - (32 - a_bits) & 31, 31 - TARGET_PAGE_BITS); + (32 - a_bits) & 31, 31 - s->page_bits); } else if (a_bits == 0) { - tcg_out_rld(s, RLDICR, TCG_REG_R0, t, 0, 63 - TARGET_PAGE_BITS); + tcg_out_rld(s, RLDICR, TCG_REG_R0, t, 0, 63 - s->page_bits); } else { tcg_out_rld(s, RLDICL, TCG_REG_R0, t, - 64 - TARGET_PAGE_BITS, TARGET_PAGE_BITS - a_bits); - tcg_out_rld(s, RLDICL, TCG_REG_R0, TCG_REG_R0, TARGET_PAGE_BITS, 0); + 64 - s->page_bits, s->page_bits - a_bits); + tcg_out_rld(s, RLDICL, TCG_REG_R0, TCG_REG_R0, s->page_bits, 0); } } diff --git a/tcg/riscv/tcg-target.c.inc b/tcg/riscv/tcg-target.c.inc index de61edb5df..ff6334980f 100644 --- a/tcg/riscv/tcg-target.c.inc +++ b/tcg/riscv/tcg-target.c.inc @@ -937,7 +937,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, TCGReg *pbase, tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP1, TCG_AREG0, table_ofs); tcg_out_opc_imm(s, OPC_SRLI, TCG_REG_TMP2, addr_reg, - TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); + s->page_bits - CPU_TLB_ENTRY_BITS); tcg_out_opc_reg(s, OPC_AND, TCG_REG_TMP2, TCG_REG_TMP2, TCG_REG_TMP0); tcg_out_opc_reg(s, OPC_ADD, TCG_REG_TMP2, TCG_REG_TMP2, TCG_REG_TMP1); @@ -952,7 +952,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, TCGReg *pbase, tcg_out_opc_imm(s, TARGET_LONG_BITS == 32 ? OPC_ADDIW : OPC_ADDI, addr_adj, addr_reg, s_mask - a_mask); } - compare_mask = TARGET_PAGE_MASK | a_mask; + compare_mask = s->page_mask | a_mask; if (compare_mask == sextreg(compare_mask, 0, 12)) { tcg_out_opc_imm(s, OPC_ANDI, TCG_REG_TMP1, addr_adj, compare_mask); } else { diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc index 7606073c81..503126cd66 100644 --- a/tcg/s390x/tcg-target.c.inc +++ b/tcg/s390x/tcg-target.c.inc @@ -1768,7 +1768,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, ldst->addrlo_reg = addr_reg; tcg_out_sh64(s, RSY_SRLG, TCG_TMP0, addr_reg, TCG_REG_NONE, - TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); + s->page_bits - CPU_TLB_ENTRY_BITS); QEMU_BUILD_BUG_ON(TLB_MASK_TABLE_OFS(0) > 0); QEMU_BUILD_BUG_ON(TLB_MASK_TABLE_OFS(0) < -(1 << 19)); @@ -1781,7 +1781,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, * cross pages using the address of the last byte of the access. */ a_off = (a_mask >= s_mask ? 0 : s_mask - a_mask); - tlb_mask = (uint64_t)TARGET_PAGE_MASK | a_mask; + tlb_mask = (uint64_t)s->page_mask | a_mask; if (a_off == 0) { tgen_andi_risbg(s, TCG_REG_R0, addr_reg, tlb_mask); } else { diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index 6e6c26d470..d2d0f604c2 100644 --- a/tcg/sparc64/tcg-target.c.inc +++ b/tcg/sparc64/tcg-target.c.inc @@ -1056,7 +1056,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, /* Extract the page index, shifted into place for tlb index. */ tcg_out_arithi(s, TCG_REG_T1, addr_reg, - TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS, SHIFT_SRL); + s->page_bits - CPU_TLB_ENTRY_BITS, SHIFT_SRL); tcg_out_arith(s, TCG_REG_T1, TCG_REG_T1, TCG_REG_T2, ARITH_AND); /* Add the tlb_table pointer, creating the CPUTLBEntry address into R2. */ @@ -1068,7 +1068,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, h->base = TCG_REG_T1; /* Mask out the page offset, except for the required alignment. */ - compare_mask = TARGET_PAGE_MASK | a_mask; + compare_mask = s->page_mask | a_mask; if (check_fit_tl(compare_mask, 13)) { tcg_out_arithi(s, TCG_REG_T3, addr_reg, compare_mask, ARITH_AND); } else { From patchwork Tue May 16 19:41:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682421 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp21063wrt; Tue, 16 May 2023 12:59:29 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6eQTCq2e1D2RuEBQiyAQZN3l4KvnWQThNNoFr/GvAdsC0gXPG9zTIA96zgLmRa6J7edrkt X-Received: by 2002:adf:ef47:0:b0:2f0:e287:1fbc with SMTP id c7-20020adfef47000000b002f0e2871fbcmr27125186wrp.11.1684267169302; Tue, 16 May 2023 12:59:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684267169; cv=none; d=google.com; s=arc-20160816; b=vnRmSCjvL6ImHX9Z/UadK+7wPUitkAluMCFPoJ28odZn+wY9SSB7a1cLcVvwOZNMhr BW3JUCw+zAoJsfNXMwE8T4aKdzJTfca5QO4HR+W2z0EoAurhMrLdRTKqSpoNWlKxpf6G V4Re/UppI2jcggq2oZlLjVSlPEc0oyz+qe/w7lNnWmt2SVF/BnFY5EQcWb/ljtCuwkO8 q8TOm3OQP/yz2lLNpyZjNlseamUlZZ6vgPqGJo7vV6rt+qMNfWYv66iEx0Coe3CJPm78 yaI8osVa5fPhwKUTIe/IazqhsnkFaUTJcwI7D3sgS4mP9zw/5uVz28EPiqwFn/HX20gW NQsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=Mg31tFrpqaSLWLouZqalEVgyQQeTv3Dfr7kkgOUM8nY=; b=VdgKXstLA72fpkSCk5+Zo3nYc8zL0zha4wbpSu/+iYMS07CksLMJ4Pgl4b6CJGz/ao zzxDpnWfNaLpf1P6dQUShFVnAiFt5iePGoCGkCv+i2tRGCUe1uWXIlFev5qRr0tnuB+T z61CyGHdiS5GTHkzj94Lf5iOj6uiJGCMJYRL/UK7TWnrGCkcq0Ljjyr2vIgJYXEA6Wa0 jn23JJ17QdPIvUKpV1UZkF3z5if764sXuWPlhJ1Z12UxLewoZyx5wtDf1jhUKKwAQh8Y H88PO9JJPv28WIxVV7NJhDDID1MopAm0lFVEFNKQA2UqVtKnKjaWXI3Yy0qTXXRh03Tj NKWA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=wTW8P7Ye; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t7-20020adfe447000000b0030382a5814fsi198260wrm.266.2023.05.16.12.59.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:59:29 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=wTW8P7Ye; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0ek-000339-Rc; Tue, 16 May 2023 15:48:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0cU-0000hY-0c for qemu-devel@nongnu.org; Tue, 16 May 2023 15:46:02 -0400 Received: from mail-pl1-x630.google.com ([2607:f8b0:4864:20::630]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0cQ-0003qe-0L for qemu-devel@nongnu.org; Tue, 16 May 2023 15:46:01 -0400 Received: by mail-pl1-x630.google.com with SMTP id d9443c01a7336-1ae4f28454bso402895ad.3 for ; Tue, 16 May 2023 12:45:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266355; x=1686858355; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Mg31tFrpqaSLWLouZqalEVgyQQeTv3Dfr7kkgOUM8nY=; b=wTW8P7YeA5Fly63x91sPNM8g0SRl/OogBYqWeajqbbLQKrZnFS/P6PojlSxQPCbO43 za5TWZkpe7kbQpWxcVKelsrlAaG4L/ICP57wXJwlzF759uF6uPhUgbcwViOZSMNoJGpz itFg2CH1osDbqZO1VflKEAFvgN3F1G5Zaiuu1slV39+9+pLsz5qvVrtaFBAUswU/wD+j +8n6wIqfXqNu7bFJ+1t9cVi8XkFqnIZN1eQf+anZE08VEDVmYVeOxWiLbxEg5k1hHlIy ITpHGT07W9P2Lz0iM+tZXNaECYa0UJfDrX/bzPegd4/MuX+eNMDMdJ/5CjafOMvl9ERe 2+OQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266355; x=1686858355; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Mg31tFrpqaSLWLouZqalEVgyQQeTv3Dfr7kkgOUM8nY=; b=bsLduWCmrZABSS7s0y3siFiEL91ReyzdZ1U9fUJXQg4UrJykRASXusl0qsTMCJmL6m NjRm1lPT20Q8uvzUj+HsrBDKl4A1DxKtv6qddwpMWjjUCV5hooV4REnv7T0ztclFsnWS YkcUias8pTEZMVzD9nZwylUXQ2YAvO3/mEq3snkBeFzy+WOtTmjf63Cj2q+ej12K0rMQ fSHsiIju/1YSI23Z7YhR/3OXXn8SNzXn5yMEMke28EwFhvE99jxYy9v6zW1kGIQX4sg7 /yaq/O6QKzYFS7zS7MOi+vlrrWev2w8CJL9Ay2QNToxDkpraMdlR7hMk/l5zJjG6ZVbX Kg3w== X-Gm-Message-State: AC+VfDxyUgCpnmFhmFpaJqx6RZiRCraULroo0rg0pjbKLW4QdzT6MLMO OFe7xfiXtOl//O59DAocHh+Q7QrZJCW/gnw+cVo= X-Received: by 2002:a17:90b:4c8c:b0:252:8910:db42 with SMTP id my12-20020a17090b4c8c00b002528910db42mr19930992pjb.1.1684266355747; Tue, 16 May 2023 12:45:55 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id y12-20020a17090abd0c00b002508d73f4e8sm25555pjr.57.2023.05.16.12.45.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:45:55 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Anton Johansson Subject: [PULL 79/80] tcg: Add tlb_dyn_max_bits to TCGContext Date: Tue, 16 May 2023 12:41:44 -0700 Message-Id: <20230516194145.1749305-80-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::630; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x630.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Disconnect guest tlb parameters from TCG compilation. Reviewed-by: Anton Johansson Signed-off-by: Richard Henderson --- include/tcg/tcg.h | 1 + accel/tcg/translate-all.c | 1 + tcg/aarch64/tcg-target.c.inc | 2 +- tcg/i386/tcg-target.c.inc | 2 +- 4 files changed, 4 insertions(+), 2 deletions(-) diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h index db57c4d492..cd6327b175 100644 --- a/include/tcg/tcg.h +++ b/include/tcg/tcg.h @@ -563,6 +563,7 @@ struct TCGContext { #ifdef CONFIG_SOFTMMU int page_mask; uint8_t page_bits; + uint8_t tlb_dyn_max_bits; #endif TCGRegSet reserved_regs; diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c index ca306f67da..353849ca6d 100644 --- a/accel/tcg/translate-all.c +++ b/accel/tcg/translate-all.c @@ -360,6 +360,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu, #ifdef CONFIG_SOFTMMU tcg_ctx->page_bits = TARGET_PAGE_BITS; tcg_ctx->page_mask = TARGET_PAGE_MASK; + tcg_ctx->tlb_dyn_max_bits = CPU_TLB_DYN_MAX_BITS; #endif tb_overflow: diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index 8b7c679349..b58b39a07f 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -1685,7 +1685,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, ldst->oi = oi; ldst->addrlo_reg = addr_reg; - mask_type = (s->page_bits + CPU_TLB_DYN_MAX_BITS > 32 + mask_type = (s->page_bits + s->tlb_dyn_max_bits > 32 ? TCG_TYPE_I64 : TCG_TYPE_I32); /* Load env_tlb(env)->f[mmu_idx].{mask,table} into {x0,x1}. */ diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index d766e8652c..51dfd18ab0 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -1979,7 +1979,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, trexw = (ttype == TCG_TYPE_I32 ? 0 : P_REXW); if (TCG_TYPE_PTR == TCG_TYPE_I64) { hrexw = P_REXW; - if (s->page_bits + CPU_TLB_DYN_MAX_BITS > 32) { + if (s->page_bits + s->tlb_dyn_max_bits > 32) { tlbtype = TCG_TYPE_I64; tlbrexw = P_REXW; } From patchwork Tue May 16 19:41:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 682396 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp19605wrt; Tue, 16 May 2023 12:54:11 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ59sNTDyzFTu+vfzLDcp+omgJyWR6NGilp3AVoQUGZydmd4BlprWlnYcddFthtk3fEWADCj X-Received: by 2002:a05:622a:194:b0:3f5:2831:7420 with SMTP id s20-20020a05622a019400b003f528317420mr14880919qtw.29.1684266851645; Tue, 16 May 2023 12:54:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684266851; cv=none; d=google.com; s=arc-20160816; b=OMHUPkb48MoxxjEtoAWN7KKrNBSVaqOGdfAqOGkOqBd6ciqIaT/jFQEwv3fSminnex Ey40g6ATqUJLV1M5r8kDBvv2gz8wgVNr6Qvgb2J1vII9wVy0h2FAsXhn2RwZKQSKqdzN hKEfxaIFMme0ZLSL0Nw366Cmaro6NlrTXCiTInGi5sy+UxHbUSHTDn0H9Ir2lFbmG1cH fa+ZEtvcJ1cN/Nb1joasunmOS1/YHdjZlEolrUhlHWBzG4AjRjPne9eX+DpBM6v1HOgt 3L7AegoApxIonMuFmcaMSvJBT/okFxB7kxqtjprRWRtVq12ZVeJ2qHEZazm9R8DSBCD3 ITKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=4zI9fsHzivwwTbDjNnN4jTG49Yza6ytpst3ZQDVjthU=; b=o3/K1zxKp/qJAu06f+rUV7TbPhgToU0hS6+kpd33br+7c5C78Ww3CCDENX5xsEIAbC oQMbMewsTSGHGlSneV1Qg/PzojWC3lRl+oe2mH6fnqgMOVpRVY/IcWbgPyw+WiC5ulFT gvWoM0omfmNJIcMN/FkoGotnfvADRn98N6CyA2W9BdxXZneyAOxCoLPN0+jTlxq+Ot+0 4Q3tbbJ5kbAuWa6xZJsyCAQs4LjA8046Cayd9nhqxauQmuDsMfoSVqhCoL6+B4yOUAnT Ywpll39af4EHqqxa4QExZ5XHhbdWxXAih6lLNlRg5Tw90XFqOIS1/xHpcte7uvwB6xx9 OH6A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=g6JdDYWa; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i5-20020a05620a144500b007465ad84195si252136qkl.183.2023.05.16.12.54.11 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 May 2023 12:54:11 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=g6JdDYWa; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pz0fY-0005q0-LI; Tue, 16 May 2023 15:49:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pz0fM-0005mi-8a for qemu-devel@nongnu.org; Tue, 16 May 2023 15:49:02 -0400 Received: from mail-pl1-x62a.google.com ([2607:f8b0:4864:20::62a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pz0fK-0004Lg-NK for qemu-devel@nongnu.org; Tue, 16 May 2023 15:49:00 -0400 Received: by mail-pl1-x62a.google.com with SMTP id d9443c01a7336-1ab1ce53ca6so529945ad.0 for ; Tue, 16 May 2023 12:48:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684266537; x=1686858537; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4zI9fsHzivwwTbDjNnN4jTG49Yza6ytpst3ZQDVjthU=; b=g6JdDYWasT+SIIL3iUOf4qecqUOOxw5eor5KoNaKhzOZucpwwe8BcEm8NDVYutp01X HRtBv9bILLXNvOS4px4PRwM6Q+lNSElbcqO3O2Yb8DFT2d6pU8Ykdp/Im69aA0lxZ56R tPRsSO1XwWmKCCgbLCX8IcSTIITRY/zdzENzKnDGKitTbQFdVdXoPVO4BLf0DGO6cTmt T2L73lMFNfQaP//bZJa6AXy1TxmDibED5Kn705WqlsMy5CzgAqkQBmq/2RcaFHleyuqn RFO+9+mbbUWHwI9JVeG/qpwO+XB15F0NQLqtJcmr0o2cOugml7hB/r7UwOG1Vt4b2+yB a9gQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684266537; x=1686858537; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4zI9fsHzivwwTbDjNnN4jTG49Yza6ytpst3ZQDVjthU=; b=k41tp4HjDUvVZZ4uZJTx1c2qff5US3fPCf5Oaz1PqsNOM0lylbOMbcJlrbb44J60rK WscPoUaioQIIBm7Fb8Y3E+xNmW0AJvQmKv+psuDuqIdEW3x7u0DaKFPCvbuudxRAlXdn gLb3J6DzJOPqsvEasxAHgWm3m44gtNMpbLWIS8DFYrYW4R3jS1QoWC4Qdr76cARKV+Qq TbCXaru/TK0Geawh/EcOBokWERKxJKgbw/JNhRhxvOe1tnBakwIna0v6hqH9FyEgbRvy pcKWsBDjeg8md4ya7WWAcQmwxcTKOZVJZS8qxlPW8cl8Rw/6oLiJBvvEou9M4NzJNh2g l7fA== X-Gm-Message-State: AC+VfDweEDmxjQI01gnGSzpGyXhYNRkvD7tEZ91TONDs1NLpyxqMX6HZ 4S1l9pxmSRn6PnvshNHz/W02nzMZmMgac1aB1mY= X-Received: by 2002:a17:90a:bc88:b0:24f:13ec:ecac with SMTP id x8-20020a17090abc8800b0024f13ececacmr37937326pjr.26.1684266537122; Tue, 16 May 2023 12:48:57 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:ec81:440e:33a4:40b9]) by smtp.gmail.com with ESMTPSA id 78-20020a630351000000b0051815eae23esm13843233pgd.27.2023.05.16.12.48.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 12:48:56 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Thomas Huth Subject: [PULL 80/80] tcg: Split out exec/user/guest-base.h Date: Tue, 16 May 2023 12:41:45 -0700 Message-Id: <20230516194145.1749305-81-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230516194145.1749305-1-richard.henderson@linaro.org> References: <20230516194145.1749305-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62a; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org TCG will need this declaration, without all of the other bits that come with cpu-all.h. Reviewed-by: Thomas Huth Signed-off-by: Richard Henderson --- include/exec/cpu-all.h | 5 +---- include/exec/user/guest-base.h | 12 ++++++++++++ tcg/tcg.c | 3 +++ 3 files changed, 16 insertions(+), 4 deletions(-) create mode 100644 include/exec/user/guest-base.h diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h index ad824fee52..78d258af44 100644 --- a/include/exec/cpu-all.h +++ b/include/exec/cpu-all.h @@ -84,11 +84,8 @@ #if defined(CONFIG_USER_ONLY) #include "exec/user/abitypes.h" +#include "exec/user/guest-base.h" -/* On some host systems the guest address space is reserved on the host. - * This allows the guest address space to be offset to a convenient location. - */ -extern uintptr_t guest_base; extern bool have_guest_base; /* diff --git a/include/exec/user/guest-base.h b/include/exec/user/guest-base.h new file mode 100644 index 0000000000..afe2ab7fbb --- /dev/null +++ b/include/exec/user/guest-base.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Declaration of guest_base. + * Copyright (c) 2003 Fabrice Bellard + */ + +#ifndef EXEC_USER_GUEST_BASE_H +#define EXEC_USER_GUEST_BASE_H + +extern uintptr_t guest_base; + +#endif diff --git a/tcg/tcg.c b/tcg/tcg.c index 4bd598c18b..6735d3f08d 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -63,6 +63,9 @@ #include "tcg/tcg-temp-internal.h" #include "tcg-internal.h" #include "accel/tcg/perf.h" +#ifdef CONFIG_USER_ONLY +#include "exec/user/guest-base.h" +#endif /* Forward declarations for functions declared in tcg-target.c.inc and used here. */