From patchwork Fri May 26 00:23:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 685907 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp643342wrt; Thu, 25 May 2023 17:26:37 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6LtDGmPeA/KyIHPxD2q2tHPoXXspYR/TQhbYSR8Q8wksnU+iTvn/i4+iGsh7SsmxV90Kjl X-Received: by 2002:ac8:4e88:0:b0:3f8:3d7:449f with SMTP id 8-20020ac84e88000000b003f803d7449fmr272125qtp.39.1685060796997; Thu, 25 May 2023 17:26:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685060796; cv=none; d=google.com; s=arc-20160816; b=C84qplXiGfdFEG6vCnWkLgccB/iiVi/kQNBh3qv42rqaBQpOM00OuHOHR7ddTEM2D4 2zIrfKQfnsKc4ZtaoJqMM75X1VT287qhFJQWJ98wQZiK+9cvKqq+elnUSbS/c95s+xkV ct3WGFSl7Ktejir8gRbwI7WV8Hf+MQ+Kc/HH29hgdwr+hC3RAkNuI8ss5HzkHORVHjim FWTsbFr4F5ILLxEmj3OWWzL86DdcZCDDcMPKQ12ufkq5N4nT8Qt7T7qs3WBGvieIuHrC h8rn5BwUNGrnp701bnVB0g+8uFQOy4H86fXJ05BxJHrJueqwjPsgtf9YH3zfhVnP/KIk uE0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=o6cJbzSvYvQOTGDtz8/CCuAgm74mnMDf42v4W8C12Tg=; b=Pw0vpdrnRGqpaG7Nr8WqSeGCQTPwRyE+xqGI0ntYet4eVtS6EFIS5mimrbV/4GyhMH UsCrDJREINpaiypk6ismYkBE3kfqKIKQa0o38N2GdL9VO5Ndu/BAJcDuJxklw6onuETY f6jyUhKbKt2qzBg6s+qtWDv2jsWDpSoWj3oURmG4JcR+3OhIx8iGgLhFtz8/8Qr5jqY/ +p1KDVBtZVWohRIxgckq7FMSAnJ7VgxVtj1hBqOOZ8+d5XyoUzPA37Yz6qvYgZCAqZTE 4++fsV8BA9iMGQNg4dzqs+IanF/QsmM3AQNdpq1WI9fXSBJGbfA1m69J0qbX7Th9hYAc 957Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bjI4a7DZ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x8-20020ac85f08000000b003f6c46a79c5si953917qta.454.2023.05.25.17.26.36 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 May 2023 17:26:36 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bjI4a7DZ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2LF8-0002Vl-OF; Thu, 25 May 2023 20:23:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2LF7-0002UR-AB for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:41 -0400 Received: from mail-pj1-x1030.google.com ([2607:f8b0:4864:20::1030]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q2LF3-00028L-K8 for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:40 -0400 Received: by mail-pj1-x1030.google.com with SMTP id 98e67ed59e1d1-25376483f66so240999a91.0 for ; Thu, 25 May 2023 17:23:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1685060616; x=1687652616; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=o6cJbzSvYvQOTGDtz8/CCuAgm74mnMDf42v4W8C12Tg=; b=bjI4a7DZKzaLg63y9jmQ1vqRt/PVvjdAvq1V3UUxVgb8h1HorQmQl0fKN2e69v+QwM CW2azV3Zz6wHhwXWL5julyJ1uo5u98IWahjqcjlr7pbcQKNfTvfgTAV5vDp3fqQPT8cU Il8aHv5juxFNOkN8Cz/74kcUMHFSLvUgaMuxpBelbqPVqf2qQqcX7rRddw9PZXR/947E Iu4R7jikTOHbkthUSbp5MCfiPHpWgbsIliqrkF2Ph4x/q9VnZOXJg5hyO2lgr4mwmkFQ kswjhWLNykBwkRrotYQw/bl1wsNxt1+ACkCi13Sk3GaKAc3TS0+6jc8BoAc1n2+g6y6s oN3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685060616; x=1687652616; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=o6cJbzSvYvQOTGDtz8/CCuAgm74mnMDf42v4W8C12Tg=; b=At7ic463ox8uRVY0Oc8Gn/kudKQ1iE2y/jvlJwc2GwDl9eChuJ2VIVqALknaoPymKa DKPg+Gq0vzv+A9jYHXSot37cY0DNRRgZau1xaTIEai0/fL0UUqutHWPv7oP6ngO7CUVj /9Hrj4LeXlrlTB5y/Bbaa8cShf196nHxAmb/IC7W9xnlCh720st6N6UfbEjOrY2sNtWx dDyadqyzOmkb/BEWneKvYofnuhFHlHd8pY1rWo+I3Hqt7Fpkded8Ib8Q3/YhX+VgQOnp aYYze61doFksrKwaZcd1l5EVb3bvCVdZf+nRPxfU6utOk55LFlv1xZlNq6AKJl0c5BWP 0RyQ== X-Gm-Message-State: AC+VfDwl0gb7uXhGypsdZKy7fvphnSGFGKYXkCpQMojOwx2sHemc1SsG 87bA7FzdIV3xfhaaZTGq/Vc//h/8l9rG7ShXLMA= X-Received: by 2002:a17:90a:f996:b0:250:faff:e201 with SMTP id cq22-20020a17090af99600b00250faffe201mr442321pjb.36.1685060616262; Thu, 25 May 2023 17:23:36 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:7ac5:31cc:3997:3a16]) by smtp.gmail.com with ESMTPSA id s188-20020a635ec5000000b0053491d92b65sm1675593pgb.84.2023.05.25.17.23.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 May 2023 17:23:35 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH v4 01/16] tcg: Fix register move type in tcg_out_ld_helper_ret Date: Thu, 25 May 2023 17:23:19 -0700 Message-Id: <20230526002334.1760495-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526002334.1760495-1-richard.henderson@linaro.org> References: <20230526002334.1760495-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1030; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1030.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org The first move was incorrectly using TCG_TYPE_I32 while the second move was correctly using TCG_TYPE_REG. This prevents a 64-bit host from moving all 128-bits of the return value. Fixes: ebebea53ef8 ("tcg: Support TCG_TYPE_I128 in tcg_out_{ld,st}_helper_{args,ret}") Signed-off-by: Richard Henderson Reviewed-by: Peter Maydell --- tcg/tcg.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tcg/tcg.c b/tcg/tcg.c index ac30d484f5..2352ca4ade 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -5736,8 +5736,8 @@ static void tcg_out_ld_helper_ret(TCGContext *s, const TCGLabelQemuLdst *ldst, mov[0].dst = ldst->datalo_reg; mov[0].src = tcg_target_call_oarg_reg(TCG_CALL_RET_NORMAL, HOST_BIG_ENDIAN); - mov[0].dst_type = TCG_TYPE_I32; - mov[0].src_type = TCG_TYPE_I32; + mov[0].dst_type = TCG_TYPE_REG; + mov[0].src_type = TCG_TYPE_REG; mov[0].src_ext = TCG_TARGET_REG_BITS == 32 ? MO_32 : MO_64; mov[1].dst = ldst->datahi_reg; From patchwork Fri May 26 00:23:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 685898 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp642747wrt; Thu, 25 May 2023 17:24:59 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5Oqe516QOYNfKuiXo2ZEOX8gXMMMKaBKaEoyCkj7iMrO74h97oTwJFooTmx+S8oN2Pqilp X-Received: by 2002:ad4:5dee:0:b0:61b:6c88:4bd3 with SMTP id jn14-20020ad45dee000000b0061b6c884bd3mr319450qvb.47.1685060699475; Thu, 25 May 2023 17:24:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685060699; cv=none; d=google.com; s=arc-20160816; b=l37HHgQtk27Q0pbtHUYZMIxv2fzEOU10felplMtt256PZk6XxuubfDQspyjSpj1tlN zpkBYF1Q/mnLtKtE+ZCHB03hyHbMHrRssbsAECznhAi3HLrP4XD8O1fOJ3LVMh9080jS mTUNUkn6jd/4JNs1DCnMKGU01kgGvqx+KGlhWd0a8AkudfbTTR+Fkox+cBvER/6ik1ad lkI+HsVrwIjvxUa4dTIRz2T5o0mkqQN8gQUPsOEY3oInpOfdhCIzANg+2iVeRe8hIJJy DI6rQArdXzLZAz5MH0+ExrAeF/NuT0mLG/bWnXD8vDKnKUZFunVUqWc27PFffjsb8O8r zeww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=grxtGqzdDW3pij35DCIXzbwm/3Rbmj2aBugCYZSOGiE=; b=LSxAb93PnXTLMmnf+cHXT6if3QH64XUyxs6vs+dvdkjRZGoNZZPE8TW9u/3HNM9T3s tSS0iwfPLer2WOucbszPnq1tTpy6wAvFKdukjxkzvEYp2OsIA8BgCqR0jMHMArE0JthB h6A8ilyWIfz2a/TjQbuevWd0uOO4Rr5AUDgZQUPBfwbuOWMg+PV98qv6iZ1jIhrTwJx6 xXg8tc92C8Zs6/L01L6mfuzVE0qy829r34faGAhJWnsvs11fKRW7/VsSIeluPApSDYY+ /k69rYkaXXzyytdqdFaKUvrvM6GSReMOBSDwMB4seai/EB43343KEYQ1W9Pz7dF47csq Z3Aw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tvlUdHzC; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f13-20020a0562141d2d00b00571bfc39612si972897qvd.346.2023.05.25.17.24.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 May 2023 17:24:59 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tvlUdHzC; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2LF9-0002We-EK; Thu, 25 May 2023 20:23:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2LF7-0002UT-C8 for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:41 -0400 Received: from mail-pg1-x52d.google.com ([2607:f8b0:4864:20::52d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q2LF4-00028Z-Mr for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:41 -0400 Received: by mail-pg1-x52d.google.com with SMTP id 41be03b00d2f7-5144a9c11c7so141619a12.2 for ; Thu, 25 May 2023 17:23:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1685060617; x=1687652617; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=grxtGqzdDW3pij35DCIXzbwm/3Rbmj2aBugCYZSOGiE=; b=tvlUdHzCVoKaXX6W+KUro9LXb56GERYrvgfmOtyN/SPD7y5pi7t8KaitkKm+bDNYEl cj+DSCtXDMW70tVMoAEtEEmWQH960aPnid+TuoWwTdFcVc17ZzX36ELy+MYogEzb6/V4 Z2YrhSB1uUl++RDQmLl9es9wb00sG1vcDroCSx1mayVdcvi00uAGqXYiBiaL105k1xLz Je8JxKVMBuvegTF+M2I9xRPcga1yp3xanjq9QppKmBIjkDu1bICFJPYrKuPLogyRJwqa iLXCX6j0r+zTygUFxem7kCzy5niP0P7r67R33HAgbryhkiLXt+bMrH7pkWNUcSjmTxbp SO+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685060617; x=1687652617; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=grxtGqzdDW3pij35DCIXzbwm/3Rbmj2aBugCYZSOGiE=; b=P9vaOXFMEbrH6gYVKVGDVEYh2izrdtvXbSINAIThY7meheeHFNJl25+b/w94ss1Rmo LbXPlWFOu7FMmCvBruFU2A5OvEZ37QAr6pSRCUo4MBN93lVXyYC1cIK+vG3WhjMmJ9ns rsTKGlaj8bKzGc4T34bOHnbGugeDoILFHgRMMwYdsHvyhlq730OaZ0pBo22H7v6ufDHx gK8PnW4uabcRk4FFwXmjQA+qM7K0ASSbD39GQapbtD8nz4fPwcArqJZEBZwrf0Qs5/6I WBG0ECodk+bp/ZxfOkzi1ZBw7nZuMPmPXx9XYgPKjSgW3J44WcFDrIhYj2QJponjyDdX h5EQ== X-Gm-Message-State: AC+VfDxpO9MpFE1NCkWFioNZw5rzVV1NKNk6VL/cR5UW5MvDdCzJTfrz kGz+fgm4N6JeWHufgmf9b9Y8XWz7XlvmNljQjtg= X-Received: by 2002:a05:6a20:7287:b0:10c:3cf3:ef7e with SMTP id o7-20020a056a20728700b0010c3cf3ef7emr14678608pzk.42.1685060617076; Thu, 25 May 2023 17:23:37 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:7ac5:31cc:3997:3a16]) by smtp.gmail.com with ESMTPSA id s188-20020a635ec5000000b0053491d92b65sm1675593pgb.84.2023.05.25.17.23.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 May 2023 17:23:36 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH v4 02/16] accel/tcg: Fix check for page writeability in load_atomic16_or_exit Date: Thu, 25 May 2023 17:23:20 -0700 Message-Id: <20230526002334.1760495-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526002334.1760495-1-richard.henderson@linaro.org> References: <20230526002334.1760495-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52d; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org PAGE_WRITE is current writability, as modified by TB protection; PAGE_WRITE_ORG is the original page writability. Fixes: cdfac37be0d ("accel/tcg: Honor atomicity of loads") Signed-off-by: Richard Henderson Reviewed-by: Peter Maydell --- accel/tcg/ldst_atomicity.c.inc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/accel/tcg/ldst_atomicity.c.inc b/accel/tcg/ldst_atomicity.c.inc index 0f6b3f8ab6..57163f5ca2 100644 --- a/accel/tcg/ldst_atomicity.c.inc +++ b/accel/tcg/ldst_atomicity.c.inc @@ -191,7 +191,7 @@ static Int128 load_atomic16_or_exit(CPUArchState *env, uintptr_t ra, void *pv) * another process, because the fallback start_exclusive solution * provides no protection across processes. */ - if (!page_check_range(h2g(p), 16, PAGE_WRITE)) { + if (!page_check_range(h2g(p), 16, PAGE_WRITE_ORG)) { return *p; } #endif From patchwork Fri May 26 00:23:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 685906 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp643212wrt; Thu, 25 May 2023 17:26:17 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ45joB4wwE00Sb6yE7AdpbU1s8vqdfdZJ5SAE45aBD8xd9x5p5mswCm7ZJeWbDj04TVzdo9 X-Received: by 2002:a05:622a:1107:b0:3f5:3b4b:b2bb with SMTP id e7-20020a05622a110700b003f53b4bb2bbmr207164qty.41.1685060777217; Thu, 25 May 2023 17:26:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685060777; cv=none; d=google.com; s=arc-20160816; b=OLp6BnS5tJ9MIAFVU7o0ZpJmWYeml8jOIfNdK6XOJuBrl7b4v73hmZ0kkPPZsc6i9T Rp0YtTiMq7Kfu7py6TdloSbiIG0u6c5igg0nCMHZ5MNzfrkLaWd5UV7sDGkBFH5L5OwY tog5bfWsZnlCpg633ZsZR962OZjiG7g63emFkA+6d3KePERr8ahlOaelVqgnT5z+/1DD cNY6anNppIaF+ytn5eTNRepmi7s1lf0NeHAIT6/mTzu0+AyhOGSbKKKz4vtcu5A3deQN PG/8T5KmUEt5p+jKhDMfumhPMAUM5DKnRqDrozROfB5UUUo+PyRXModrvsFC3JrwJTjK XWhg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=gjqymQC8hq32i71MwJu3fBXWpYLB+n1ecpePYVgdZ/4=; b=RY40PBuFBWpqxW79JqpmllYAUyUbTZ6i72OW//IfPUczO0LGokX6SUZINEbBSeBJz+ zB41LZFPQWY5VLy6dxBki+Hyo+pMJsVqclcSpylbv7A2CVIvZ5g5mbeaPzeIF9SMwkpl qBhaMktLVYQK8B8O1T1PEuAcWHWuENDqS+bUOyMwLLv+DWAPDcGW2jWvqQ/1B8PKODIM ICYt8cY/TCjY+d0Cbjj0ZP12ncWP3dIUE74HFOeFQPsSQ95h+sFxpYqA53nZahtgLZzH XeuhVFHT3Km5Yhnn7UwJQz34f8SdA2OiZS03vd+fH3Xf/MTsuIRtOt1JpG6J9w34oOPr yRSA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=nGoxVflE; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 18-20020ac85952000000b003f3908326dasi946756qtz.259.2023.05.25.17.26.17 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 May 2023 17:26:17 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=nGoxVflE; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2LFB-0002X6-0M; Thu, 25 May 2023 20:23:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2LF8-0002W1-Ow for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:42 -0400 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q2LF5-00028f-83 for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:42 -0400 Received: by mail-pg1-x530.google.com with SMTP id 41be03b00d2f7-53063897412so157199a12.0 for ; Thu, 25 May 2023 17:23:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1685060618; x=1687652618; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gjqymQC8hq32i71MwJu3fBXWpYLB+n1ecpePYVgdZ/4=; b=nGoxVflEL8AOyv0piJWRnwYflCRKLMbwnBRXIL9/o5/DC+9/OCEzNhcUxda+sUftxz aDlJ9la1gW2q4IE/Y+8ewO78UATICfX0/WJUje1HphqYVtt01bU79Fh0A4hbGd2OvqZt i4mcIDsVlcTQRvuCkfLICJuhx3avfDx/re7naiNkw/P4eL/NBDiFMi6IjH8UObmruJy9 4kZk357tcW89+/KeuHkqaKdIUDjpZ8YWxRNHtIWD8G96QSTkek/5j6+huf2NYJ1/OZss KvP4KNZr4Fia6AV7g4crSmRGNVEmnwin1whRXOym3x89zcIaaPWuEd/bmRs3r1QeYKrO NcwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685060618; x=1687652618; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gjqymQC8hq32i71MwJu3fBXWpYLB+n1ecpePYVgdZ/4=; b=HED32l0SjgKnT/u7ZGdIHKzxvnL7X1tuq1i/dpC7mF0+sKNSyBMiK2IdlHO87m/mzd wldHvhC6VkQowYk2MjxHwYEJti+52DgkrR1WMEceL62mxsp3FX6teYGhB7oRggAGs7dY krm8tVb+iKGO3yePPalG3DgfpCNojrXwBOe6RVJWBwlTzk5bwFuQzItBTYUTGgl9ZX5V qfQxYh/a3/Vc2yJYbi/PgxzqxSUOEoG+zN++sBBqIjKKoz6hG7ieGxp2ub4KBE+T84xR tmrsL9QaH8Oon6HH7lOQ3N8KhsCzX0tRSmtNMvT5oW2bNDFqi63TXhyYjd41vg5T0S25 FI7w== X-Gm-Message-State: AC+VfDxJBocIhf6ku3RsaBypYtCJhbpQxMkqOkWTlCCp/W7TcQcs4WP8 tIVhlzG+Uunl18omoRsRwuzbyKITzQyczz/Zh9w= X-Received: by 2002:a05:6a20:c1a0:b0:10c:4e7f:1a5a with SMTP id bg32-20020a056a20c1a000b0010c4e7f1a5amr12752187pzb.49.1685060617971; Thu, 25 May 2023 17:23:37 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:7ac5:31cc:3997:3a16]) by smtp.gmail.com with ESMTPSA id s188-20020a635ec5000000b0053491d92b65sm1675593pgb.84.2023.05.25.17.23.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 May 2023 17:23:37 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH v4 03/16] meson: Split test for __int128_t type from __int128_t arithmetic Date: Thu, 25 May 2023 17:23:21 -0700 Message-Id: <20230526002334.1760495-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526002334.1760495-1-richard.henderson@linaro.org> References: <20230526002334.1760495-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::530; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x530.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Older versions of clang have missing runtime functions for arithmetic with -fsanitize=undefined (see 464e3671f9d5c), so we cannot use __int128_t for implementing Int128. But __int128_t is present, data movement works, and can be use for atomic128. Probe for both CONFIG_INT128_TYPE and CONFIG_INT128, adjust qemu/int128.h to define Int128Alias if CONFIG_INT128_TYPE, and adjust the meson probe for atomics to use has_int128_type. Signed-off-by: Richard Henderson Reviewed-by: Peter Maydell --- meson.build | 15 ++++++++++----- include/qemu/int128.h | 4 ++-- 2 files changed, 12 insertions(+), 7 deletions(-) diff --git a/meson.build b/meson.build index 78890f0155..b5d923ad2f 100644 --- a/meson.build +++ b/meson.build @@ -2545,7 +2545,13 @@ config_host_data.set('CONFIG_ATOMIC64', cc.links(''' return 0; }''')) -has_int128 = cc.links(''' +has_int128_type = cc.compiles(''' + __int128_t a; + __uint128_t b; + int main(void) { b = a; }''') +config_host_data.set('CONFIG_INT128_TYPE', has_int128_type) + +has_int128 = has_int128_type and cc.links(''' __int128_t a; __uint128_t b; int main (void) { @@ -2554,10 +2560,9 @@ has_int128 = cc.links(''' a = a * a; return 0; }''') - config_host_data.set('CONFIG_INT128', has_int128) -if has_int128 +if has_int128_type # "do we have 128-bit atomics which are handled inline and specifically not # via libatomic". The reason we can't use libatomic is documented in the # comment starting "GCC is a house divided" in include/qemu/atomic128.h. @@ -2566,7 +2571,7 @@ if has_int128 # __alignof(unsigned __int128) for the host. atomic_test_128 = ''' int main(int ac, char **av) { - unsigned __int128 *p = __builtin_assume_aligned(av[ac - 1], 16); + __uint128_t *p = __builtin_assume_aligned(av[ac - 1], 16); p[1] = __atomic_load_n(&p[0], __ATOMIC_RELAXED); __atomic_store_n(&p[2], p[3], __ATOMIC_RELAXED); __atomic_compare_exchange_n(&p[4], &p[5], p[6], 0, __ATOMIC_RELAXED, __ATOMIC_RELAXED); @@ -2588,7 +2593,7 @@ if has_int128 config_host_data.set('CONFIG_CMPXCHG128', cc.links(''' int main(void) { - unsigned __int128 x = 0, y = 0; + __uint128_t x = 0, y = 0; __sync_val_compare_and_swap_16(&x, y, x); return 0; } diff --git a/include/qemu/int128.h b/include/qemu/int128.h index 9e46cfaefc..73624e8be7 100644 --- a/include/qemu/int128.h +++ b/include/qemu/int128.h @@ -481,7 +481,7 @@ static inline void bswap128s(Int128 *s) * a possible structure and the native types. Ease parameter passing * via use of the transparent union extension. */ -#ifdef CONFIG_INT128 +#ifdef CONFIG_INT128_TYPE typedef union { __uint128_t u; __int128_t i; @@ -489,6 +489,6 @@ typedef union { } Int128Alias __attribute__((transparent_union)); #else typedef Int128 Int128Alias; -#endif /* CONFIG_INT128 */ +#endif /* CONFIG_INT128_TYPE */ #endif /* INT128_H */ From patchwork Fri May 26 00:23:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 685901 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp642924wrt; Thu, 25 May 2023 17:25:26 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7xkuONkMbiF1l3CQHfFkUSWT/9eVSHt/8jhZBgQi/vsAGq+2pBm4GOGV7WlkACprXlg+uT X-Received: by 2002:a05:6214:20e3:b0:5c8:403a:22f8 with SMTP id 3-20020a05621420e300b005c8403a22f8mr384131qvk.5.1685060726283; Thu, 25 May 2023 17:25:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685060726; cv=none; d=google.com; s=arc-20160816; b=MYoMH5gOQ6nPJhr9ET/Ff1cDLjH5mwVy3XoUu9/xcx/IPQgx7YaSHsi3JOxhxzkleK 8uyizBebJ83P+5wbBQDIBieXXLmt9ze1q6aU7f5NJQham3gl3l8zC4/9qj1ezmzgtDtZ b1zWjKtqXeeM+CW+ZLC0ume1oRSAKzx7t8vC2W5ohOrz9Jmh8t+SrnearFR7dNlPY+NL JAmOgenNT7iUUQyymbTh222z1XLgOJIltBdnE1Vu/WzIgmMSybwEtIFxU4+YYQ2bYMWD Jcss8+pjm2yb9PO+gW7+DvBRoAKK11WoXBZjDQCiufdo6nkPyOnOpKKB8J5wbBsQNDUD hhXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=pGac7UlZVA4AZdD7xsv1J+iiIMdZOvspUsCJpDvtWoE=; b=eyWc7zYR6lLlIqyjOW4oPku47PRAzSmlzTtYsIuxqgwlk4To7hpujRFI6a9FxdiR0p J9uaNG8Fy/6FMabFESaQ2mC8knCbLs4S+4AbppW7UtCikFQ3HN4sfFjyBkERO9J2wZRW SI+3AHc6GfFK561hhWkxnBRKyE8UIcr8/uJONgtJKY+pfGDdZRDxdg96sIodxxJL/lOb 7qruYn+8GXPeXkTjv1C8G2Ey9nmOwRhN4bPV0RIe4JYaEzlOGKqss3nqhcNTAjcuWmAm jTA0ty3gVrDDdS0q/eY0CEbNTdjTKWKyG8WhVmTbOQVXFi1I4bsiDTSX4pUgEK70AOjS SYaA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ZSWlFJTO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d1-20020a05620a204100b0074a0dd62f45si947492qka.757.2023.05.25.17.25.26 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 May 2023 17:25:26 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ZSWlFJTO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2LFB-0002XU-Fz; Thu, 25 May 2023 20:23:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2LF8-0002W3-RP for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:43 -0400 Received: from mail-pg1-x531.google.com ([2607:f8b0:4864:20::531]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q2LF6-00028u-38 for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:42 -0400 Received: by mail-pg1-x531.google.com with SMTP id 41be03b00d2f7-53063897412so157208a12.0 for ; Thu, 25 May 2023 17:23:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1685060619; x=1687652619; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pGac7UlZVA4AZdD7xsv1J+iiIMdZOvspUsCJpDvtWoE=; b=ZSWlFJTOFcRK1Sy5MZLSOtwj/l+plhdh/pw7euLATZrRd6SntatWa0e4bgShZjflCZ JrQKiVszYbJNmb6IZqo4khgtb+wFyVvHxQEbsP444yuVMd3FQkWBl9mP8xDpWqPKgZGL mzG+D5Dq99qwquplogQr3EUS2qBC05y8MIe3mDGHDSWY6nSs+MjukpHo+bAt+ta7esaV O6XB65JcOsFlgpCY5BRmxIAS2xyDVn1XDVqy3VpMHYcgl1BJ/qtFpqSI3mQzzowUMQ8I vexjeLdh9Sm/7YaHoDEGP/MNik1e+azX8JCxTm9y/cFzaX/TJy2qorijXglQHvr8+WEu bqxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685060619; x=1687652619; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pGac7UlZVA4AZdD7xsv1J+iiIMdZOvspUsCJpDvtWoE=; b=Y5e9ff6ZFXJICJcWy92mqik30t68eUqnoPXcJuo0X7BySEUkUqafi+10HdLFUglQN+ 2yuFyWPPfPjm2j41peuFQRlBQ78W3+bq9hK6klTarnCgZK2zaFxhnXpSDAValDSkCizK 6hT6bUdQeQ58OR1hOuQiAweZdeJt3/uM4WU2OC/2pz/kFxL0sdjfp4WCcxbSKZWQ8nDE O7SqbTDP2ja57JBdkzTBvRwr30H+y3Jj/xpVvrI/3NZnaYWEpX7hYSen98wGCX2xzEYm roi2NuB3KoG8h9J08Azr3TfbQjLcfzz6MZEbrRxPMfyHyTPR4jLecdCR3XbckrXa6Zv4 z3fw== X-Gm-Message-State: AC+VfDw17E3LcKDNB6d2EVwNVMcXRpB3PC9EizQLMLlw05EhuK8Nj7Hd 3egFtJHYowLMOGKGp+NXyy/sTgvMpc8+Y9JSE7I= X-Received: by 2002:a05:6a20:430b:b0:10c:3535:162f with SMTP id h11-20020a056a20430b00b0010c3535162fmr123866pzk.0.1685060618826; Thu, 25 May 2023 17:23:38 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:7ac5:31cc:3997:3a16]) by smtp.gmail.com with ESMTPSA id s188-20020a635ec5000000b0053491d92b65sm1675593pgb.84.2023.05.25.17.23.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 May 2023 17:23:38 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PATCH v4 04/16] qemu/atomic128: Add x86_64 atomic128-ldst.h Date: Thu, 25 May 2023 17:23:22 -0700 Message-Id: <20230526002334.1760495-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526002334.1760495-1-richard.henderson@linaro.org> References: <20230526002334.1760495-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::531; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x531.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org With CPUINFO_ATOMIC_VMOVDQA, we can perform proper atomic load/store without cmpxchg16b. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- host/include/x86_64/host/atomic128-ldst.h | 68 +++++++++++++++++++++++ 1 file changed, 68 insertions(+) create mode 100644 host/include/x86_64/host/atomic128-ldst.h diff --git a/host/include/x86_64/host/atomic128-ldst.h b/host/include/x86_64/host/atomic128-ldst.h new file mode 100644 index 0000000000..adc9332f91 --- /dev/null +++ b/host/include/x86_64/host/atomic128-ldst.h @@ -0,0 +1,68 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Load/store for 128-bit atomic operations, x86_64 version. + * + * Copyright (C) 2023 Linaro, Ltd. + * + * See docs/devel/atomics.rst for discussion about the guarantees each + * atomic primitive is meant to provide. + */ + +#ifndef AARCH64_ATOMIC128_LDST_H +#define AARCH64_ATOMIC128_LDST_H + +#ifdef CONFIG_INT128_TYPE +#include "host/cpuinfo.h" +#include "tcg/debug-assert.h" + +/* + * Through clang 16, with -mcx16, __atomic_load_n is incorrectly + * expanded to a read-write operation: lock cmpxchg16b. + */ + +#define HAVE_ATOMIC128_RO likely(cpuinfo & CPUINFO_ATOMIC_VMOVDQA) +#define HAVE_ATOMIC128_RW 1 + +static inline Int128 atomic16_read_ro(const Int128 *ptr) +{ + Int128Alias r; + + tcg_debug_assert(HAVE_ATOMIC128_RO); + asm("vmovdqa %1, %0" : "=x" (r.i) : "m" (*ptr)); + + return r.s; +} + +static inline Int128 atomic16_read_rw(Int128 *ptr) +{ + __int128_t *ptr_align = __builtin_assume_aligned(ptr, 16); + Int128Alias r; + + if (HAVE_ATOMIC128_RO) { + asm("vmovdqa %1, %0" : "=x" (r.i) : "m" (*ptr_align)); + } else { + r.i = __sync_val_compare_and_swap_16(ptr_align, 0, 0); + } + return r.s; +} + +static inline void atomic16_set(Int128 *ptr, Int128 val) +{ + __int128_t *ptr_align = __builtin_assume_aligned(ptr, 16); + Int128Alias new = { .s = val }; + + if (HAVE_ATOMIC128_RO) { + asm("vmovdqa %1, %0" : "=m"(*ptr_align) : "x" (new.i)); + } else { + __int128_t old; + do { + old = *ptr_align; + } while (!__sync_bool_compare_and_swap_16(ptr_align, old, new.i)); + } +} +#else +/* Provide QEMU_ERROR stubs. */ +#include "host/include/generic/host/atomic128-ldst.h" +#endif + +#endif /* AARCH64_ATOMIC128_LDST_H */ From patchwork Fri May 26 00:23:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 685903 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp643048wrt; Thu, 25 May 2023 17:25:52 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4UFy9tYps/GGqefL1LFI56JamHuRbYO5ELVJhEaSA82kC6GZD5AuNDCfPXRaBawY837AUc X-Received: by 2002:a05:6214:2245:b0:616:7a62:a7d0 with SMTP id c5-20020a056214224500b006167a62a7d0mr516215qvc.16.1685060751909; Thu, 25 May 2023 17:25:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685060751; cv=none; d=google.com; s=arc-20160816; b=A/qV8BCBx2KqSq5NlTyR1G9n84I+DAwCSeSHmO+tOCRk6rW0i3pIfh0YLqyusDkOFZ JpN5Lb3oMVfsgQFsH2m0MEXWCt9iWViY1IINOvNNeCcYy9qvbhKjC92nOYHJQYRcJSPE 2EOHw2N+kmByrbjLgEqD+XhdVCitCF0zulq4m5b/uO2hTgPV6ypQL30uHiYqYRbdlooC i64f2yi+V2JLtXGirpOZ3ETPyJbamrqlFsrTf03WVVD3sUMvRjr+epZx3+R5U0POxgO3 i9mWIfXcMHUVPSwfM8pMWaT2N1Ft/IBRgTaNAV3IEfZ51H6JkFizpBc9ztsMRrfOVdEn vN3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=rWWtKXFzofB3xieOiujeZSnQEAfLPZis5BTEhJ0EIUA=; b=oVW0mheU1ophVw8JcQr72iRbsKl0URTuWJGEujBJx4VXEA/ul39d1flKnazHX4RETw gM0qUqVDF9NUSXExhsAtvyNy78wbOt1J9BL3OQLYOzHe4f+BfGhelSDIOg4BkXzMc9nv bKFUFvPsw8HOmhOvrXFJ2seBXbnpgHNYahO0COqhQ50yZkHHoW0nxB04zL3/1awUYz3e gJO92jGClk/IuH0j6UUhosIBzCiHmKUKxZPR4AiJigOxHCyt3xB35nh/KJmvckVwAN0g 32yO54or8gH7cR2cYpPlL9SFzWlYVW16PghRPl23+u3xMYsQ6mq/zVKT2BiMsWrK+XVj c23w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WpcgsvbI; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id gf13-20020a056214250d00b00571bb7c3c01si955259qvb.337.2023.05.25.17.25.51 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 May 2023 17:25:51 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WpcgsvbI; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2LFD-0002Z7-Fp; Thu, 25 May 2023 20:23:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2LFB-0002XX-Mp for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:45 -0400 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q2LF7-00029G-En for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:44 -0400 Received: by mail-pg1-x530.google.com with SMTP id 41be03b00d2f7-53f448cde66so135802a12.1 for ; Thu, 25 May 2023 17:23:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1685060620; x=1687652620; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rWWtKXFzofB3xieOiujeZSnQEAfLPZis5BTEhJ0EIUA=; b=WpcgsvbI6E91nMBpFgp9dJBi8r1ahWvxEf4TjB3sMrzMJ2YaKbpJ1BK5RDDEbNkIgo xBoL0k4e2GpHL2OC0I8f2t0j/n46m68gsD5yzsqfWPiOi0ZRviBJStFDPkUOtTXt2NTy 9tydRPfAogLC0xx3/BDGNm3FOy8COV00j0qAt3A5YVn979RLBAj1hcdzsYATcjv5hXnN RsQLYJIHuG35e16NgdgC26rF9Xrg8EoGG8w602oKaVs6R89gdF5a8YV8+tfPb36SJcJa SRPZEikkaFPYVc/HHqkKCIgP9ScyJ6xRcVyYBtoukInhcXhHky6XZDW+PnnyqsCiVVJO 4m/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685060620; x=1687652620; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rWWtKXFzofB3xieOiujeZSnQEAfLPZis5BTEhJ0EIUA=; b=H9LAh77JtHDYDA+Z5k1gRs3Spt+L5U+U+9ziIUr/dpFeJXnCBMtVqS2r+JbfaeQDkZ WG1OJyTfjG+ic0wOYLLW3+48Y124a4T/yzaKrwHbD1SKPIRAa/4RVVGN2JCUhSe+2bWR 7YbUvbhjorQJu8svP8v+1XSMip6vnztqZ4h5CJ03OE96Yfh6JIYMGofW1MX5U+jgZqGB NJU86ksNtL3E0WnJj+doMbHNwkwrHvQDYHYl62e8eEoVvqgcW/fSlU2f9qi+h91eVU8F 6xpVI4qWnpDAwwbNtlQ4mX8RQNnLrM+OVJOW5jj78ZWEB2ltxBCJP6Uyykn+vRIfwXJb 5o3g== X-Gm-Message-State: AC+VfDw0TyHb8h5IM4hcuC4xpNWYjJOtK4hWxZHQgAD8uZhgi1LLguMh CrPqVzwX794GieKaqk7FmbL1m7Um7L9+7M4RPmA= X-Received: by 2002:a05:6a21:3391:b0:10f:8798:9ff3 with SMTP id yy17-20020a056a21339100b0010f87989ff3mr28095pzb.8.1685060619672; Thu, 25 May 2023 17:23:39 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:7ac5:31cc:3997:3a16]) by smtp.gmail.com with ESMTPSA id s188-20020a635ec5000000b0053491d92b65sm1675593pgb.84.2023.05.25.17.23.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 May 2023 17:23:39 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH v4 05/16] tcg/i386: Support 128-bit load/store Date: Thu, 25 May 2023 17:23:23 -0700 Message-Id: <20230526002334.1760495-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526002334.1760495-1-richard.henderson@linaro.org> References: <20230526002334.1760495-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::530; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x530.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Peter Maydell --- tcg/i386/tcg-target.h | 4 +- tcg/i386/tcg-target.c.inc | 191 +++++++++++++++++++++++++++++++++++++- 2 files changed, 190 insertions(+), 5 deletions(-) diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index 0106946996..b167f1e8d6 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -118,7 +118,6 @@ typedef enum { #define have_avx1 (cpuinfo & CPUINFO_AVX1) #define have_avx2 (cpuinfo & CPUINFO_AVX2) #define have_movbe (cpuinfo & CPUINFO_MOVBE) -#define have_atomic16 (cpuinfo & CPUINFO_ATOMIC_VMOVDQA) /* * There are interesting instructions in AVX512, so long as we have AVX512VL, @@ -202,7 +201,8 @@ typedef enum { #define TCG_TARGET_HAS_qemu_st8_i32 1 #endif -#define TCG_TARGET_HAS_qemu_ldst_i128 0 +#define TCG_TARGET_HAS_qemu_ldst_i128 \ + (TCG_TARGET_REG_BITS == 64 && (cpuinfo & CPUINFO_ATOMIC_VMOVDQA)) /* We do not support older SSE systems, only beginning with AVX1. */ #define TCG_TARGET_HAS_v64 have_avx1 diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index bfe9d98b7e..ae54e5fbf3 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -91,6 +91,8 @@ static const int tcg_target_reg_alloc_order[] = { #endif }; +#define TCG_TMP_VEC TCG_REG_XMM5 + static const int tcg_target_call_iarg_regs[] = { #if TCG_TARGET_REG_BITS == 64 #if defined(_WIN64) @@ -319,6 +321,8 @@ static bool tcg_target_const_match(int64_t val, TCGType type, int ct) #define OPC_PCMPGTW (0x65 | P_EXT | P_DATA16) #define OPC_PCMPGTD (0x66 | P_EXT | P_DATA16) #define OPC_PCMPGTQ (0x37 | P_EXT38 | P_DATA16) +#define OPC_PEXTRD (0x16 | P_EXT3A | P_DATA16) +#define OPC_PINSRD (0x22 | P_EXT3A | P_DATA16) #define OPC_PMAXSB (0x3c | P_EXT38 | P_DATA16) #define OPC_PMAXSW (0xee | P_EXT | P_DATA16) #define OPC_PMAXSD (0x3d | P_EXT38 | P_DATA16) @@ -1753,7 +1757,21 @@ typedef struct { bool tcg_target_has_memory_bswap(MemOp memop) { - return have_movbe; + TCGAtomAlign aa; + + if (!have_movbe) { + return false; + } + if ((memop & MO_SIZE) < MO_128) { + return true; + } + + /* + * Reject 16-byte memop with 16-byte atomicity, i.e. VMOVDQA, + * but do allow a pair of 64-bit operations, i.e. MOVBEQ. + */ + aa = atom_and_align_for_opc(tcg_ctx, memop, MO_ATOM_IFALIGN, true); + return aa.atom < MO_128; } /* @@ -1781,6 +1799,30 @@ static const TCGLdstHelperParam ldst_helper_param = { static const TCGLdstHelperParam ldst_helper_param = { }; #endif +static void tcg_out_vec_to_pair(TCGContext *s, TCGType type, + TCGReg l, TCGReg h, TCGReg v) +{ + int rexw = type == TCG_TYPE_I32 ? 0 : P_REXW; + + /* vpmov{d,q} %v, %l */ + tcg_out_vex_modrm(s, OPC_MOVD_EyVy + rexw, v, 0, l); + /* vpextr{d,q} $1, %v, %h */ + tcg_out_vex_modrm(s, OPC_PEXTRD + rexw, v, 0, h); + tcg_out8(s, 1); +} + +static void tcg_out_pair_to_vec(TCGContext *s, TCGType type, + TCGReg v, TCGReg l, TCGReg h) +{ + int rexw = type == TCG_TYPE_I32 ? 0 : P_REXW; + + /* vmov{d,q} %l, %v */ + tcg_out_vex_modrm(s, OPC_MOVD_VyEy + rexw, v, 0, l); + /* vpinsr{d,q} $1, %h, %v, %v */ + tcg_out_vex_modrm(s, OPC_PINSRD + rexw, v, v, h); + tcg_out8(s, 1); +} + /* * Generate code for the slow path for a load at the end of block */ @@ -1870,6 +1912,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, { TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); + MemOp s_bits = opc & MO_SIZE; unsigned a_mask; #ifdef CONFIG_SOFTMMU @@ -1880,7 +1923,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, *h = x86_guest_base; #endif h->base = addrlo; - h->aa = atom_and_align_for_opc(s, opc, MO_ATOM_IFALIGN, false); + h->aa = atom_and_align_for_opc(s, opc, MO_ATOM_IFALIGN, s_bits == MO_128); a_mask = (1 << h->aa.align) - 1; #ifdef CONFIG_SOFTMMU @@ -1890,7 +1933,6 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, TCGType tlbtype = TCG_TYPE_I32; int trexw = 0, hrexw = 0, tlbrexw = 0; unsigned mem_index = get_mmuidx(oi); - unsigned s_bits = opc & MO_SIZE; unsigned s_mask = (1 << s_bits) - 1; int tlb_mask; @@ -2070,6 +2112,72 @@ static void tcg_out_qemu_ld_direct(TCGContext *s, TCGReg datalo, TCGReg datahi, h.base, h.index, 0, h.ofs + 4); } break; + + case MO_128: + tcg_debug_assert(TCG_TARGET_REG_BITS == 64); + + /* + * Without 16-byte atomicity, use integer regs. + * That is where we want the data, and it allows bswaps. + */ + if (h.aa.atom < MO_128) { + if (use_movbe) { + TCGReg t = datalo; + datalo = datahi; + datahi = t; + } + if (h.base == datalo || h.index == datalo) { + tcg_out_modrm_sib_offset(s, OPC_LEA + P_REXW, datahi, + h.base, h.index, 0, h.ofs); + tcg_out_modrm_offset(s, movop + P_REXW + h.seg, + datalo, datahi, 0); + tcg_out_modrm_offset(s, movop + P_REXW + h.seg, + datahi, datahi, 8); + } else { + tcg_out_modrm_sib_offset(s, movop + P_REXW + h.seg, datalo, + h.base, h.index, 0, h.ofs); + tcg_out_modrm_sib_offset(s, movop + P_REXW + h.seg, datahi, + h.base, h.index, 0, h.ofs + 8); + } + break; + } + + /* + * With 16-byte atomicity, a vector load is required. + * If we already have 16-byte alignment, then VMOVDQA always works. + * Else if VMOVDQU has atomicity with dynamic alignment, use that. + * Else use we require a runtime test for alignment for VMOVDQA; + * use VMOVDQU on the unaligned nonatomic path for simplicity. + */ + if (h.aa.align >= MO_128) { + tcg_out_vex_modrm_sib_offset(s, OPC_MOVDQA_VxWx + h.seg, + TCG_TMP_VEC, 0, + h.base, h.index, 0, h.ofs); + } else if (cpuinfo & CPUINFO_ATOMIC_VMOVDQU) { + tcg_out_vex_modrm_sib_offset(s, OPC_MOVDQU_VxWx + h.seg, + TCG_TMP_VEC, 0, + h.base, h.index, 0, h.ofs); + } else { + TCGLabel *l1 = gen_new_label(); + TCGLabel *l2 = gen_new_label(); + + tcg_out_testi(s, h.base, 15); + tcg_out_jxx(s, JCC_JNE, l1, true); + + tcg_out_vex_modrm_sib_offset(s, OPC_MOVDQA_VxWx + h.seg, + TCG_TMP_VEC, 0, + h.base, h.index, 0, h.ofs); + tcg_out_jxx(s, JCC_JMP, l2, true); + + tcg_out_label(s, l1); + tcg_out_vex_modrm_sib_offset(s, OPC_MOVDQU_VxWx + h.seg, + TCG_TMP_VEC, 0, + h.base, h.index, 0, h.ofs); + tcg_out_label(s, l2); + } + tcg_out_vec_to_pair(s, TCG_TYPE_I64, datalo, datahi, TCG_TMP_VEC); + break; + default: g_assert_not_reached(); } @@ -2140,6 +2248,63 @@ static void tcg_out_qemu_st_direct(TCGContext *s, TCGReg datalo, TCGReg datahi, h.base, h.index, 0, h.ofs + 4); } break; + + case MO_128: + tcg_debug_assert(TCG_TARGET_REG_BITS == 64); + + /* + * Without 16-byte atomicity, use integer regs. + * That is where we have the data, and it allows bswaps. + */ + if (h.aa.atom < MO_128) { + if (use_movbe) { + TCGReg t = datalo; + datalo = datahi; + datahi = t; + } + tcg_out_modrm_sib_offset(s, movop + P_REXW + h.seg, datalo, + h.base, h.index, 0, h.ofs); + tcg_out_modrm_sib_offset(s, movop + P_REXW + h.seg, datahi, + h.base, h.index, 0, h.ofs + 8); + break; + } + + /* + * With 16-byte atomicity, a vector store is required. + * If we already have 16-byte alignment, then VMOVDQA always works. + * Else if VMOVDQU has atomicity with dynamic alignment, use that. + * Else use we require a runtime test for alignment for VMOVDQA; + * use VMOVDQU on the unaligned nonatomic path for simplicity. + */ + tcg_out_pair_to_vec(s, TCG_TYPE_I64, TCG_TMP_VEC, datalo, datahi); + if (h.aa.align >= MO_128) { + tcg_out_vex_modrm_sib_offset(s, OPC_MOVDQA_WxVx + h.seg, + TCG_TMP_VEC, 0, + h.base, h.index, 0, h.ofs); + } else if (cpuinfo & CPUINFO_ATOMIC_VMOVDQU) { + tcg_out_vex_modrm_sib_offset(s, OPC_MOVDQU_WxVx + h.seg, + TCG_TMP_VEC, 0, + h.base, h.index, 0, h.ofs); + } else { + TCGLabel *l1 = gen_new_label(); + TCGLabel *l2 = gen_new_label(); + + tcg_out_testi(s, h.base, 15); + tcg_out_jxx(s, JCC_JNE, l1, true); + + tcg_out_vex_modrm_sib_offset(s, OPC_MOVDQA_WxVx + h.seg, + TCG_TMP_VEC, 0, + h.base, h.index, 0, h.ofs); + tcg_out_jxx(s, JCC_JMP, l2, true); + + tcg_out_label(s, l1); + tcg_out_vex_modrm_sib_offset(s, OPC_MOVDQU_WxVx + h.seg, + TCG_TMP_VEC, 0, + h.base, h.index, 0, h.ofs); + tcg_out_label(s, l2); + } + break; + default: g_assert_not_reached(); } @@ -2470,6 +2635,11 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, tcg_out_qemu_ld(s, a0, a1, a2, args[3], args[4], TCG_TYPE_I64); } break; + case INDEX_op_qemu_ld_a32_i128: + case INDEX_op_qemu_ld_a64_i128: + tcg_debug_assert(TCG_TARGET_REG_BITS == 64); + tcg_out_qemu_ld(s, a0, a1, a2, -1, args[3], TCG_TYPE_I128); + break; case INDEX_op_qemu_st_a64_i32: case INDEX_op_qemu_st8_a64_i32: @@ -2496,6 +2666,11 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, tcg_out_qemu_st(s, a0, a1, a2, args[3], args[4], TCG_TYPE_I64); } break; + case INDEX_op_qemu_st_a32_i128: + case INDEX_op_qemu_st_a64_i128: + tcg_debug_assert(TCG_TARGET_REG_BITS == 64); + tcg_out_qemu_st(s, a0, a1, a2, -1, args[3], TCG_TYPE_I128); + break; OP_32_64(mulu2): tcg_out_modrm(s, OPC_GRP3_Ev + rexw, EXT3_MUL, args[3]); @@ -3193,6 +3368,15 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_qemu_st_a64_i64: return TCG_TARGET_REG_BITS == 64 ? C_O0_I2(L, L) : C_O0_I4(L, L, L, L); + case INDEX_op_qemu_ld_a32_i128: + case INDEX_op_qemu_ld_a64_i128: + tcg_debug_assert(TCG_TARGET_REG_BITS == 64); + return C_O2_I1(r, r, L); + case INDEX_op_qemu_st_a32_i128: + case INDEX_op_qemu_st_a64_i128: + tcg_debug_assert(TCG_TARGET_REG_BITS == 64); + return C_O0_I3(L, L, L); + case INDEX_op_brcond2_i32: return C_O0_I4(r, r, ri, ri); @@ -3962,6 +4146,7 @@ static void tcg_target_init(TCGContext *s) s->reserved_regs = 0; tcg_regset_set_reg(s->reserved_regs, TCG_REG_CALL_STACK); + tcg_regset_set_reg(s->reserved_regs, TCG_TMP_VEC); #ifdef _WIN64 /* These are call saved, and we don't save them, so don't use them. */ tcg_regset_set_reg(s->reserved_regs, TCG_REG_XMM6); From patchwork Fri May 26 00:23:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 685899 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp642827wrt; Thu, 25 May 2023 17:25:13 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4TH/9Ddi9/4rsrxJIEDBcii8lbAG3iTxJhi02+WmDF/HW9CCSROZbJ7aQ1dF4N+1GFrdg9 X-Received: by 2002:ac8:5f93:0:b0:3f0:ab44:5b09 with SMTP id j19-20020ac85f93000000b003f0ab445b09mr341555qta.0.1685060713461; Thu, 25 May 2023 17:25:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685060713; cv=none; d=google.com; s=arc-20160816; b=OTnEvQT4h/fAa3axbS81sq7rAUR2HShl3tzalfs2UX4egONE3292nHaUMRT2FM+/Wj LX9MV1+aLvBzMv1vn+zT+OSDazY2Dd6sS46K7/hzHq+t772gASfqrEsSKYwFhG77b4x+ 7Ldbn2U5jkzEMyjqbQbN7uCYIGwmy+RP4Dxo5SkjHzyn7JSt/gDGZ/FmRfDTOBiWn/Mb s+tQyLUinsviB9lHOE0j782sWqj8b1X+zE45Uo+BbDPqH09rWs0s09WWDs/dJ2wLh9LX Px6lrDedAxLVTO+BiOm7lBfwITD0+L14d902wt98OrEoXKELd+ui54nFWr3JIZrAK7Ym ojCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=bnFiJZaU3RAw6rmWn/CeET2Ge0Ind0gP67YhCDMwmlw=; b=Ou8Ys81zh1nRqBOH1wHr+UbiUKNFfxWj8MlM0hQYtxW5unHLR/qV80jEZ2m9Oq7mf7 xBOG5E6+uZPV+uXoe0t9e8xvb/CDXXWCz3G+yc8vGPEQhkQja1/KS3Fl4SzHZOf8sBz2 HvzickekC1Uj7RSTehI53sUEeZQgVRxWtHvj69fBxrG0yq5oNnCOU1SPI3F1AcaH9d1q ZeN1Y3yatj2VFsYxGXF8A3O1dajHyZ72pCCmibK44AtFx1SSLQj1aE+tgz6NUldT3BIa o1jB0lR5r7BJbhmWZ5+I58oCtqMS1TcLhD17jxd9ibGtp5s1BpWavdnoK4bIH544+i+m w/WQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=wOTcdzHf; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id p11-20020a05622a00cb00b003f6856c175fsi937096qtw.330.2023.05.25.17.25.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 May 2023 17:25:13 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=wOTcdzHf; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2LFC-0002Y1-E4; Thu, 25 May 2023 20:23:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2LFB-0002XF-Ko for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:45 -0400 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q2LF8-00029Q-Aa for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:44 -0400 Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-64d604cc0aaso292964b3a.2 for ; Thu, 25 May 2023 17:23:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1685060620; x=1687652620; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bnFiJZaU3RAw6rmWn/CeET2Ge0Ind0gP67YhCDMwmlw=; b=wOTcdzHfJLidJwrK2ZGfFideXFVkfNy3hIlZUczETfyt1pLxVUR0ka/S46txtHmieB 466rfL8rr7mgg7keVqTjfyRX/hk2/Xo64F5pSwFO33ru1gnNt2PQFr1B+1Mpr1/ptERG gITCFrurCli8Z7Kuglh7C2fz2Fu7ogIQQwobqi8o+nrr+EvjfaUvDO0xIrSzv0+rfWRr C3ry8M8LIIqT7lB8vPEAI5DDenxHjFr7sGtIU+i2FPN4NDTBq+zYAJZKBT2OMy6TeXpU IMY6htobFLdTfh09tGV5L/IHPamXLr2iQvyTCKP9A6o9a7wmDySFxKZF0Uh/mnQX9aVd uoKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685060620; x=1687652620; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bnFiJZaU3RAw6rmWn/CeET2Ge0Ind0gP67YhCDMwmlw=; b=lrjjijI9kbt8hqB97L44aXtZyZ0KpesexCOwO+IswU2KJa8SV/xWEIEYyIIVNTi9fo ieeihLELYmRIu5iw4qRMuVYnZ5jY2t/0oSMvYqoJWpaDbgog01pGbjSZ7W8B7t1Hsle8 BI/xJ2uhEt7WDiZcGBFLjQSO6DFQtcgH3axJ6chR2DQVoZJwzsh45s06arWRDbiiCZCu KyyuDrJmClwoOe2QLJ2wjt8UB8bdOggibpwNPoPmDpJn3/joJ22JFLvs/Pq/aHB1uspc LUJFiNOrGycnS6AJQ7cfqX963PVyR9l9Pgf9BXkWxacR+enPsZOe3L6hPB6Qqvq+BtFe x5aQ== X-Gm-Message-State: AC+VfDw+cXHkzGtXq1vKaRPWxZ6naLPQEqXlDqnlgC+rVKjxpp9ET9+f tUgbKAkrwXE8rVRr5AkdbFo7hcgqPrY4HBTJtXA= X-Received: by 2002:a05:6a00:2e92:b0:643:b489:246d with SMTP id fd18-20020a056a002e9200b00643b489246dmr906127pfb.3.1685060620479; Thu, 25 May 2023 17:23:40 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:7ac5:31cc:3997:3a16]) by smtp.gmail.com with ESMTPSA id s188-20020a635ec5000000b0053491d92b65sm1675593pgb.84.2023.05.25.17.23.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 May 2023 17:23:40 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, Peter Maydell Subject: [PATCH v4 06/16] tcg/aarch64: Rename temporaries Date: Thu, 25 May 2023 17:23:24 -0700 Message-Id: <20230526002334.1760495-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526002334.1760495-1-richard.henderson@linaro.org> References: <20230526002334.1760495-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::432; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x432.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org We will need to allocate a second general-purpose temporary. Rename the existing temps to add a distinguishing number. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.c.inc | 50 ++++++++++++++++++------------------ 1 file changed, 25 insertions(+), 25 deletions(-) diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index 84283665e7..8996e29ca9 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -71,8 +71,8 @@ static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) return TCG_REG_X0 + slot; } -#define TCG_REG_TMP TCG_REG_X30 -#define TCG_VEC_TMP TCG_REG_V31 +#define TCG_REG_TMP0 TCG_REG_X30 +#define TCG_VEC_TMP0 TCG_REG_V31 #ifndef CONFIG_SOFTMMU #define TCG_REG_GUEST_BASE TCG_REG_X28 @@ -984,7 +984,7 @@ static bool tcg_out_dup_vec(TCGContext *s, TCGType type, unsigned vece, static bool tcg_out_dupm_vec(TCGContext *s, TCGType type, unsigned vece, TCGReg r, TCGReg base, intptr_t offset) { - TCGReg temp = TCG_REG_TMP; + TCGReg temp = TCG_REG_TMP0; if (offset < -0xffffff || offset > 0xffffff) { tcg_out_movi(s, TCG_TYPE_PTR, temp, offset); @@ -1136,8 +1136,8 @@ static void tcg_out_ldst(TCGContext *s, AArch64Insn insn, TCGReg rd, } /* Worst-case scenario, move offset to temp register, use reg offset. */ - tcg_out_movi(s, TCG_TYPE_I64, TCG_REG_TMP, offset); - tcg_out_ldst_r(s, insn, rd, rn, TCG_TYPE_I64, TCG_REG_TMP); + tcg_out_movi(s, TCG_TYPE_I64, TCG_REG_TMP0, offset); + tcg_out_ldst_r(s, insn, rd, rn, TCG_TYPE_I64, TCG_REG_TMP0); } static bool tcg_out_mov(TCGContext *s, TCGType type, TCGReg ret, TCGReg arg) @@ -1353,8 +1353,8 @@ static void tcg_out_call_int(TCGContext *s, const tcg_insn_unit *target) if (offset == sextract64(offset, 0, 26)) { tcg_out_insn(s, 3206, BL, offset); } else { - tcg_out_movi(s, TCG_TYPE_I64, TCG_REG_TMP, (intptr_t)target); - tcg_out_insn(s, 3207, BLR, TCG_REG_TMP); + tcg_out_movi(s, TCG_TYPE_I64, TCG_REG_TMP0, (intptr_t)target); + tcg_out_insn(s, 3207, BLR, TCG_REG_TMP0); } } @@ -1491,7 +1491,7 @@ static void tcg_out_addsub2(TCGContext *s, TCGType ext, TCGReg rl, AArch64Insn insn; if (rl == ah || (!const_bh && rl == bh)) { - rl = TCG_REG_TMP; + rl = TCG_REG_TMP0; } if (const_bl) { @@ -1508,7 +1508,7 @@ static void tcg_out_addsub2(TCGContext *s, TCGType ext, TCGReg rl, possibility of adding 0+const in the low part, and the immediate add instructions encode XSP not XZR. Don't try anything more elaborate here than loading another zero. */ - al = TCG_REG_TMP; + al = TCG_REG_TMP0; tcg_out_movi(s, ext, al, 0); } tcg_out_insn_3401(s, insn, ext, rl, al, bl); @@ -1549,7 +1549,7 @@ static void tcg_out_cltz(TCGContext *s, TCGType ext, TCGReg d, { TCGReg a1 = a0; if (is_ctz) { - a1 = TCG_REG_TMP; + a1 = TCG_REG_TMP0; tcg_out_insn(s, 3507, RBIT, ext, a1, a0); } if (const_b && b == (ext ? 64 : 32)) { @@ -1558,7 +1558,7 @@ static void tcg_out_cltz(TCGContext *s, TCGType ext, TCGReg d, AArch64Insn sel = I3506_CSEL; tcg_out_cmp(s, ext, a0, 0, 1); - tcg_out_insn(s, 3507, CLZ, ext, TCG_REG_TMP, a1); + tcg_out_insn(s, 3507, CLZ, ext, TCG_REG_TMP0, a1); if (const_b) { if (b == -1) { @@ -1571,7 +1571,7 @@ static void tcg_out_cltz(TCGContext *s, TCGType ext, TCGReg d, b = d; } } - tcg_out_insn_3506(s, sel, ext, d, TCG_REG_TMP, b, TCG_COND_NE); + tcg_out_insn_3506(s, sel, ext, d, TCG_REG_TMP0, b, TCG_COND_NE); } } @@ -1588,7 +1588,7 @@ bool tcg_target_has_memory_bswap(MemOp memop) } static const TCGLdstHelperParam ldst_helper_param = { - .ntmp = 1, .tmp = { TCG_REG_TMP } + .ntmp = 1, .tmp = { TCG_REG_TMP0 } }; static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) @@ -1847,7 +1847,7 @@ static void tcg_out_goto_tb(TCGContext *s, int which) set_jmp_insn_offset(s, which); tcg_out32(s, I3206_B); - tcg_out_insn(s, 3207, BR, TCG_REG_TMP); + tcg_out_insn(s, 3207, BR, TCG_REG_TMP0); set_jmp_reset_offset(s, which); } @@ -1866,7 +1866,7 @@ void tb_target_set_jmp_target(const TranslationBlock *tb, int n, ptrdiff_t i_offset = i_addr - jmp_rx; /* Note that we asserted this in range in tcg_out_goto_tb. */ - insn = deposit32(I3305_LDR | TCG_REG_TMP, 5, 19, i_offset >> 2); + insn = deposit32(I3305_LDR | TCG_REG_TMP0, 5, 19, i_offset >> 2); } qatomic_set((uint32_t *)jmp_rw, insn); flush_idcache_range(jmp_rx, jmp_rw, 4); @@ -2060,13 +2060,13 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_rem_i64: case INDEX_op_rem_i32: - tcg_out_insn(s, 3508, SDIV, ext, TCG_REG_TMP, a1, a2); - tcg_out_insn(s, 3509, MSUB, ext, a0, TCG_REG_TMP, a2, a1); + tcg_out_insn(s, 3508, SDIV, ext, TCG_REG_TMP0, a1, a2); + tcg_out_insn(s, 3509, MSUB, ext, a0, TCG_REG_TMP0, a2, a1); break; case INDEX_op_remu_i64: case INDEX_op_remu_i32: - tcg_out_insn(s, 3508, UDIV, ext, TCG_REG_TMP, a1, a2); - tcg_out_insn(s, 3509, MSUB, ext, a0, TCG_REG_TMP, a2, a1); + tcg_out_insn(s, 3508, UDIV, ext, TCG_REG_TMP0, a1, a2); + tcg_out_insn(s, 3509, MSUB, ext, a0, TCG_REG_TMP0, a2, a1); break; case INDEX_op_shl_i64: @@ -2110,8 +2110,8 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, if (c2) { tcg_out_rotl(s, ext, a0, a1, a2); } else { - tcg_out_insn(s, 3502, SUB, 0, TCG_REG_TMP, TCG_REG_XZR, a2); - tcg_out_insn(s, 3508, RORV, ext, a0, a1, TCG_REG_TMP); + tcg_out_insn(s, 3502, SUB, 0, TCG_REG_TMP0, TCG_REG_XZR, a2); + tcg_out_insn(s, 3508, RORV, ext, a0, a1, TCG_REG_TMP0); } break; @@ -2517,8 +2517,8 @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc, break; } } - tcg_out_dupi_vec(s, type, MO_8, TCG_VEC_TMP, 0); - a2 = TCG_VEC_TMP; + tcg_out_dupi_vec(s, type, MO_8, TCG_VEC_TMP0, 0); + a2 = TCG_VEC_TMP0; } if (is_scalar) { insn = cmp_scalar_insn[cond]; @@ -2900,9 +2900,9 @@ static void tcg_target_init(TCGContext *s) s->reserved_regs = 0; tcg_regset_set_reg(s->reserved_regs, TCG_REG_SP); tcg_regset_set_reg(s->reserved_regs, TCG_REG_FP); - tcg_regset_set_reg(s->reserved_regs, TCG_REG_TMP); tcg_regset_set_reg(s->reserved_regs, TCG_REG_X18); /* platform register */ - tcg_regset_set_reg(s->reserved_regs, TCG_VEC_TMP); + tcg_regset_set_reg(s->reserved_regs, TCG_REG_TMP0); + tcg_regset_set_reg(s->reserved_regs, TCG_VEC_TMP0); } /* Saving pairs: (X19, X20) .. (X27, X28), (X29(fp), X30(lr)). */ From patchwork Fri May 26 00:23:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 685912 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp643545wrt; Thu, 25 May 2023 17:27:13 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7BqIMkZrkd8JptfHkvbiBnPnSpVNpQqp3tzyxeizSr1FV5k/w+MxQx9QP0I+yZrsPl8tiN X-Received: by 2002:a05:620a:190d:b0:75b:23a1:47c with SMTP id bj13-20020a05620a190d00b0075b23a1047cmr577034qkb.66.1685060833155; Thu, 25 May 2023 17:27:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685060833; cv=none; d=google.com; s=arc-20160816; b=r+ZfdIMpMLBCudXOimbSW9zpeLGivbIYfjf+M8yOzHbMMz0GexeqiioSzyLdnhq75F VAHppTXE3+cN9o+e1/FYvze6bKOFKhabJ7OB/DnEsnKqvg7e1n4Lnm2vUkhxiEKuWuu8 hm9a1tl6ANhC/oKLF2j0vZ1lVdn/undrnWHOs2JTwdVNDRCFJkspF3iB0bdqgZVsOne1 XR1T018MQgboTJvwOPiw1C4Gq8mioEeD4RJ/So/nR5ZpRbn+8g5PowCNtVHxQ2pmNJGo /KIye2v9AuyWDGiA+DalEShK3I0VAlsh+6BQQkEEG3yx2Plc1XWgkLAmU1JSy3wr4blr itsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=0AAV8qNIzM2y2qtbP5R8DqIlK8an3kV/cLXGaTOytOE=; b=jueCPNdu8bKBOVY+EgCgNEZeZvDXDAtphhkc4HRC3Be+RPooApjLJDvJyoXaXwot3a g8/P53RvWvk5YdA/0kvYJddts7PJG+4W+y+0DUNBz3BOe34S/oVXGM6YgqUWJp6wY5f9 nW8otsx73KF3ZyWgUkIy6xgjALHJqWwOfskGyZeLNuYZn3S2OnHSvYDbg9onrZ9y484m qSxsnY0tUwG2MB7K7RNzZW6Xia3e9WyMcMvQ8qOWS3SkDWfJ7w375n9M7uQHCI0jQ9jS zpxCOSyvJruQ692lLFDg6aNGuwN4ZfEsvtQbwvseMp5/TNHtrPRu/XzWUBzl+ftpc+dQ K1dg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Gd7lrdjm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id o25-20020a05620a15d900b0074df1d28aadsi946020qkm.688.2023.05.25.17.27.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 May 2023 17:27:13 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Gd7lrdjm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2LFE-0002Zl-HN; Thu, 25 May 2023 20:23:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2LFC-0002YO-Uy for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:46 -0400 Received: from mail-pg1-x52a.google.com ([2607:f8b0:4864:20::52a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q2LF8-0002A2-Mc for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:46 -0400 Received: by mail-pg1-x52a.google.com with SMTP id 41be03b00d2f7-517ab9a4a13so141125a12.1 for ; Thu, 25 May 2023 17:23:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1685060621; x=1687652621; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0AAV8qNIzM2y2qtbP5R8DqIlK8an3kV/cLXGaTOytOE=; b=Gd7lrdjmRVyo9O2UCPi9Z71mChg/fJ/nbl4T/7Nc3D2v7gIjtDu9/LYxECJ4kGoCBm IaLnM7bWAItfvzykn2elY5de/1as3Qhi+YWDlM0PvuO5H4hTgiCglgm6i0WfvHnKRcqc AbfI0uj6RMyGbrAQHu+61DgSj94eKjC4FkxKj5RtV8oBc/NwsvJa+isHRCrBiYEPcSgC SOcDX6fZ/x/GI69EvofTrvhXJfmSlMhR8HKy2KtwJR6D28hcQHe3njdO0QG3eWDsTAMp +DAUgaiCbbn6YIFXciKJt/mXe1RyXNondZ/MOIRaRQqgJzyIaGo6JUplJW6eiP8YqXH9 jzZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685060621; x=1687652621; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0AAV8qNIzM2y2qtbP5R8DqIlK8an3kV/cLXGaTOytOE=; b=knyo8bXVIPjcjJDGm/WHNeOnk4jDsSULZsV4Dft5N47PYma2DT9r4358rHF+pASqkI 3yOaa7oPllFehVSjsavwCJVMuaMSMyr2ote+Z1XBBidjItR4qkbtWDuTlT3oVRpFxSr8 G4h4oDQoAW3N8xGB5s5QhZ2KrT30c+NWEkd8QHrNkYrNbvfxMKl7wZf3dLJ8zkfit/aj H/9DGXC+QpA4NQtUvB7S5/VBRLNO5jDOCI1DPGR/P50bf0/fLCWfk9Bm+KP24woIwu0r TM9fCFi2uVdp99S3LB5+t9twfwIS1kdgydI6GIkLxEe34R5zBcoYnqbRe9wsdlMy6Ibk YVmQ== X-Gm-Message-State: AC+VfDzcmxNMGXeIEphLb7FWI9iPeRjYQpV1UGh5cY7aE23diZJ2Rsjo 1R2j7EbYYwFNai+2BdbiF/3g5LsCkSbb0QtW3dw= X-Received: by 2002:a05:6a20:728f:b0:10c:b9ed:6a38 with SMTP id o15-20020a056a20728f00b0010cb9ed6a38mr11531391pzk.28.1685060621323; Thu, 25 May 2023 17:23:41 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:7ac5:31cc:3997:3a16]) by smtp.gmail.com with ESMTPSA id s188-20020a635ec5000000b0053491d92b65sm1675593pgb.84.2023.05.25.17.23.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 May 2023 17:23:40 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH v4 07/16] tcg/aarch64: Reserve TCG_REG_TMP1, TCG_REG_TMP2 Date: Thu, 25 May 2023 17:23:25 -0700 Message-Id: <20230526002334.1760495-8-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526002334.1760495-1-richard.henderson@linaro.org> References: <20230526002334.1760495-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52a; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Peter Maydell --- tcg/aarch64/tcg-target.c.inc | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index 8996e29ca9..5e7ac6fb76 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -40,11 +40,12 @@ static const int tcg_target_reg_alloc_order[] = { TCG_REG_X8, TCG_REG_X9, TCG_REG_X10, TCG_REG_X11, TCG_REG_X12, TCG_REG_X13, TCG_REG_X14, TCG_REG_X15, - TCG_REG_X16, TCG_REG_X17, TCG_REG_X0, TCG_REG_X1, TCG_REG_X2, TCG_REG_X3, TCG_REG_X4, TCG_REG_X5, TCG_REG_X6, TCG_REG_X7, + /* X16 reserved as temporary */ + /* X17 reserved as temporary */ /* X18 reserved by system */ /* X19 reserved for AREG0 */ /* X29 reserved as fp */ @@ -71,7 +72,9 @@ static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) return TCG_REG_X0 + slot; } -#define TCG_REG_TMP0 TCG_REG_X30 +#define TCG_REG_TMP0 TCG_REG_X16 +#define TCG_REG_TMP1 TCG_REG_X17 +#define TCG_REG_TMP2 TCG_REG_X30 #define TCG_VEC_TMP0 TCG_REG_V31 #ifndef CONFIG_SOFTMMU @@ -2902,6 +2905,8 @@ static void tcg_target_init(TCGContext *s) tcg_regset_set_reg(s->reserved_regs, TCG_REG_FP); tcg_regset_set_reg(s->reserved_regs, TCG_REG_X18); /* platform register */ tcg_regset_set_reg(s->reserved_regs, TCG_REG_TMP0); + tcg_regset_set_reg(s->reserved_regs, TCG_REG_TMP1); + tcg_regset_set_reg(s->reserved_regs, TCG_REG_TMP2); tcg_regset_set_reg(s->reserved_regs, TCG_VEC_TMP0); } From patchwork Fri May 26 00:23:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 685905 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp643122wrt; Thu, 25 May 2023 17:26:04 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5B7xqgfDZgqIC6OV/EIk9muRSo2hE2HgQ1ZFjNPnibo5+iJz0QOwR3WYdXiAWbx63UhYoy X-Received: by 2002:a05:6214:ca6:b0:5c5:1a25:edf0 with SMTP id s6-20020a0562140ca600b005c51a25edf0mr477547qvs.26.1685060764514; Thu, 25 May 2023 17:26:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685060764; cv=none; d=google.com; s=arc-20160816; b=pHf8VQ/4S4sexvggl4s0Qs+IN3YOSB4uAzsLEqs68WZp0yZSdWbzFKWnEKSKDRT6in A/ek/fEZqPgoBgPEpOXa0vbxJzwjokExrGMZ64vqtcqSXeuPjMb0ScaRXgpGng5udk2m M/qCQVC1DZjXhQNZVx9t3Lc9gdIvAM5AtMxiZHk83C1QW9jO7igsdqpHrklfp+5HIa5F BhOVsz7WzPOnjeObx1mq7M4zMV/AR85ZEfyZ3ByKI365TOIcj99PzH4xdcJrldSUA8ql /3k8rznx6MbYfW1MTUIF7h8UmYR6Kf07AGdRjB2Fn9ffMIuxTjMurCpGPLiojcdcddxQ c5tw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=6cPi7/umNFT9yA55UTYS9B+2Cw8yXVqd5/fVGdM9ye4=; b=Sy4riHbcnOYQIpWg+svTdcATAx8J/qH3a1GATHAC/+PEMjkSBMhoHkMk0bw7cuEo// f2SbaMeuAw0yo6sP37Hi9LBNd7lnRqEqfPAdjvvZfaqnHCV8ff1oAqzpy2Rq5K72qKpC q91DRxYXVysZtx2tuCAnV2KpNdNHO2YhE/FWJiwnPn064zXEOLTTa2i+m+f356kmmo3+ nQvahbIlih05RXzniSo4FEuIh9FuuhW9v/5cRuaKo4UJtqXb5VQyskOLfABf5WMUu0E7 W1l0DDtaa0fSytlEu/PdDL60MKb2lCpJMONW8owQbVFm+AcPTUpVkF2Ij1RxxMaIgbZ7 uCPQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ETKEwRHj; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id gw6-20020a0562140f0600b0062575bed1f8si983891qvb.272.2023.05.25.17.26.04 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 May 2023 17:26:04 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ETKEwRHj; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2LFG-0002bC-BF; Thu, 25 May 2023 20:23:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2LFE-0002Zk-GG for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:48 -0400 Received: from mail-pg1-x52c.google.com ([2607:f8b0:4864:20::52c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q2LFB-0002AL-Tk for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:48 -0400 Received: by mail-pg1-x52c.google.com with SMTP id 41be03b00d2f7-528dd896165so104908a12.2 for ; Thu, 25 May 2023 17:23:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1685060622; x=1687652622; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6cPi7/umNFT9yA55UTYS9B+2Cw8yXVqd5/fVGdM9ye4=; b=ETKEwRHj8aLnNU+hFxol8zCD0AIva0jScKufj/IqpYdcjrF11ORc40YwjpPzkFZG8B Ki8LKqM9FlGI8n5RhatSNZZaGwgVJd7QCU9EtDWxswfprUFvLsrMs+kNiuT9okM0vXYT GBNr3uF2sjUiN9Y0MH6MSKefqorLsWgcdiFUftdDmxCbG8UvASR9/9u2YfddEysy6jLs sJqXo0CHYCiSUCyOkFvhkwer5LVn02Tuy2ciFd/T0mLevWv1v0UuBYz2It6SLKcGZxSE 9hUsrod85kgUUQ5VChTcOzrrVySH+bpMyi0voXKlwO2Gb3fyhTyN7imWdqe/tI80y+Sv owKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685060622; x=1687652622; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6cPi7/umNFT9yA55UTYS9B+2Cw8yXVqd5/fVGdM9ye4=; b=UDL2GmZbm46WYMMoJJsF1/B8L7qCiWFMk1dxTaCAgPLNqL55mRmivroaXMXY5p3JAL vW3k/YhTcIh6i64LjZmQvAc/a8PE99PbOqC70OJZWoiI8ZXmMDhjQF96vtpisSvrch96 Piy0dJs4+4I5l5srb/64xiOH8znyLz+RgEvHk/QkvucNsdqavpVyqo+LegGEsPBskIP/ 04122ms8lUeRlrRhe5di++uE3vGaDr5e6o0eZ3i/Yw96TsNe3rdZ2TPKZ94icK5Ihr84 rOyorFPk7iBCl0AZIUAvyJ8Kgw34Usbnyy0/iuskpe8+Kfw7t7ntG/FohUjeDsHAl5aU 6qtg== X-Gm-Message-State: AC+VfDz/f4x4SwTBSSZAxkrB2A6rZmpLf0rT2YErreiBnVMZSGUSKmQ8 hXtDHuDCnRcY2mt7VmaFHtvrI2mIs4UI6Md3w+M= X-Received: by 2002:a05:6a20:244f:b0:10b:5347:304 with SMTP id t15-20020a056a20244f00b0010b53470304mr27353pzc.7.1685060622022; Thu, 25 May 2023 17:23:42 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:7ac5:31cc:3997:3a16]) by smtp.gmail.com with ESMTPSA id s188-20020a635ec5000000b0053491d92b65sm1675593pgb.84.2023.05.25.17.23.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 May 2023 17:23:41 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH v4 08/16] tcg/aarch64: Simplify constraints on qemu_ld/st Date: Thu, 25 May 2023 17:23:26 -0700 Message-Id: <20230526002334.1760495-9-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526002334.1760495-1-richard.henderson@linaro.org> References: <20230526002334.1760495-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52c; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Adjust the softmmu tlb to use TMP[0-2], not any of the normally available registers. Since we handle overlap betwen inputs and helper arguments, we can allow any allocatable reg. Signed-off-by: Richard Henderson Reviewed-by: Peter Maydell --- tcg/aarch64/tcg-target-con-set.h | 2 -- tcg/aarch64/tcg-target-con-str.h | 1 - tcg/aarch64/tcg-target.c.inc | 45 ++++++++++++++------------------ 3 files changed, 19 insertions(+), 29 deletions(-) diff --git a/tcg/aarch64/tcg-target-con-set.h b/tcg/aarch64/tcg-target-con-set.h index d6c6866878..109f4ab97c 100644 --- a/tcg/aarch64/tcg-target-con-set.h +++ b/tcg/aarch64/tcg-target-con-set.h @@ -10,11 +10,9 @@ * tcg-target-con-str.h; the constraint combination is inclusive or. */ C_O0_I1(r) -C_O0_I2(lZ, l) C_O0_I2(r, rA) C_O0_I2(rZ, r) C_O0_I2(w, r) -C_O1_I1(r, l) C_O1_I1(r, r) C_O1_I1(w, r) C_O1_I1(w, w) diff --git a/tcg/aarch64/tcg-target-con-str.h b/tcg/aarch64/tcg-target-con-str.h index 00adb64594..fb1a845b4f 100644 --- a/tcg/aarch64/tcg-target-con-str.h +++ b/tcg/aarch64/tcg-target-con-str.h @@ -9,7 +9,6 @@ * REGS(letter, register_mask) */ REGS('r', ALL_GENERAL_REGS) -REGS('l', ALL_QLDST_REGS) REGS('w', ALL_VECTOR_REGS) /* diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index 5e7ac6fb76..00d9033e2f 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -132,14 +132,6 @@ static bool patch_reloc(tcg_insn_unit *code_ptr, int type, #define ALL_GENERAL_REGS 0xffffffffu #define ALL_VECTOR_REGS 0xffffffff00000000ull -#ifdef CONFIG_SOFTMMU -#define ALL_QLDST_REGS \ - (ALL_GENERAL_REGS & ~((1 << TCG_REG_X0) | (1 << TCG_REG_X1) | \ - (1 << TCG_REG_X2) | (1 << TCG_REG_X3))) -#else -#define ALL_QLDST_REGS ALL_GENERAL_REGS -#endif - /* Match a constant valid for addition (12-bit, optionally shifted). */ static inline bool is_aimm(uint64_t val) { @@ -1648,7 +1640,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, unsigned s_bits = opc & MO_SIZE; unsigned s_mask = (1u << s_bits) - 1; unsigned mem_index = get_mmuidx(oi); - TCGReg x3; + TCGReg addr_adj; TCGType mask_type; uint64_t compare_mask; @@ -1660,27 +1652,27 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, mask_type = (s->page_bits + s->tlb_dyn_max_bits > 32 ? TCG_TYPE_I64 : TCG_TYPE_I32); - /* Load env_tlb(env)->f[mmu_idx].{mask,table} into {x0,x1}. */ + /* Load env_tlb(env)->f[mmu_idx].{mask,table} into {tmp0,tmp1}. */ QEMU_BUILD_BUG_ON(TLB_MASK_TABLE_OFS(0) > 0); QEMU_BUILD_BUG_ON(TLB_MASK_TABLE_OFS(0) < -512); QEMU_BUILD_BUG_ON(offsetof(CPUTLBDescFast, mask) != 0); QEMU_BUILD_BUG_ON(offsetof(CPUTLBDescFast, table) != 8); - tcg_out_insn(s, 3314, LDP, TCG_REG_X0, TCG_REG_X1, TCG_AREG0, + tcg_out_insn(s, 3314, LDP, TCG_REG_TMP0, TCG_REG_TMP1, TCG_AREG0, TLB_MASK_TABLE_OFS(mem_index), 1, 0); /* Extract the TLB index from the address into X0. */ tcg_out_insn(s, 3502S, AND_LSR, mask_type == TCG_TYPE_I64, - TCG_REG_X0, TCG_REG_X0, addr_reg, + TCG_REG_TMP0, TCG_REG_TMP0, addr_reg, s->page_bits - CPU_TLB_ENTRY_BITS); - /* Add the tlb_table pointer, creating the CPUTLBEntry address into X1. */ - tcg_out_insn(s, 3502, ADD, 1, TCG_REG_X1, TCG_REG_X1, TCG_REG_X0); + /* Add the tlb_table pointer, forming the CPUTLBEntry address in TMP1. */ + tcg_out_insn(s, 3502, ADD, 1, TCG_REG_TMP1, TCG_REG_TMP1, TCG_REG_TMP0); - /* Load the tlb comparator into X0, and the fast path addend into X1. */ - tcg_out_ld(s, addr_type, TCG_REG_X0, TCG_REG_X1, + /* Load the tlb comparator into TMP0, and the fast path addend into TMP1. */ + tcg_out_ld(s, addr_type, TCG_REG_TMP0, TCG_REG_TMP1, is_ld ? offsetof(CPUTLBEntry, addr_read) : offsetof(CPUTLBEntry, addr_write)); - tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_X1, TCG_REG_X1, + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP1, TCG_REG_TMP1, offsetof(CPUTLBEntry, addend)); /* @@ -1689,25 +1681,26 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, * cross pages using the address of the last byte of the access. */ if (a_mask >= s_mask) { - x3 = addr_reg; + addr_adj = addr_reg; } else { + addr_adj = TCG_REG_TMP2; tcg_out_insn(s, 3401, ADDI, addr_type, - TCG_REG_X3, addr_reg, s_mask - a_mask); - x3 = TCG_REG_X3; + addr_adj, addr_reg, s_mask - a_mask); } compare_mask = (uint64_t)s->page_mask | a_mask; - /* Store the page mask part of the address into X3. */ - tcg_out_logicali(s, I3404_ANDI, addr_type, TCG_REG_X3, x3, compare_mask); + /* Store the page mask part of the address into TMP2. */ + tcg_out_logicali(s, I3404_ANDI, addr_type, TCG_REG_TMP2, + addr_adj, compare_mask); /* Perform the address comparison. */ - tcg_out_cmp(s, addr_type, TCG_REG_X0, TCG_REG_X3, 0); + tcg_out_cmp(s, addr_type, TCG_REG_TMP0, TCG_REG_TMP2, 0); /* If not equal, we jump to the slow path. */ ldst->label_ptr[0] = s->code_ptr; tcg_out_insn(s, 3202, B_C, TCG_COND_NE, 0); - h->base = TCG_REG_X1, + h->base = TCG_REG_TMP1; h->index = addr_reg; h->index_ext = addr_type; #else @@ -2802,12 +2795,12 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_qemu_ld_a64_i32: case INDEX_op_qemu_ld_a32_i64: case INDEX_op_qemu_ld_a64_i64: - return C_O1_I1(r, l); + return C_O1_I1(r, r); case INDEX_op_qemu_st_a32_i32: case INDEX_op_qemu_st_a64_i32: case INDEX_op_qemu_st_a32_i64: case INDEX_op_qemu_st_a64_i64: - return C_O0_I2(lZ, l); + return C_O0_I2(rZ, r); case INDEX_op_deposit_i32: case INDEX_op_deposit_i64: From patchwork Fri May 26 00:23:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 685910 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp643485wrt; Thu, 25 May 2023 17:27:04 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ50+v5ZF8O4Ft+edQoUyGEYa+bRerZXRI1F+sb5qA0zj43i7G26hxj5ky2DCF8HvVcIQrQP X-Received: by 2002:a05:622a:1487:b0:3f5:47c9:e0e with SMTP id t7-20020a05622a148700b003f547c90e0emr263455qtx.31.1685060824164; Thu, 25 May 2023 17:27:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685060824; cv=none; d=google.com; s=arc-20160816; b=bfvRavYwUsPFXXDOuFMv+aOfTp6ecbskcUQrYcbHioaiBHzd5jVG7YBafkCcboFIfj ee/2FbWSfHBfmWJ5vqOys1DAA20Oowm3k5qUaMo1nP3HDp0PXc3Esp6dH1Oioy2FX9FK nExjXgTuR5SfKG2jMuZVa1z5s4e7BdH06sdPS2VcgH9Qk8CIEaz2GnYYRi5aNJ56TTmb 3UiXcZIMYaZhdYCifapd7iVdAYyjDSNlGeXg2vAQIKr5PBZMo2+MTbnSiEZOiMDREMXI VEnoOMiixHK8Hw/Vj0T2l5aDND4ne6CUvwk6oE15Rb+LUiY/owuuqJvv0e8uSUanZatn rF/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=zEZrTsR2PvGkZpVHW0X7uxlBuHoyyk6ikGwddeWtJk8=; b=CJzVrYRdK2+d5VqOeK/cpYD4bp/IWAE/v/L9waKfWjgKW5mFfRhyVlGgEHX4GSeQpB NAb3wYRI8udlV5sRfWPj5ce43cU6QgrB9JsSaJUBP9X+S+hcxqbBsZCLPpm+tnGKv+o0 oLeTRVeSBeMi5c2CWwFag8rps8DIBaErkaBcQoqzufgHxPGdb8qA5MSeYCH4uYOWdi2A Qt1+VnU4DhnIzxhipKpX2LGts7GDhrNgBGdyhUb0EaQ+z9XNgCgN37OYnGQzjgDkaCnr cE1FzYQsyLZ/keR/DX4Ak/x2NKJojOHlUM707aZYajnqC9mLHJUDHTNU6L4H2NBlqibp TJPQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=aCisnnu5; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id c9-20020ac87dc9000000b003f0842d3c40si934223qte.703.2023.05.25.17.27.04 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 May 2023 17:27:04 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=aCisnnu5; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2LFE-0002Z9-2Z; Thu, 25 May 2023 20:23:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2LFC-0002Y6-Jo for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:46 -0400 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q2LFA-0002Ah-8p for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:46 -0400 Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-64d426e63baso458056b3a.0 for ; Thu, 25 May 2023 17:23:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1685060623; x=1687652623; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zEZrTsR2PvGkZpVHW0X7uxlBuHoyyk6ikGwddeWtJk8=; b=aCisnnu5QWtkH4w8dubH+oPT2hQk9c4BZzGCgf/mVbBwMSu5rA9AVp8H6uV+uxldDJ p4x5j5HO+WtVI7JtaOUci+4SzYyF+Q8FtPU6aYS91v6mpuzg1xtcX8eiy2KehKoZnw6O KefkNdlivhZX5H5kf9j+X1ZPnS/QzAOEjcLim6IWiB1Yk7KK6jzkuZedrMmiZyzI71O9 RAIyE3kZ+Ie0HxK9Yp3hVX9ZvYgz59lGQL/9lY32YCUwnseEkfbqiIKfwWbnoYGLGrDX xsCTUevn8EfmUhMkI0FLAc/libehth2ZnkBHDBjTfOm3Hwm+vEd9/HtqP1W7KDoXxwSC 2c0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685060623; x=1687652623; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zEZrTsR2PvGkZpVHW0X7uxlBuHoyyk6ikGwddeWtJk8=; b=aA8KtuYA1F6zZgwpvppKIUcbQtynHV7tRyYfb6v5zcEeBpFICtM2gsgiX/k6FAwZJm K5ddH4a/baNiR6va7MgYZTmhia4FiT78rpIOD+7Wfth0FMBuK4raAR3t3gfsIGDiwLhR WEpzrUBOaNFAy95OTOIJ4htmxfCrIPxgicSkgGijKDDl++JpzilateDIomtWdJZwDvHO 9yjnoFvxUWUjZCfYsJ9Uy0Vx3F6bplTjZty7Ffa4SgjNOLrcGhroqZKDovWtgg1U6KmZ gdwlNSEZogdL88R7gViqL4Dsy/z0qwWSln0wAZ98WNOpPw/YkJAa+iz1anpi0OLMPdcL md9w== X-Gm-Message-State: AC+VfDzdfzmPJ1iDmiCezk8rYm/PtUwirf0MK7oqAiXGtlN4197lTsEv pdEk3r+tafhNny6tWuY1W2bmTPf4vcM/s34SsLo= X-Received: by 2002:a05:6a00:1508:b0:641:3bf8:6514 with SMTP id q8-20020a056a00150800b006413bf86514mr732263pfu.10.1685060622841; Thu, 25 May 2023 17:23:42 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:7ac5:31cc:3997:3a16]) by smtp.gmail.com with ESMTPSA id s188-20020a635ec5000000b0053491d92b65sm1675593pgb.84.2023.05.25.17.23.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 May 2023 17:23:42 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, Peter Maydell Subject: [PATCH v4 09/16] tcg/aarch64: Support 128-bit load/store Date: Thu, 25 May 2023 17:23:27 -0700 Message-Id: <20230526002334.1760495-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526002334.1760495-1-richard.henderson@linaro.org> References: <20230526002334.1760495-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::432; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x432.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org With FEAT_LSE2, LDP/STP suffices. Without FEAT_LSE2, use LDXP+STXP 16-byte atomicity is required and LDP/STP otherwise. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target-con-set.h | 2 + tcg/aarch64/tcg-target.h | 11 ++- tcg/aarch64/tcg-target.c.inc | 141 ++++++++++++++++++++++++++++++- 3 files changed, 151 insertions(+), 3 deletions(-) diff --git a/tcg/aarch64/tcg-target-con-set.h b/tcg/aarch64/tcg-target-con-set.h index 109f4ab97c..3fdee26a3d 100644 --- a/tcg/aarch64/tcg-target-con-set.h +++ b/tcg/aarch64/tcg-target-con-set.h @@ -13,6 +13,7 @@ C_O0_I1(r) C_O0_I2(r, rA) C_O0_I2(rZ, r) C_O0_I2(w, r) +C_O0_I3(rZ, rZ, r) C_O1_I1(r, r) C_O1_I1(w, r) C_O1_I1(w, w) @@ -31,4 +32,5 @@ C_O1_I2(w, w, wO) C_O1_I2(w, w, wZ) C_O1_I3(w, w, w, w) C_O1_I4(r, r, rA, rZ, rZ) +C_O2_I1(r, r, r) C_O2_I4(r, r, rZ, rZ, rA, rMZ) diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index d5f7614880..192a2758c5 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -131,7 +131,16 @@ typedef enum { #define TCG_TARGET_HAS_muluh_i64 1 #define TCG_TARGET_HAS_mulsh_i64 1 -#define TCG_TARGET_HAS_qemu_ldst_i128 0 +/* + * Without FEAT_LSE2, we must use LDXP+STXP to implement atomic 128-bit load, + * which requires writable pages. We must defer to the helper for user-only, + * but in system mode all ram is writable for the host. + */ +#ifdef CONFIG_USER_ONLY +#define TCG_TARGET_HAS_qemu_ldst_i128 have_lse2 +#else +#define TCG_TARGET_HAS_qemu_ldst_i128 1 +#endif #define TCG_TARGET_HAS_v64 1 #define TCG_TARGET_HAS_v128 1 diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index 00d9033e2f..261ad25210 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -385,6 +385,10 @@ typedef enum { I3305_LDR_v64 = 0x5c000000, I3305_LDR_v128 = 0x9c000000, + /* Load/store exclusive. */ + I3306_LDXP = 0xc8600000, + I3306_STXP = 0xc8200000, + /* Load/store register. Described here as 3.3.12, but the helper that emits them can transform to 3.3.10 or 3.3.13. */ I3312_STRB = 0x38000000 | LDST_ST << 22 | MO_8 << 30, @@ -449,6 +453,9 @@ typedef enum { I3406_ADR = 0x10000000, I3406_ADRP = 0x90000000, + /* Add/subtract extended register instructions. */ + I3501_ADD = 0x0b200000, + /* Add/subtract shifted register instructions (without a shift). */ I3502_ADD = 0x0b000000, I3502_ADDS = 0x2b000000, @@ -619,6 +626,12 @@ static void tcg_out_insn_3305(TCGContext *s, AArch64Insn insn, tcg_out32(s, insn | (imm19 & 0x7ffff) << 5 | rt); } +static void tcg_out_insn_3306(TCGContext *s, AArch64Insn insn, TCGReg rs, + TCGReg rt, TCGReg rt2, TCGReg rn) +{ + tcg_out32(s, insn | rs << 16 | rt2 << 10 | rn << 5 | rt); +} + static void tcg_out_insn_3201(TCGContext *s, AArch64Insn insn, TCGType ext, TCGReg rt, int imm19) { @@ -701,6 +714,14 @@ static void tcg_out_insn_3406(TCGContext *s, AArch64Insn insn, tcg_out32(s, insn | (disp & 3) << 29 | (disp & 0x1ffffc) << (5 - 2) | rd); } +static inline void tcg_out_insn_3501(TCGContext *s, AArch64Insn insn, + TCGType sf, TCGReg rd, TCGReg rn, + TCGReg rm, int opt, int imm3) +{ + tcg_out32(s, insn | sf << 31 | rm << 16 | opt << 13 | + imm3 << 10 | rn << 5 | rd); +} + /* This function is for both 3.5.2 (Add/Subtract shifted register), for the rare occasion when we actually want to supply a shift amount. */ static inline void tcg_out_insn_3502S(TCGContext *s, AArch64Insn insn, @@ -1628,16 +1649,16 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, TCGType addr_type = s->addr_type; TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); + MemOp s_bits = opc & MO_SIZE; unsigned a_mask; h->aa = atom_and_align_for_opc(s, opc, have_lse2 ? MO_ATOM_WITHIN16 : MO_ATOM_IFALIGN, - false); + s_bits == MO_128); a_mask = (1 << h->aa.align) - 1; #ifdef CONFIG_SOFTMMU - unsigned s_bits = opc & MO_SIZE; unsigned s_mask = (1u << s_bits) - 1; unsigned mem_index = get_mmuidx(oi); TCGReg addr_adj; @@ -1818,6 +1839,108 @@ static void tcg_out_qemu_st(TCGContext *s, TCGReg data_reg, TCGReg addr_reg, } } +static void tcg_out_qemu_ldst_i128(TCGContext *s, TCGReg datalo, TCGReg datahi, + TCGReg addr_reg, MemOpIdx oi, bool is_ld) +{ + TCGLabelQemuLdst *ldst; + HostAddress h; + TCGReg base; + bool use_pair; + + ldst = prepare_host_addr(s, &h, addr_reg, oi, is_ld); + + /* Compose the final address, as LDP/STP have no indexing. */ + if (h.index == TCG_REG_XZR) { + base = h.base; + } else { + base = TCG_REG_TMP2; + if (h.index_ext == TCG_TYPE_I32) { + /* add base, base, index, uxtw */ + tcg_out_insn(s, 3501, ADD, TCG_TYPE_I64, base, + h.base, h.index, MO_32, 0); + } else { + /* add base, base, index */ + tcg_out_insn(s, 3502, ADD, 1, base, h.base, h.index); + } + } + + use_pair = h.aa.atom < MO_128 || have_lse2; + + if (!use_pair) { + tcg_insn_unit *branch = NULL; + TCGReg ll, lh, sl, sh; + + /* + * If we have already checked for 16-byte alignment, that's all + * we need. Otherwise we have determined that misaligned atomicity + * may be handled with two 8-byte loads. + */ + if (h.aa.align < MO_128) { + /* + * TODO: align should be MO_64, so we only need test bit 3, + * which means we could use TBNZ instead of ANDS+B_C. + */ + tcg_out_logicali(s, I3404_ANDSI, 0, TCG_REG_XZR, addr_reg, 15); + branch = s->code_ptr; + tcg_out_insn(s, 3202, B_C, TCG_COND_NE, 0); + use_pair = true; + } + + if (is_ld) { + /* + * 16-byte atomicity without LSE2 requires LDXP+STXP loop: + * ldxp lo, hi, [base] + * stxp t0, lo, hi, [base] + * cbnz t0, .-8 + * Require no overlap between data{lo,hi} and base. + */ + if (datalo == base || datahi == base) { + tcg_out_mov(s, TCG_TYPE_REG, TCG_REG_TMP2, base); + base = TCG_REG_TMP2; + } + ll = sl = datalo; + lh = sh = datahi; + } else { + /* + * 16-byte atomicity without LSE2 requires LDXP+STXP loop: + * 1: ldxp t0, t1, [base] + * stxp t0, lo, hi, [base] + * cbnz t0, 1b + */ + tcg_debug_assert(base != TCG_REG_TMP0 && base != TCG_REG_TMP1); + ll = TCG_REG_TMP0; + lh = TCG_REG_TMP1; + sl = datalo; + sh = datahi; + } + + tcg_out_insn(s, 3306, LDXP, TCG_REG_XZR, ll, lh, base); + tcg_out_insn(s, 3306, STXP, TCG_REG_TMP0, sl, sh, base); + tcg_out_insn(s, 3201, CBNZ, 0, TCG_REG_TMP0, -2); + + if (use_pair) { + /* "b .+8", branching across the one insn of use_pair. */ + tcg_out_insn(s, 3206, B, 2); + reloc_pc19(branch, tcg_splitwx_to_rx(s->code_ptr)); + } + } + + if (use_pair) { + if (is_ld) { + tcg_out_insn(s, 3314, LDP, datalo, datahi, base, 0, 1, 0); + } else { + tcg_out_insn(s, 3314, STP, datalo, datahi, base, 0, 1, 0); + } + } + + if (ldst) { + ldst->type = TCG_TYPE_I128; + ldst->datalo_reg = datalo; + ldst->datahi_reg = datahi; + ldst->raddr = tcg_splitwx_to_rx(s->code_ptr); + } +} + static const tcg_insn_unit *tb_ret_addr; static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) @@ -2157,6 +2280,14 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_qemu_st_a64_i64: tcg_out_qemu_st(s, REG0(0), a1, a2, ext); break; + case INDEX_op_qemu_ld_a32_i128: + case INDEX_op_qemu_ld_a64_i128: + tcg_out_qemu_ldst_i128(s, a0, a1, a2, args[3], true); + break; + case INDEX_op_qemu_st_a32_i128: + case INDEX_op_qemu_st_a64_i128: + tcg_out_qemu_ldst_i128(s, REG0(0), REG0(1), a2, args[3], false); + break; case INDEX_op_bswap64_i64: tcg_out_rev(s, TCG_TYPE_I64, MO_64, a0, a1); @@ -2796,11 +2927,17 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_qemu_ld_a32_i64: case INDEX_op_qemu_ld_a64_i64: return C_O1_I1(r, r); + case INDEX_op_qemu_ld_a32_i128: + case INDEX_op_qemu_ld_a64_i128: + return C_O2_I1(r, r, r); case INDEX_op_qemu_st_a32_i32: case INDEX_op_qemu_st_a64_i32: case INDEX_op_qemu_st_a32_i64: case INDEX_op_qemu_st_a64_i64: return C_O0_I2(rZ, r); + case INDEX_op_qemu_st_a32_i128: + case INDEX_op_qemu_st_a64_i128: + return C_O0_I3(rZ, rZ, r); case INDEX_op_deposit_i32: case INDEX_op_deposit_i64: From patchwork Fri May 26 00:23:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 685914 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp643583wrt; Thu, 25 May 2023 17:27:18 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6He0pVh29ExBv1Q9FAks+5rFYCuLY+195/S0a8L5yFCueWSEaCW/QjXMLYMulVV7C384yf X-Received: by 2002:ac8:570f:0:b0:3f4:e020:63dd with SMTP id 15-20020ac8570f000000b003f4e02063ddmr267858qtw.7.1685060837944; Thu, 25 May 2023 17:27:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685060837; cv=none; d=google.com; s=arc-20160816; b=kys0+7P9sNYPtMAgLhhkUOu2zYXoJALaAGTZdMXtiyTmDC+EOxNFiQME14lRU8i+uc TCSHmnq6//gEzSf5XUD2PnyhsWMiX32RCK6jBP49eRJiIhzqU4w34/pet2H0UYCBebdk zqZ4l4Cfd87tVkVd77JccA4HdhOu2RX/ppEg0XmdWVWeg71kQJp7RQ/eT+2fUgKb8+SM W50BIcrBdZ46G6n+Txq0kQIQ9PY9ys3dEyaQAaiXdRPEv/4EMTbgOlr+gxbthVqYDXkP Kwe3nNSGxB0QgcPo+cvcozfokyzf42ClEg/uuQMkcZgk6ratVSG+HpsQuchvwbxsp1DF rr5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=8m1gy3VcWcPhyrnWVeS5QjTxUcewlY10qC8szGENLB4=; b=nSKoViDbMMv2yhZt7L2mCLDntUxihO0LP+g0GzD8Pdfum8RyajPEC3aPnlM/RFBm1x WTYvdrF0xhAG3EAKvEJ3KflIUMVQ0r9a70uzAh8eO0ZiZVYnnmDdye/yRcrjYjrMB7VC RTGR+U3Hd69JwP96ao5x+wsQ+0rys/AhLehwf9psdVd4w6omNFl2GdKa1+O+sLDe9i61 V1g5nBCMcsh/AFW/mwFttQaj7uUBFl/IeM3HsahzOBMgI6mW+0nLHuBM4vTGCVXhi9Zk iB85tY67hZzDZsTUtcOjBXC3qRgL8qG8/F3lHH6ZwlAWkD2V+gTaybGlkcAguDVjPzEt QyoQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bAIULuHX; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r8-20020ac85c88000000b003f6b025c0b7si926186qta.541.2023.05.25.17.27.17 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 May 2023 17:27:17 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bAIULuHX; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2LFI-0002cW-Hu; Thu, 25 May 2023 20:23:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2LFG-0002bE-4O for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:50 -0400 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q2LFC-0002Az-CG for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:49 -0400 Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-64d604cc0aaso292993b3a.2 for ; Thu, 25 May 2023 17:23:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1685060624; x=1687652624; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8m1gy3VcWcPhyrnWVeS5QjTxUcewlY10qC8szGENLB4=; b=bAIULuHXyP5kYF+g/POXMt06p5DqPcMYvJ/GZHgzWA9BqA64aLSudFUKqPFo4dDIz5 yDNAzLEe+AryKO6f+qF9N5PH86LFa7Iu0UM8SgV1DH42GuRpS7GegQIbxhdHV+rwKNh1 4TGttgiq1sfVKBRxM7ZTkEy+II0fzKCoM/QJnN7QkQJc5hiWg+AckbgGHzkGIVIps817 0YmeC8uRkn+0pXhp1YlSOSIB3nsherFnywQeHn7J+octDz2HGEqu09NGVflBG7vbwUNI X0s24RHsb9Jwcuu+nid7TwXSpnmH0fBc6jWDLTXSywVWQVo06tli9Oq+S+/2aC0foq3F 1ZRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685060624; x=1687652624; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8m1gy3VcWcPhyrnWVeS5QjTxUcewlY10qC8szGENLB4=; b=jQMyVkm8TiuWAcEzSoNUiVLdQGk8sJH2htNJdQnCr8nAElEa/IRd3m8OGBd0UUmyd8 qvffoh5R7xFXoIQXst+zud6VNrT2Exb6MmEU11RQJHze5rn13dw2AwNZmdD1ARlmmJ0j rrgYaohqOSc2v7UgPTpvevdZ/mhQRNleYEvvFRysGIkfwfq9nUoPqJFZjIzgFo5F20Zd /XCg83luM199cIWhLL5fhxdz6cuUyyQ+6aURJE+D10WpLc7Ynv1MkdezymAsCsGfTMPx hTPIUC0XhugHVI4NYgi4wxn2m1O2RYpX+D0v5COfTcZdANH9yp0nXr2BBiIXR3XTdwS5 VFzQ== X-Gm-Message-State: AC+VfDxBkQHLQsDhmfg1CEUcvi5n30NkwW3dm0EUdkUAL+SK2pm5/cMQ JxzjgtLxXMwILSexMNjWPjkA0rLaZydZgHcumv4= X-Received: by 2002:a05:6a20:72a8:b0:10b:b166:8836 with SMTP id o40-20020a056a2072a800b0010bb1668836mr17307229pzk.47.1685060623841; Thu, 25 May 2023 17:23:43 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:7ac5:31cc:3997:3a16]) by smtp.gmail.com with ESMTPSA id s188-20020a635ec5000000b0053491d92b65sm1675593pgb.84.2023.05.25.17.23.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 May 2023 17:23:43 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, Daniel Henrique Barboza Subject: [PATCH v4 10/16] tcg/ppc: Support 128-bit load/store Date: Thu, 25 May 2023 17:23:28 -0700 Message-Id: <20230526002334.1760495-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526002334.1760495-1-richard.henderson@linaro.org> References: <20230526002334.1760495-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::434; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x434.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use LQ/STQ with ISA v2.07, and 16-byte atomicity is required. Note that these instructions do not require 16-byte alignment. Reviewed-by: Daniel Henrique Barboza Signed-off-by: Richard Henderson --- tcg/ppc/tcg-target-con-set.h | 2 + tcg/ppc/tcg-target-con-str.h | 1 + tcg/ppc/tcg-target.h | 3 +- tcg/ppc/tcg-target.c.inc | 108 +++++++++++++++++++++++++++++++---- 4 files changed, 101 insertions(+), 13 deletions(-) diff --git a/tcg/ppc/tcg-target-con-set.h b/tcg/ppc/tcg-target-con-set.h index f206b29205..bbd7b21247 100644 --- a/tcg/ppc/tcg-target-con-set.h +++ b/tcg/ppc/tcg-target-con-set.h @@ -14,6 +14,7 @@ C_O0_I2(r, r) C_O0_I2(r, ri) C_O0_I2(v, r) C_O0_I3(r, r, r) +C_O0_I3(o, m, r) C_O0_I4(r, r, ri, ri) C_O0_I4(r, r, r, r) C_O1_I1(r, r) @@ -34,6 +35,7 @@ C_O1_I3(v, v, v, v) C_O1_I4(r, r, ri, rZ, rZ) C_O1_I4(r, r, r, ri, ri) C_O2_I1(r, r, r) +C_O2_I1(o, m, r) C_O2_I2(r, r, r, r) C_O2_I4(r, r, rI, rZM, r, r) C_O2_I4(r, r, r, r, rI, rZM) diff --git a/tcg/ppc/tcg-target-con-str.h b/tcg/ppc/tcg-target-con-str.h index 094613cbcb..20846901de 100644 --- a/tcg/ppc/tcg-target-con-str.h +++ b/tcg/ppc/tcg-target-con-str.h @@ -9,6 +9,7 @@ * REGS(letter, register_mask) */ REGS('r', ALL_GENERAL_REGS) +REGS('o', ALL_GENERAL_REGS & 0xAAAAAAAAu) /* odd registers */ REGS('v', ALL_VECTOR_REGS) /* diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h index 0914380bd7..204b70f86a 100644 --- a/tcg/ppc/tcg-target.h +++ b/tcg/ppc/tcg-target.h @@ -149,7 +149,8 @@ extern bool have_vsx; #define TCG_TARGET_HAS_mulsh_i64 1 #endif -#define TCG_TARGET_HAS_qemu_ldst_i128 0 +#define TCG_TARGET_HAS_qemu_ldst_i128 \ + (TCG_TARGET_REG_BITS == 64 && have_isa_2_07) /* * While technically Altivec could support V64, it has no 64-bit store diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc index d4269dffcf..d47a9e3478 100644 --- a/tcg/ppc/tcg-target.c.inc +++ b/tcg/ppc/tcg-target.c.inc @@ -295,25 +295,27 @@ static bool tcg_target_const_match(int64_t val, TCGType type, int ct) #define B OPCD( 18) #define BC OPCD( 16) + #define LBZ OPCD( 34) #define LHZ OPCD( 40) #define LHA OPCD( 42) #define LWZ OPCD( 32) #define LWZUX XO31( 55) -#define STB OPCD( 38) -#define STH OPCD( 44) -#define STW OPCD( 36) - -#define STD XO62( 0) -#define STDU XO62( 1) -#define STDX XO31(149) - #define LD XO58( 0) #define LDX XO31( 21) #define LDU XO58( 1) #define LDUX XO31( 53) #define LWA XO58( 2) #define LWAX XO31(341) +#define LQ OPCD( 56) + +#define STB OPCD( 38) +#define STH OPCD( 44) +#define STW OPCD( 36) +#define STD XO62( 0) +#define STDU XO62( 1) +#define STDX XO31(149) +#define STQ XO62( 2) #define ADDIC OPCD( 12) #define ADDI OPCD( 14) @@ -2020,7 +2022,18 @@ typedef struct { bool tcg_target_has_memory_bswap(MemOp memop) { - return true; + TCGAtomAlign aa; + + if ((memop & MO_SIZE) <= MO_64) { + return true; + } + + /* + * Reject 16-byte memop with 16-byte atomicity, + * but do allow a pair of 64-bit operations. + */ + aa = atom_and_align_for_opc(tcg_ctx, memop, MO_ATOM_IFALIGN, true); + return aa.atom <= MO_64; } /* @@ -2035,7 +2048,7 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, { TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); - MemOp a_bits; + MemOp a_bits, s_bits; /* * Book II, Section 1.4, Single-Copy Atomicity, specifies: @@ -2047,10 +2060,11 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, * As of 3.0, "the non-atomic access is performed as described in * the corresponding list", which matches MO_ATOM_SUBALIGN. */ + s_bits = opc & MO_SIZE; h->aa = atom_and_align_for_opc(s, opc, have_isa_3_00 ? MO_ATOM_SUBALIGN : MO_ATOM_IFALIGN, - false); + s_bits == MO_128); a_bits = h->aa.align; #ifdef CONFIG_SOFTMMU @@ -2060,7 +2074,6 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, int fast_off = TLB_MASK_TABLE_OFS(mem_index); int mask_off = fast_off + offsetof(CPUTLBDescFast, mask); int table_off = fast_off + offsetof(CPUTLBDescFast, table); - unsigned s_bits = opc & MO_SIZE; ldst = new_ldst_label(s); ldst->is_ld = is_ld; @@ -2303,6 +2316,60 @@ static void tcg_out_qemu_st(TCGContext *s, TCGReg datalo, TCGReg datahi, } } +static void tcg_out_qemu_ldst_i128(TCGContext *s, TCGReg datalo, TCGReg datahi, + TCGReg addr_reg, MemOpIdx oi, bool is_ld) +{ + TCGLabelQemuLdst *ldst; + HostAddress h; + bool need_bswap; + uint32_t insn; + TCGReg index; + + ldst = prepare_host_addr(s, &h, addr_reg, -1, oi, is_ld); + + /* Compose the final address, as LQ/STQ have no indexing. */ + index = h.index; + if (h.base != 0) { + index = TCG_REG_TMP1; + tcg_out32(s, ADD | TAB(index, h.base, h.index)); + } + need_bswap = get_memop(oi) & MO_BSWAP; + + if (h.aa.atom == MO_128) { + tcg_debug_assert(!need_bswap); + tcg_debug_assert(datalo & 1); + tcg_debug_assert(datahi == datalo - 1); + insn = is_ld ? LQ : STQ; + tcg_out32(s, insn | TAI(datahi, index, 0)); + } else { + TCGReg d1, d2; + + if (HOST_BIG_ENDIAN ^ need_bswap) { + d1 = datahi, d2 = datalo; + } else { + d1 = datalo, d2 = datahi; + } + + if (need_bswap) { + tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_R0, 8); + insn = is_ld ? LDBRX : STDBRX; + tcg_out32(s, insn | TAB(d1, 0, index)); + tcg_out32(s, insn | TAB(d2, index, TCG_REG_R0)); + } else { + insn = is_ld ? LD : STD; + tcg_out32(s, insn | TAI(d1, index, 0)); + tcg_out32(s, insn | TAI(d2, index, 8)); + } + } + + if (ldst) { + ldst->type = TCG_TYPE_I128; + ldst->datalo_reg = datalo; + ldst->datahi_reg = datahi; + ldst->raddr = tcg_splitwx_to_rx(s->code_ptr); + } +} + static void tcg_out_nop_fill(tcg_insn_unit *p, int count) { int i; @@ -2860,6 +2927,11 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, args[4], TCG_TYPE_I64); } break; + case INDEX_op_qemu_ld_a32_i128: + case INDEX_op_qemu_ld_a64_i128: + tcg_debug_assert(TCG_TARGET_REG_BITS == 64); + tcg_out_qemu_ldst_i128(s, args[0], args[1], args[2], args[3], true); + break; case INDEX_op_qemu_st_a64_i32: if (TCG_TARGET_REG_BITS == 32) { @@ -2889,6 +2961,11 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, args[4], TCG_TYPE_I64); } break; + case INDEX_op_qemu_st_a32_i128: + case INDEX_op_qemu_st_a64_i128: + tcg_debug_assert(TCG_TARGET_REG_BITS == 64); + tcg_out_qemu_ldst_i128(s, args[0], args[1], args[2], args[3], false); + break; case INDEX_op_setcond_i32: tcg_out_setcond(s, TCG_TYPE_I32, args[3], args[0], args[1], args[2], @@ -3722,6 +3799,13 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_qemu_st_a64_i64: return TCG_TARGET_REG_BITS == 64 ? C_O0_I2(r, r) : C_O0_I4(r, r, r, r); + case INDEX_op_qemu_ld_a32_i128: + case INDEX_op_qemu_ld_a64_i128: + return C_O2_I1(o, m, r); + case INDEX_op_qemu_st_a32_i128: + case INDEX_op_qemu_st_a64_i128: + return C_O0_I3(o, m, r); + case INDEX_op_add_vec: case INDEX_op_sub_vec: case INDEX_op_mul_vec: From patchwork Fri May 26 00:23:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 685908 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp643349wrt; Thu, 25 May 2023 17:26:38 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5VbI6I9AlFp3l6wMbJ3VMWbzWazzqimuwiHxs8aPYQn7LVEvH1COdGMoDZxQDxK7Po5Ksc X-Received: by 2002:ac8:4e88:0:b0:3f6:b49a:c137 with SMTP id 8-20020ac84e88000000b003f6b49ac137mr280096qtp.39.1685060798544; Thu, 25 May 2023 17:26:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685060798; cv=none; d=google.com; s=arc-20160816; b=UsIG3TYERPXNhh03MbjHzZMGePSLR+t4T6t8Tl4oh8Lh0s0Akq39YjNwdpW2+gsE4L unt1DMqr2T+GDSmHGMocXefeC5KNhgt+gNRVUKDjPKPBEUgPZmKV7vdW1G3wjhTeUtqp Uhx2pyU7M+yS//NNSN3QjF0mCo4PITHoSH5OAwEHNyWg2NuzxXoZeoJTlPv2Luys7PIN qsv9pbZEoag8x1tL0tb2H7JlIqYGrpbwlqXRspIiPaNUlzwjPaqHvdZV1rFeumfQzhNI pElmGivFYXeMusGYrj+FCvLAonAE3wOjADXBskS7cxAS/8uDSaxWCiDXr/eYZSjnjD/F 93ng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=4rfAPkT1hpqfPm3VGw5E8xpnHHska/uvJShls7SqHCU=; b=UysMwpIdaI0/g8YFIjRnU5kHOJpKInhsssqFp9OkavnBbaxJr9/sXwiYQTZ1nxXr5C grqPDP9UJzu/+h8V8aZpzvm6qnPP0cG9AXMDN0SzzM9y7ijR/Wc/pN8FhG7wdnkX+VbK ao1P2SBhWNqkfm53a93eiIDGRXKCxTbgQaH0yih9XHIhGLCi0kzJnEKCo4zj0SX5TCup O1sE4RKXREZFSHnPibNwDruWtzIzh9gH8C8Zf6qpktxnvRG/lB6I3OzxXO8cGZ39qeVP QPgV3VWD/TaIiCfGd/rCU1eoAqfq0m5J6fCJ7o8iXubXq6TwQ7S7tO0Ih6NEUVQ19uWR W6lQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GlCf2IC3; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k4-20020ac85fc4000000b003f6a6e9edc7si955739qta.54.2023.05.25.17.26.38 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 May 2023 17:26:38 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GlCf2IC3; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2LFG-0002bH-CF; Thu, 25 May 2023 20:23:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2LFE-0002Zu-Lb for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:48 -0400 Received: from mail-pf1-x42f.google.com ([2607:f8b0:4864:20::42f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q2LFC-0002BG-8Y for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:48 -0400 Received: by mail-pf1-x42f.google.com with SMTP id d2e1a72fcca58-64d5f65a2f7so270325b3a.1 for ; Thu, 25 May 2023 17:23:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1685060624; x=1687652624; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4rfAPkT1hpqfPm3VGw5E8xpnHHska/uvJShls7SqHCU=; b=GlCf2IC3Pa8TGJ4c7IryuJyvM9dp1+6r58syS9CNDePVg0UYRdG1Q4Y291NjSpPNUN kIBIZnF4SK/BXY+WaeC0nmexrUsRpJWnsB1Q6hq1GgfZiW4r51+jQE4+O8wj8hVLG+JS e/pcsmtV80zBTncPhDamgHm3RwjgrtykvxRpYdSxZdOuGPx+MNByEhUEyE5B763oJ88B JcpwKGsRiADRoWd/SxBfJGbXsqP5StrHwtcsMbZ+0fhJY8it0DPEENqe664FiRJv9dan KqbFjt18dxWHDDkV66P/JFi59kx1VnxoMoYqp27BQbawQZkxm8xXUOmbdoDEO1myBH2V kcEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685060624; x=1687652624; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4rfAPkT1hpqfPm3VGw5E8xpnHHska/uvJShls7SqHCU=; b=UzH18r11JkiAl88/5d0mOVxAYv/CHloS4StQHoMwMN3DrDPm9AFuYMufZeTXZpbzA9 ZhUOYADOZS3Odplwe9DzaZq8YjDBhULi8DGsUH5LTGmrFGlGbD2jsOBv7PEXF03T8z0z KoOf72DmeXbEaCH39/mWf7TM9PXWdrtmT2oi3eurgvyNgYfNFhPR0Q/MoQkSRU0Cun+1 49FgngZ00G5B8HfcBKGQdVC7mi1r24QBLfryRusZWeoO0/X6zsbhdzGP4ACibyxTvnvo VjTcZobKpo8bgSv5oBdqMYwSXBhfUEkcqH+u96Oco5O0gOKmtmjteKf9GaBHSK10KBHY s8Yg== X-Gm-Message-State: AC+VfDwINRG+RFnS/sHF1gCd2McyUC0HaJojxw3e8jM9JxHCBp9kjoqE lbgpkDJH35MP9NsV8uFExaAACySbG1vUCx1xJ0U= X-Received: by 2002:a05:6a21:33a9:b0:10b:bf3d:bc5d with SMTP id yy41-20020a056a2133a900b0010bbf3dbc5dmr15918368pzb.47.1685060624666; Thu, 25 May 2023 17:23:44 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:7ac5:31cc:3997:3a16]) by smtp.gmail.com with ESMTPSA id s188-20020a635ec5000000b0053491d92b65sm1675593pgb.84.2023.05.25.17.23.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 May 2023 17:23:44 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, Peter Maydell Subject: [PATCH v4 11/16] tcg/s390x: Support 128-bit load/store Date: Thu, 25 May 2023 17:23:29 -0700 Message-Id: <20230526002334.1760495-12-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526002334.1760495-1-richard.henderson@linaro.org> References: <20230526002334.1760495-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42f; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use LPQ/STPQ when 16-byte atomicity is required. Note that these instructions require 16-byte alignment. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/s390x/tcg-target-con-set.h | 2 + tcg/s390x/tcg-target.h | 2 +- tcg/s390x/tcg-target.c.inc | 103 ++++++++++++++++++++++++++++++++- 3 files changed, 103 insertions(+), 4 deletions(-) diff --git a/tcg/s390x/tcg-target-con-set.h b/tcg/s390x/tcg-target-con-set.h index ecc079bb6d..cbad91b2b5 100644 --- a/tcg/s390x/tcg-target-con-set.h +++ b/tcg/s390x/tcg-target-con-set.h @@ -14,6 +14,7 @@ C_O0_I2(r, r) C_O0_I2(r, ri) C_O0_I2(r, rA) C_O0_I2(v, r) +C_O0_I3(o, m, r) C_O1_I1(r, r) C_O1_I1(v, r) C_O1_I1(v, v) @@ -36,6 +37,7 @@ C_O1_I2(v, v, v) C_O1_I3(v, v, v, v) C_O1_I4(r, r, ri, rI, r) C_O1_I4(r, r, rA, rI, r) +C_O2_I1(o, m, r) C_O2_I2(o, m, 0, r) C_O2_I2(o, m, r, r) C_O2_I3(o, m, 0, 1, r) diff --git a/tcg/s390x/tcg-target.h b/tcg/s390x/tcg-target.h index 170007bea5..ec96952172 100644 --- a/tcg/s390x/tcg-target.h +++ b/tcg/s390x/tcg-target.h @@ -140,7 +140,7 @@ extern uint64_t s390_facilities[3]; #define TCG_TARGET_HAS_muluh_i64 0 #define TCG_TARGET_HAS_mulsh_i64 0 -#define TCG_TARGET_HAS_qemu_ldst_i128 0 +#define TCG_TARGET_HAS_qemu_ldst_i128 1 #define TCG_TARGET_HAS_v64 HAVE_FACILITY(VECTOR) #define TCG_TARGET_HAS_v128 HAVE_FACILITY(VECTOR) diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc index dfaa34c264..2e63305279 100644 --- a/tcg/s390x/tcg-target.c.inc +++ b/tcg/s390x/tcg-target.c.inc @@ -243,6 +243,7 @@ typedef enum S390Opcode { RXY_LLGF = 0xe316, RXY_LLGH = 0xe391, RXY_LMG = 0xeb04, + RXY_LPQ = 0xe38f, RXY_LRV = 0xe31e, RXY_LRVG = 0xe30f, RXY_LRVH = 0xe31f, @@ -253,6 +254,7 @@ typedef enum S390Opcode { RXY_STG = 0xe324, RXY_STHY = 0xe370, RXY_STMG = 0xeb24, + RXY_STPQ = 0xe38e, RXY_STRV = 0xe33e, RXY_STRVG = 0xe32f, RXY_STRVH = 0xe33f, @@ -1577,7 +1579,18 @@ typedef struct { bool tcg_target_has_memory_bswap(MemOp memop) { - return true; + TCGAtomAlign aa; + + if ((memop & MO_SIZE) <= MO_64) { + return true; + } + + /* + * Reject 16-byte memop with 16-byte atomicity, + * but do allow a pair of 64-bit operations. + */ + aa = atom_and_align_for_opc(tcg_ctx, memop, MO_ATOM_IFALIGN, true); + return aa.atom <= MO_64; } static void tcg_out_qemu_ld_direct(TCGContext *s, MemOp opc, TCGReg data, @@ -1734,13 +1747,13 @@ static TCGLabelQemuLdst *prepare_host_addr(TCGContext *s, HostAddress *h, { TCGLabelQemuLdst *ldst = NULL; MemOp opc = get_memop(oi); + MemOp s_bits = opc & MO_SIZE; unsigned a_mask; - h->aa = atom_and_align_for_opc(s, opc, MO_ATOM_IFALIGN, false); + h->aa = atom_and_align_for_opc(s, opc, MO_ATOM_IFALIGN, s_bits == MO_128); a_mask = (1 << h->aa.align) - 1; #ifdef CONFIG_SOFTMMU - unsigned s_bits = opc & MO_SIZE; unsigned s_mask = (1 << s_bits) - 1; int mem_index = get_mmuidx(oi); int fast_off = TLB_MASK_TABLE_OFS(mem_index); @@ -1865,6 +1878,80 @@ static void tcg_out_qemu_st(TCGContext* s, TCGReg data_reg, TCGReg addr_reg, } } +static void tcg_out_qemu_ldst_i128(TCGContext *s, TCGReg datalo, TCGReg datahi, + TCGReg addr_reg, MemOpIdx oi, bool is_ld) +{ + TCGLabel *l1 = NULL, *l2 = NULL; + TCGLabelQemuLdst *ldst; + HostAddress h; + bool need_bswap; + bool use_pair; + S390Opcode insn; + + ldst = prepare_host_addr(s, &h, addr_reg, oi, is_ld); + + use_pair = h.aa.atom < MO_128; + need_bswap = get_memop(oi) & MO_BSWAP; + + if (!use_pair) { + /* + * Atomicity requires we use LPQ. If we've already checked for + * 16-byte alignment, that's all we need. If we arrive with + * lesser alignment, we have determined that less than 16-byte + * alignment can be satisfied with two 8-byte loads. + */ + if (h.aa.align < MO_128) { + use_pair = true; + l1 = gen_new_label(); + l2 = gen_new_label(); + + tcg_out_insn(s, RI, TMLL, addr_reg, 15); + tgen_branch(s, 7, l1); /* CC in {1,2,3} */ + } + + tcg_debug_assert(!need_bswap); + tcg_debug_assert(datalo & 1); + tcg_debug_assert(datahi == datalo - 1); + insn = is_ld ? RXY_LPQ : RXY_STPQ; + tcg_out_insn_RXY(s, insn, datahi, h.base, h.index, h.disp); + + if (use_pair) { + tgen_branch(s, S390_CC_ALWAYS, l2); + tcg_out_label(s, l1); + } + } + if (use_pair) { + TCGReg d1, d2; + + if (need_bswap) { + d1 = datalo, d2 = datahi; + insn = is_ld ? RXY_LRVG : RXY_STRVG; + } else { + d1 = datahi, d2 = datalo; + insn = is_ld ? RXY_LG : RXY_STG; + } + + if (h.base == d1 || h.index == d1) { + tcg_out_insn(s, RXY, LAY, TCG_TMP0, h.base, h.index, h.disp); + h.base = TCG_TMP0; + h.index = TCG_REG_NONE; + h.disp = 0; + } + tcg_out_insn_RXY(s, insn, d1, h.base, h.index, h.disp); + tcg_out_insn_RXY(s, insn, d2, h.base, h.index, h.disp + 8); + } + if (l2) { + tcg_out_label(s, l2); + } + + if (ldst) { + ldst->type = TCG_TYPE_I128; + ldst->datalo_reg = datalo; + ldst->datahi_reg = datahi; + ldst->raddr = tcg_splitwx_to_rx(s->code_ptr); + } +} + static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) { /* Reuse the zeroing that exists for goto_ptr. */ @@ -2226,6 +2313,12 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_qemu_st_a64_i64: tcg_out_qemu_st(s, args[0], args[1], args[2], TCG_TYPE_I64); break; + case INDEX_op_qemu_ld_i128: + tcg_out_qemu_ldst_i128(s, args[0], args[1], args[2], args[3], true); + break; + case INDEX_op_qemu_st_i128: + tcg_out_qemu_ldst_i128(s, args[0], args[1], args[2], args[3], false); + break; case INDEX_op_ld16s_i64: tcg_out_mem(s, 0, RXY_LGH, args[0], args[1], TCG_REG_NONE, args[2]); @@ -3107,6 +3200,10 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_qemu_st_a32_i32: case INDEX_op_qemu_st_a64_i32: return C_O0_I2(r, r); + case INDEX_op_qemu_ld_i128: + return C_O2_I1(o, m, r); + case INDEX_op_qemu_st_i128: + return C_O0_I3(o, m, r); case INDEX_op_deposit_i32: case INDEX_op_deposit_i64: From patchwork Fri May 26 00:23:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 685913 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp643550wrt; Thu, 25 May 2023 17:27:13 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6LrItxByugbELmOhHvPA1jAtdLUFdCKafIhAyTmfSIGb1UOQr7fmhlnp5OuK+spvcA32CE X-Received: by 2002:a05:6214:2627:b0:625:947f:d2b9 with SMTP id gv7-20020a056214262700b00625947fd2b9mr557870qvb.5.1685060833725; Thu, 25 May 2023 17:27:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685060833; cv=none; d=google.com; s=arc-20160816; b=Fe/5M+Ht2M7A7Elq/PhT+TWVWgFDLilla1tDYxhOhos0WUmYCxvJUr2PbYd3Z9Nvma T/unwAWPlwVwlj+IC10dHOt/OgPRmqdQm07QzdTBYxEh0eE37Uht1GwsZ8wK/UlQhRbf 3TH+nH+hEHdhvEIPuh4/fmZf1JlzPsL4LtxDsTykDo/mrbzqyy/o7sETgQjnTBLNi6mI Y0wvhHJyLY7DzkmRsR/xeo+9zjH30l/1D10PMf6/wIM0KPQ3yIl36BYDCLx553qLsk2B WXBgNp5NLXbZDQ0/0mwgRn70oju7lBqyDW1693WM0LYSiUY2g0gModau1F1muvWowTEA T7Gg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=k9zblclqYPHjXAc9J8dhxRq0a0SpKKfcw5wbjQnIz2w=; b=ivCawZEEM4KX0GMkn5liJnF+W9xVJhUNXTm07xrGXGm4i/tIhieYlMLHwDrMO1a5w6 OmAJKk3sGHHEZrHL7InmGk/Day4QVPTuPqPSNK4Lakq1pZS0o8P+A80CHgEV5DxziVwq 9kbei7Xvhhw5yhAkLMEATrf3FW7YPUPEi0BcCL/yjRSTjM7pTfYOgmL5AdRY2bxmNCdD C5RSOTrWxd4sPeaif8svAMeKO1YIbTwsgnk6ke8WlvXJAICW95CSmjSuxYfUmGTx3ZkO Wgy0z9WllbWELvRwVUooVA0RAeQr55Hg9ZBW6O5kJz6o4TWkdfTJ1FWblNq7r+3JRtTW +Paw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="l6gyj/Yh"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id kc24-20020a056214411800b005e321046288si973589qvb.5.2023.05.25.17.27.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 May 2023 17:27:13 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="l6gyj/Yh"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2LFI-0002ca-J0; Thu, 25 May 2023 20:23:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2LFG-0002bD-3g for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:50 -0400 Received: from mail-pf1-x431.google.com ([2607:f8b0:4864:20::431]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q2LFD-0002BT-7R for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:49 -0400 Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-64d2f99c8c3so307139b3a.0 for ; Thu, 25 May 2023 17:23:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1685060625; x=1687652625; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=k9zblclqYPHjXAc9J8dhxRq0a0SpKKfcw5wbjQnIz2w=; b=l6gyj/YhlNXXL4dL6FTfvAuvz9MScaOHX50l0Lk23OP1Vr7MLgUKgZugwRMek//kUx X0xvkJWzs/886npzuUhm11Xw7WQNjIEV/V1G3FwZEAG0rEwU0Hx+1WPXtDs8rkLSGeE1 7+diKi9VUnobGUOgvzmuDFxxotuMcMV6jbnDVPeNoWtKDerqKyNznR/mvJh1irSG+7+n F3VNYGGUmXTWxP3rnnbtq0IA12v54BEZ8yzsthuWOUyfjXI5hp4naYrbKUGAqmfXUnP1 5x2IbP+K+J/PPGleRmcwKof9ifpjAQobQ6QRiPiVU5Nm01fTPuiOndzRXJ0gMpv6bvV0 PSWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685060625; x=1687652625; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=k9zblclqYPHjXAc9J8dhxRq0a0SpKKfcw5wbjQnIz2w=; b=jjwYor0s1QsbeGD8boGChTcqHdY8Ujam8f8c0CYt5hHfOFnus7vvE6jOahGltY1Car hhWWJkK0vlvqIgOhKZkQcEooIvOIqPdIzEBPZzUWvUsJqrWVaXAZWejkz4m7HrGAG8sT UzlhOFSW7fJrPwkT6Uf998zAFPCJUhbmzMHF2QSiJrncRB8pd66XItmPDOYPksMFFEnT YNMgb41A0/7DOu7sIv4E2fyX+6gJJA77sgLlvus1LZmFLbnrGB1fsbIQovocZ6wx+sX8 j0rkVg2u2e5c3bF40M0pzJbfchjBdYiCSR/y3LUpeinUaEQgvgEDIbsp+UKkU+figuVd 9C1w== X-Gm-Message-State: AC+VfDwX8OGViq+PK8Gm10lUag7qQ/4S/J64/5LUTLmow4kBVST//mA3 K14uON45g18sDU2DEUjz+WzqQ36YzTAFVjEv2Pg= X-Received: by 2002:a05:6a20:3955:b0:10b:1c98:59b6 with SMTP id r21-20020a056a20395500b0010b1c9859b6mr20157pzg.14.1685060625475; Thu, 25 May 2023 17:23:45 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:7ac5:31cc:3997:3a16]) by smtp.gmail.com with ESMTPSA id s188-20020a635ec5000000b0053491d92b65sm1675593pgb.84.2023.05.25.17.23.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 May 2023 17:23:45 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH v4 12/16] accel/tcg: Extract load_atom_extract_al16_or_al8 to host header Date: Thu, 25 May 2023 17:23:30 -0700 Message-Id: <20230526002334.1760495-13-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526002334.1760495-1-richard.henderson@linaro.org> References: <20230526002334.1760495-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::431; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x431.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Peter Maydell --- .../generic/host/load-extract-al16-al8.h | 45 +++++++++++++++++++ accel/tcg/ldst_atomicity.c.inc | 36 +-------------- 2 files changed, 47 insertions(+), 34 deletions(-) create mode 100644 host/include/generic/host/load-extract-al16-al8.h diff --git a/host/include/generic/host/load-extract-al16-al8.h b/host/include/generic/host/load-extract-al16-al8.h new file mode 100644 index 0000000000..d95556130f --- /dev/null +++ b/host/include/generic/host/load-extract-al16-al8.h @@ -0,0 +1,45 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Atomic extract 64 from 128-bit, generic version. + * + * Copyright (C) 2023 Linaro, Ltd. + */ + +#ifndef HOST_LOAD_EXTRACT_AL16_AL8_H +#define HOST_LOAD_EXTRACT_AL16_AL8_H + +/** + * load_atom_extract_al16_or_al8: + * @pv: host address + * @s: object size in bytes, @s <= 8. + * + * Load @s bytes from @pv, when pv % s != 0. If [p, p+s-1] does not + * cross an 16-byte boundary then the access must be 16-byte atomic, + * otherwise the access must be 8-byte atomic. + */ +static inline uint64_t ATTRIBUTE_ATOMIC128_OPT +load_atom_extract_al16_or_al8(void *pv, int s) +{ + uintptr_t pi = (uintptr_t)pv; + int o = pi & 7; + int shr = (HOST_BIG_ENDIAN ? 16 - s - o : o) * 8; + Int128 r; + + pv = (void *)(pi & ~7); + if (pi & 8) { + uint64_t *p8 = __builtin_assume_aligned(pv, 16, 8); + uint64_t a = qatomic_read__nocheck(p8); + uint64_t b = qatomic_read__nocheck(p8 + 1); + + if (HOST_BIG_ENDIAN) { + r = int128_make128(b, a); + } else { + r = int128_make128(a, b); + } + } else { + r = atomic16_read_ro(pv); + } + return int128_getlo(int128_urshift(r, shr)); +} + +#endif /* HOST_LOAD_EXTRACT_AL16_AL8_H */ diff --git a/accel/tcg/ldst_atomicity.c.inc b/accel/tcg/ldst_atomicity.c.inc index 57163f5ca2..39ad89800d 100644 --- a/accel/tcg/ldst_atomicity.c.inc +++ b/accel/tcg/ldst_atomicity.c.inc @@ -9,6 +9,8 @@ * See the COPYING file in the top-level directory. */ +#include "host/load-extract-al16-al8.h" + #ifdef CONFIG_ATOMIC64 # define HAVE_al8 true #else @@ -311,40 +313,6 @@ static uint64_t load_atom_extract_al16_or_exit(CPUArchState *env, uintptr_t ra, return int128_getlo(r); } -/** - * load_atom_extract_al16_or_al8: - * @p: host address - * @s: object size in bytes, @s <= 8. - * - * Load @s bytes from @p, when p % s != 0. If [p, p+s-1] does not - * cross an 16-byte boundary then the access must be 16-byte atomic, - * otherwise the access must be 8-byte atomic. - */ -static inline uint64_t ATTRIBUTE_ATOMIC128_OPT -load_atom_extract_al16_or_al8(void *pv, int s) -{ - uintptr_t pi = (uintptr_t)pv; - int o = pi & 7; - int shr = (HOST_BIG_ENDIAN ? 16 - s - o : o) * 8; - Int128 r; - - pv = (void *)(pi & ~7); - if (pi & 8) { - uint64_t *p8 = __builtin_assume_aligned(pv, 16, 8); - uint64_t a = qatomic_read__nocheck(p8); - uint64_t b = qatomic_read__nocheck(p8 + 1); - - if (HOST_BIG_ENDIAN) { - r = int128_make128(b, a); - } else { - r = int128_make128(a, b); - } - } else { - r = atomic16_read_ro(pv); - } - return int128_getlo(int128_urshift(r, shr)); -} - /** * load_atom_4_by_2: * @pv: host address From patchwork Fri May 26 00:23:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 685904 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp643123wrt; Thu, 25 May 2023 17:26:04 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6wAMys1e8/kUIfDlJGSWh69UN60e49wxxBlB2D2Q9W7lDSxLYkfroh0gsvsAC4qHpBmoMz X-Received: by 2002:a05:620a:84ca:b0:75b:23a0:d9f8 with SMTP id pq10-20020a05620a84ca00b0075b23a0d9f8mr443975qkn.78.1685060764531; Thu, 25 May 2023 17:26:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685060764; cv=none; d=google.com; s=arc-20160816; b=l0do7WW95/+k3I4SICPQn9Yha1n4P/HOOlba/0Iu9z8IBWrKuQgIFjuC3ulFzX9Yrx P2IZ6ryx0kvpX1xVCNkTWuLC/w9QEBnWAOo3mGpzObfnVZU2xUZYdlfCbq/y9/cSw0kC KSVB9XH+FP7hwiAfaHltuU836zBMulE8+4NueH5J49GYhPc4byJbkajOhcVpd8yAr7de UIlJMR4Sqds/fLBLiOEkZZemO7Rb2Xdq0pbApPMV0Rnhu74WEJ609ced6mptdhApST7N CFFPMqDKa6nhmqMGtOMUip2OHoIQo4PFjeXZSGHKiJSUoZBCLOgeCGYfaHJTlWZ9JrRy 2CNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=MUe+ePpAXqeWQugAYIOwy5+kFWDdU+DeS/QoHodF2ms=; b=G6VfszLuMi15AWG1XONuFZL0i5BEdRW28ltr5BsDvGsJ5g02MnddCrAh10R8tyD6Zm FgxIHiUyHyvUGzP310V8Pj+FhTot1KlyYECGmvmHVo5cX9Xf4r8yWh48DJ+xp/MCeSQI 4ipa0dm0E1ARIx4i0wUg8HmVtCfbwzI2gIPjF92nPu7+jot9mYtay2e6FeKlk3iSkHjr t46K2UQchCXeIZ1T/tvGCJ9qEX2F0ihs8Oo1TjrkBUv9cmsoLJ42btbfOXVM/wRmRLGd HzB7+FG5N8CP0g1xmxgJ/DkITYDcV34fLQCVyW7R09X0v++W/yl83AA5NG3n82oO5SsC qvRw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YPJL4LXa; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 4-20020a05621420c400b0057104cd4825si970627qve.68.2023.05.25.17.26.04 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 May 2023 17:26:04 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YPJL4LXa; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2LFL-0002eE-9S; Thu, 25 May 2023 20:23:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2LFK-0002do-8o for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:54 -0400 Received: from mail-pf1-x430.google.com ([2607:f8b0:4864:20::430]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q2LFD-0002Bu-H7 for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:54 -0400 Received: by mail-pf1-x430.google.com with SMTP id d2e1a72fcca58-64d5f65a2f7so270344b3a.1 for ; Thu, 25 May 2023 17:23:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1685060626; x=1687652626; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MUe+ePpAXqeWQugAYIOwy5+kFWDdU+DeS/QoHodF2ms=; b=YPJL4LXaItaCdTT1Wk+w9SzMZexjbtJkqDF4P0Z5sLF/at1inAQAR+nvaNhcK9++tn 99n+8/a3lnGnrVmLBzAmxqZrrFmBW1T01xVJahK4Tc3U1knQ/dR98C9c6Zv3u+bNvD5w +MJF0eY1NwTv7PhUb761HObbb7RFYL/KRYE+XFvodJiBe793XinAMVG4rDKvRHR3wdJL bPk1WEaBrWjXkrBSOPcVUb1aNiTPZAW0KhN+7nLj0B+CrRiuWNOo5YB+XtGOYvPWNIzf CETl9Ex+I3NNlR5bKflG1kBaF4TNyG5wYFTKb4hPqaSFM3Auj/H4zk5DalazNMuxnaS2 v2/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685060626; x=1687652626; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MUe+ePpAXqeWQugAYIOwy5+kFWDdU+DeS/QoHodF2ms=; b=TViXgoIN4kR3EEbHRytK9dqCHZlOt5zZ5uQs485nI/h126Sbr6OgZWxwz2ZTcEL/eV vDnpw6/ZUBfwkuVQbkKo26gCboMMPlTppqN0iM6fzda8wcoJTIMpbRwT6CsPoAjFdLk/ rwZDJLKwNGmKeB1fSqdgjSvfE8v9ozcWSuCBZsaZNQDyPKvDH7VM1XSn0lp31JMczW0K 4PxoffjqmX8jnanI23ql1XOz/TdxrzXOJJkTVOptPLNNMowtL6Q+bhkDNkLFrtN+KdOE 6wqykr4tT2qzS8PGNf1xPEX2bOQNm0j2HOipxJkDN8+C/WUFMxT5CPJlX2tb6iKjlx4K IumA== X-Gm-Message-State: AC+VfDxc7TTaevtCIw+yHZkTP82lQglkuwZZIarl7aPDgIWNmx0kwmL0 MZ11mM6PGOBx7Jj+pqwQCd5miZB+HkflTIHWSdA= X-Received: by 2002:a05:6a20:258b:b0:105:5800:c51b with SMTP id k11-20020a056a20258b00b001055800c51bmr25388429pzd.30.1685060626284; Thu, 25 May 2023 17:23:46 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:7ac5:31cc:3997:3a16]) by smtp.gmail.com with ESMTPSA id s188-20020a635ec5000000b0053491d92b65sm1675593pgb.84.2023.05.25.17.23.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 May 2023 17:23:45 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH v4 13/16] accel/tcg: Extract store_atom_insert_al16 to host header Date: Thu, 25 May 2023 17:23:31 -0700 Message-Id: <20230526002334.1760495-14-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526002334.1760495-1-richard.henderson@linaro.org> References: <20230526002334.1760495-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::430; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x430.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Peter Maydell --- host/include/generic/host/store-insert-al16.h | 50 +++++++++++++++++++ accel/tcg/ldst_atomicity.c.inc | 40 +-------------- 2 files changed, 51 insertions(+), 39 deletions(-) create mode 100644 host/include/generic/host/store-insert-al16.h diff --git a/host/include/generic/host/store-insert-al16.h b/host/include/generic/host/store-insert-al16.h new file mode 100644 index 0000000000..4a1662183d --- /dev/null +++ b/host/include/generic/host/store-insert-al16.h @@ -0,0 +1,50 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Atomic store insert into 128-bit, generic version. + * + * Copyright (C) 2023 Linaro, Ltd. + */ + +#ifndef HOST_STORE_INSERT_AL16_H +#define HOST_STORE_INSERT_AL16_H + +/** + * store_atom_insert_al16: + * @p: host address + * @val: shifted value to store + * @msk: mask for value to store + * + * Atomically store @val to @p masked by @msk. + */ +static inline void ATTRIBUTE_ATOMIC128_OPT +store_atom_insert_al16(Int128 *ps, Int128 val, Int128 msk) +{ +#if defined(CONFIG_ATOMIC128) + __uint128_t *pu; + Int128Alias old, new; + + /* With CONFIG_ATOMIC128, we can avoid the memory barriers. */ + pu = __builtin_assume_aligned(ps, 16); + old.u = *pu; + msk = int128_not(msk); + do { + new.s = int128_and(old.s, msk); + new.s = int128_or(new.s, val); + } while (!__atomic_compare_exchange_n(pu, &old.u, new.u, true, + __ATOMIC_RELAXED, __ATOMIC_RELAXED)); +#else + Int128 old, new, cmp; + + ps = __builtin_assume_aligned(ps, 16); + old = *ps; + msk = int128_not(msk); + do { + cmp = old; + new = int128_and(old, msk); + new = int128_or(new, val); + old = atomic16_cmpxchg(ps, cmp, new); + } while (int128_ne(cmp, old)); +#endif +} + +#endif /* HOST_STORE_INSERT_AL16_H */ diff --git a/accel/tcg/ldst_atomicity.c.inc b/accel/tcg/ldst_atomicity.c.inc index 39ad89800d..6844f85d58 100644 --- a/accel/tcg/ldst_atomicity.c.inc +++ b/accel/tcg/ldst_atomicity.c.inc @@ -10,6 +10,7 @@ */ #include "host/load-extract-al16-al8.h" +#include "host/store-insert-al16.h" #ifdef CONFIG_ATOMIC64 # define HAVE_al8 true @@ -681,45 +682,6 @@ static void store_atom_insert_al8(uint64_t *p, uint64_t val, uint64_t msk) __ATOMIC_RELAXED, __ATOMIC_RELAXED)); } -/** - * store_atom_insert_al16: - * @p: host address - * @val: shifted value to store - * @msk: mask for value to store - * - * Atomically store @val to @p masked by @msk. - */ -static void ATTRIBUTE_ATOMIC128_OPT -store_atom_insert_al16(Int128 *ps, Int128Alias val, Int128Alias msk) -{ -#if defined(CONFIG_ATOMIC128) - __uint128_t *pu, old, new; - - /* With CONFIG_ATOMIC128, we can avoid the memory barriers. */ - pu = __builtin_assume_aligned(ps, 16); - old = *pu; - do { - new = (old & ~msk.u) | val.u; - } while (!__atomic_compare_exchange_n(pu, &old, new, true, - __ATOMIC_RELAXED, __ATOMIC_RELAXED)); -#elif defined(CONFIG_CMPXCHG128) - __uint128_t *pu, old, new; - - /* - * Without CONFIG_ATOMIC128, __atomic_compare_exchange_n will always - * defer to libatomic, so we must use __sync_*_compare_and_swap_16 - * and accept the sequential consistency that comes with it. - */ - pu = __builtin_assume_aligned(ps, 16); - do { - old = *pu; - new = (old & ~msk.u) | val.u; - } while (!__sync_bool_compare_and_swap_16(pu, old, new)); -#else - qemu_build_not_reached(); -#endif -} - /** * store_bytes_leN: * @pv: host address From patchwork Fri May 26 00:23:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 685900 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp642906wrt; Thu, 25 May 2023 17:25:23 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7pQhZUVk04+shvAfiev/LA/4u8wvb8Fx55WGMNhSKFJpFbJHDcBg8bEJ+mdfFY/GlEcWJ/ X-Received: by 2002:a05:6214:3001:b0:625:aa48:def6 with SMTP id ke1-20020a056214300100b00625aa48def6mr272002qvb.54.1685060722913; Thu, 25 May 2023 17:25:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685060722; cv=none; d=google.com; s=arc-20160816; b=UIe+2jZ4m2aiwxVsYI8mr9NPrbCnOiuGGhgrT7A/vcib8jf0zsMXeN9NVgLx+AW5SC xnlB+t02SIPw3LPSumcbwOiWhsuvDHYNLQsik3MZdGzgFlnObcUF60MEEzryG8N++11w LtZ9E3CctBzq51/nToR92BuK5RdirTznc91HDgMTrd7/V4JGisbASZSn82+JWB7vhOok KqLK3Ppfi7Wjjz8LktDSbONYwcoAlTK7G7xx9lyomy6hBh/3PjEQJLFVvEyTYP+BkfPM Gu4w2G2Q6murJbYV5W5T0VxlENvmHlnMUpttNjHZrM1JyqtH9rcMZ7Kaxu1E0tLD3AdN ZxWw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=XL2uu5kH3GvTJsMYlChrH/3oNrEnlWs1pl5X+z8kRGk=; b=ppLzxENtiF2CFzs1fvmjmUNgrFK9lafsgBGJNbuAxoNefU1K1TH4Ys35mL1WoYt6+E 4dQeC6K+dOazXqym4bVaAhaSBFFsjtmaRybk2Te5lZEmnj3Ph4XBD4XIDl+ZKNHOzEXo oeUgRRD7yiS8I/e+HfKGGa27UFKmxqzGbzD4Ohjq/YYYMgpvWq39Imiq/c/cPHmTqq9M 9Xiu9gAtZU2BKMJ3Pz2xkPnCe7GH8CPp8PspxCAnlr8AqRPIyAnJOcBpUf/+O7io0Xmz Z4TM5HSIxgR6ZEfruzVbWr81A/sjAc2H1bk4u3zdA20xqfFTWkYmL9lwDvfVQlUz01Vw fz5A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Nfk2pJnN; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r12-20020a0562140c8c00b0062365d4d531si955630qvr.531.2023.05.25.17.25.22 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 May 2023 17:25:22 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Nfk2pJnN; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2LFO-0002fK-Uo; Thu, 25 May 2023 20:23:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2LFM-0002eR-MB for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:56 -0400 Received: from mail-pj1-x1031.google.com ([2607:f8b0:4864:20::1031]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q2LFE-0002CS-Gc for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:56 -0400 Received: by mail-pj1-x1031.google.com with SMTP id 98e67ed59e1d1-2536b4b3398so315002a91.3 for ; Thu, 25 May 2023 17:23:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1685060627; x=1687652627; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XL2uu5kH3GvTJsMYlChrH/3oNrEnlWs1pl5X+z8kRGk=; b=Nfk2pJnNbjuYbseWwBUWPpugThBH36gR4ctAJlp11ueEmMHNcUAsgTGklugEY6eojO DrTn/k5Y5zLq/UCToC6dlJYbzdFFYrz0RyQhE+74oJZaDjKlyMxhucBgIbrHbI4snSEj mU5sqI+JkOsIpurQRr02B5u1wp46Ael3uVgnGEerx9gzWcIeqHsPyjjfnE5WrFOiigel bMhFF3UP1PKigtHvSmh5iJRVYhAa2IZ1AdFd9XDDs54LEF+eoGI/ueY6WxWzriS1aYuw yHF5zPgDbCHon5C4aMmmpYNkQBQzEcREmJKma+sN2/ihJqIsB87y3kyJ3ktj3eq1uUfk i6Zg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685060627; x=1687652627; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XL2uu5kH3GvTJsMYlChrH/3oNrEnlWs1pl5X+z8kRGk=; b=WudDQ7zCcT2SKgVpsx0QJefHG2z4QfhjnmBc+VJItjg63dUOmulb6OKUxXzkrrX92t LCDZ13NVDR98UC3wQEM7/QivI91Dtyn1rQccCvzWY/gaPSvK08Cy203Q0NxFVRNa975u RjLwCB3D8AXl1qC1u4qrQ4wz/LOwTkNGrFLzjRQ9GWJ1PPNy79Xh2madpPEW8EO66yUf XjLmE6h50jn5jqlXsax1Igrfqi/HKfzNjQcx4nEVR4dZz+yZCjpba8VUF1JR0SO789XP CQrD/tqvMr13nirH+aSzUzZ5wjmTfuPXrGFwvfSHfydwWDJUCRaQT2Ijbzoq7+AaapLO Lf4Q== X-Gm-Message-State: AC+VfDzVm/J4WrJAt7p8tU4CRL/y54xbTGvYO4kkcfW3kwbCAgl/sm7W BaIwRCzPn9fvCKfORNMDariLHiZnmIchzlC4tKA= X-Received: by 2002:a17:90a:73ce:b0:256:6b6:baa1 with SMTP id n14-20020a17090a73ce00b0025606b6baa1mr542634pjk.10.1685060627106; Thu, 25 May 2023 17:23:47 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:7ac5:31cc:3997:3a16]) by smtp.gmail.com with ESMTPSA id s188-20020a635ec5000000b0053491d92b65sm1675593pgb.84.2023.05.25.17.23.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 May 2023 17:23:46 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH v4 14/16] accel/tcg: Add x86_64 load_atom_extract_al16_or_al8 Date: Thu, 25 May 2023 17:23:32 -0700 Message-Id: <20230526002334.1760495-15-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526002334.1760495-1-richard.henderson@linaro.org> References: <20230526002334.1760495-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1031; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1031.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Peter Maydell --- .../x86_64/host/load-extract-al16-al8.h | 50 +++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 host/include/x86_64/host/load-extract-al16-al8.h diff --git a/host/include/x86_64/host/load-extract-al16-al8.h b/host/include/x86_64/host/load-extract-al16-al8.h new file mode 100644 index 0000000000..31b6fe8c45 --- /dev/null +++ b/host/include/x86_64/host/load-extract-al16-al8.h @@ -0,0 +1,50 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Atomic extract 64 from 128-bit, x86_64 version. + * + * Copyright (C) 2023 Linaro, Ltd. + */ + +#ifndef X86_64_LOAD_EXTRACT_AL16_AL8_H +#define X86_64_LOAD_EXTRACT_AL16_AL8_H + +#ifdef CONFIG_INT128_TYPE +#include "host/cpuinfo.h" + +/** + * load_atom_extract_al16_or_al8: + * @pv: host address + * @s: object size in bytes, @s <= 8. + * + * Load @s bytes from @pv, when pv % s != 0. If [p, p+s-1] does not + * cross an 16-byte boundary then the access must be 16-byte atomic, + * otherwise the access must be 8-byte atomic. + */ +static inline uint64_t ATTRIBUTE_ATOMIC128_OPT +load_atom_extract_al16_or_al8(void *pv, int s) +{ + uintptr_t pi = (uintptr_t)pv; + __int128_t *ptr_align = (__int128_t *)(pi & ~7); + int shr = (pi & 7) * 8; + Int128Alias r; + + /* + * ptr_align % 16 is now only 0 or 8. + * If the host supports atomic loads with VMOVDQU, then always use that, + * making the branch highly predictable. Otherwise we must use VMOVDQA + * when ptr_align % 16 == 0 for 16-byte atomicity. + */ + if ((cpuinfo & CPUINFO_ATOMIC_VMOVDQU) || (pi & 8)) { + asm("vmovdqu %1, %0" : "=x" (r.i) : "m" (*ptr_align)); + } else { + asm("vmovdqa %1, %0" : "=x" (r.i) : "m" (*ptr_align)); + } + return int128_getlo(int128_urshift(r.s, shr)); +} +#else +/* Fallback definition that must be optimized away, or error. */ +uint64_t QEMU_ERROR("unsupported atomic") + load_atom_extract_al16_or_al8(void *pv, int s); +#endif + +#endif /* X86_64_LOAD_EXTRACT_AL16_AL8_H */ From patchwork Fri May 26 00:23:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 685909 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp643409wrt; Thu, 25 May 2023 17:26:51 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7pD9FJb5j5j8bJzZYdVgr/wvQSpK4yNehh4C12TyynPTxWLcvgZN3/uNCwE6Kkla9qz1cG X-Received: by 2002:a05:6214:4115:b0:623:4ca9:5b24 with SMTP id kc21-20020a056214411500b006234ca95b24mr438766qvb.12.1685060810895; Thu, 25 May 2023 17:26:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685060810; cv=none; d=google.com; s=arc-20160816; b=SsQmnC21fE05F5xrFIcFU/xl6UE6xOx31KyC2hsirmK0r2JYT9IEpTavw39LplsH9e 5e35OleYMnV2OiRr/Qrq0w60XdbMU/5lI4EURd71L1FLgM8Fl0x19mNkupoeSbgAJkyW ++xlL23dBMnnmACKts2cUeFFWlI0T689fiEqF7DY4AlhCYy3+tkiHyGPbz2B/XedNb6f WJkHZayCmGGMnDOn7ahDur/eHpV3JMksqr8oSVQv7y5i/lsRrhJQmA0Os4aDk8XQgHbN 4T7x3SRrH0vtLguwcsRTooEiH6Z90f0A1SSZyrxQ0jsX2fTA56Y/WaM3eYSrxkMh5kVI McXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=8PelXVvmbJtXORMvHTpXHymW8juD44eXP1zIzDNEopo=; b=aGWn1N9c1g1VjLgumbO7IuW5KhKTue+tQtzfx1krXIg3dFW9yNNcc4FTT80ORuZKiy 2ASDbxmVVByRpe2xP21vWsp9OkAbDHwBz+5mv+Z6aAjtXAaN2/EWDl0UKWx9eFchxbSb +uDxG+j2yEoR6RlVpr1qCuqlFa7qTAQxkfgOtyn6ouBTw7O8MWSLsAg8nt//byCVvvY+ QfUz3p+CSTK77CXqfk59ku6rjEg74oPikQKlRFs8vo9o9LaxbC1Xg1c9CeycHaPtrJW5 RLKe5sI/jBnIo2qM+YAVcw2T+yC+4eTyNFH1HBexqxGUXFxPV08tfuKfACFAHsAJAjGe XhvA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TQY8ejxP; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t14-20020a05621421ae00b00620dab812d0si968697qvc.292.2023.05.25.17.26.50 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 May 2023 17:26:50 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TQY8ejxP; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2LFO-0002fC-BN; Thu, 25 May 2023 20:23:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2LFN-0002ed-MT for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:57 -0400 Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q2LFF-0002Cl-31 for qemu-devel@nongnu.org; Thu, 25 May 2023 20:23:57 -0400 Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-64d18d772bdso438518b3a.3 for ; Thu, 25 May 2023 17:23:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1685060628; x=1687652628; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8PelXVvmbJtXORMvHTpXHymW8juD44eXP1zIzDNEopo=; b=TQY8ejxP1hQn2OBtZ9M8n2clES6hnmTmDM+Po+iGq/3X7lHnhOkwIvdpFefgKIaokP RhWHHgDtPcuNa9gggpTFRjmpqSrV/zaat++QNHWlY+UhKeBQFDZEiFlRiSnn+wIf8UrW PZ8z32ALQg6Pxr079xIjgGMprbZxcNc+eOKZ0tjpcV0BiBw1fDNJv6RyQX6tZxsAT+2W fLeXTyhZHuMnoGaLcn0RA+q8UzwXsVoAAQEq7bnuJNf4Cg1LoRL+sO+sOwvZvpxrjk0E S4QKkcXZKxomrdF3ixxh2/r/Luv9P91JbJmZ8TEKr6d+7i4L1ElpbE5VdCVhorLvQd83 lC1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685060628; x=1687652628; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8PelXVvmbJtXORMvHTpXHymW8juD44eXP1zIzDNEopo=; b=eoQXXQOTXjK0MZaG6Kln/VM/eO1wQ69vH80TFLs0fmJyLHyDgWFEIry7u1gvoIMOAY 4eys2CJEHpqb/cdGy9IHMDTDQ1VgLQJqBehTAN93eTB7Ji+Tsri0v15nQf3FEggmbMBW YDVc5gF8mfaA6n/7OcpdcZQ9iOrdsEcEZM2mheuZA2ewYALzqycZYbbzy9tevswUIl0I cLL+sGGz33HC8P8vqWord9t7hHCN+0f8hYyeGbOc8LjMSb0N87V8jI3G8YyAe/DvdQju By1Sp377zncdi+eETSBgCfq+7WuNIdVgbpgCq44qBwW9QVfBm/pEjbahd1MSyGMSj9dY rD+g== X-Gm-Message-State: AC+VfDyHSfydmehq0bUyjvmN2nTuBij5Qs8D1SxsK3vrjt/P4OPItP1N 7d4WdSMAelVdcuEIHyJNzyBNsj9y2V5Tqxh/U+g= X-Received: by 2002:a05:6a21:796:b0:10d:b160:3d4f with SMTP id mg22-20020a056a21079600b0010db1603d4fmr6246334pzb.38.1685060627890; Thu, 25 May 2023 17:23:47 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:7ac5:31cc:3997:3a16]) by smtp.gmail.com with ESMTPSA id s188-20020a635ec5000000b0053491d92b65sm1675593pgb.84.2023.05.25.17.23.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 May 2023 17:23:47 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH v4 15/16] accel/tcg: Add aarch64 lse2 load_atom_extract_al16_or_al8 Date: Thu, 25 May 2023 17:23:33 -0700 Message-Id: <20230526002334.1760495-16-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526002334.1760495-1-richard.henderson@linaro.org> References: <20230526002334.1760495-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42a; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Peter Maydell --- .../aarch64/host/load-extract-al16-al8.h | 40 +++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 host/include/aarch64/host/load-extract-al16-al8.h diff --git a/host/include/aarch64/host/load-extract-al16-al8.h b/host/include/aarch64/host/load-extract-al16-al8.h new file mode 100644 index 0000000000..bd677c5e26 --- /dev/null +++ b/host/include/aarch64/host/load-extract-al16-al8.h @@ -0,0 +1,40 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Atomic extract 64 from 128-bit, AArch64 version. + * + * Copyright (C) 2023 Linaro, Ltd. + */ + +#ifndef AARCH64_LOAD_EXTRACT_AL16_AL8_H +#define AARCH64_LOAD_EXTRACT_AL16_AL8_H + +#include "host/cpuinfo.h" +#include "tcg/debug-assert.h" + +/** + * load_atom_extract_al16_or_al8: + * @pv: host address + * @s: object size in bytes, @s <= 8. + * + * Load @s bytes from @pv, when pv % s != 0. If [p, p+s-1] does not + * cross an 16-byte boundary then the access must be 16-byte atomic, + * otherwise the access must be 8-byte atomic. + */ +static inline uint64_t load_atom_extract_al16_or_al8(void *pv, int s) +{ + uintptr_t pi = (uintptr_t)pv; + __int128_t *ptr_align = (__int128_t *)(pi & ~7); + int shr = (pi & 7) * 8; + uint64_t l, h; + + /* + * With FEAT_LSE2, LDP is single-copy atomic if 16-byte aligned + * and single-copy atomic on the parts if 8-byte aligned. + * All we need do is align the pointer mod 8. + */ + tcg_debug_assert(HAVE_ATOMIC128_RO); + asm("ldp %0, %1, %2" : "=r"(l), "=r"(h) : "m"(*ptr_align)); + return (l >> shr) | (h << (-shr & 63)); +} + +#endif /* AARCH64_LOAD_EXTRACT_AL16_AL8_H */ From patchwork Fri May 26 00:23:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 685911 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp643508wrt; Thu, 25 May 2023 17:27:07 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5z3MZwnLQ+7KPIJnLBXDPWXqJuilLHL7PWMFtdjTkFfQKCdL66nfoKBN9hXS3lNLUYmqwY X-Received: by 2002:a05:622a:1a85:b0:3f6:a9a2:3b6c with SMTP id s5-20020a05622a1a8500b003f6a9a23b6cmr68360qtc.8.1685060827282; Thu, 25 May 2023 17:27:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685060827; cv=none; d=google.com; s=arc-20160816; b=NZs1L68UKh327fxBUCE8XavRWEnBNAZ32A6QZKZUVfonAAvNlGy/kSDHNcxvMmuoaR 6SZQ6Pms58bwyvQw6WmMxI/uTlC51vXupy717n6AD/dz2MMWVb9HAxRG4j8sEWYv8O2P Xe1+UHxke0/155pgYTd/eSPvtqWTWSGKvia+1WHFEY/voxgmRQangkDArCCbwA4Manuq 9PIdVJW0H9sdpUmUChBL8iSW7M9CMmVKrftxJrggYJdPX2s7yWOuyAJOREPik9hoQ3E0 TCWTlyTfyC95aps9Q+mzkhQsMKDVa2+n2P5ZZjU6Mfsk7f6DG8zE+xVmCSlQfwoB+r3I U+ug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=S6NQFZiOtbz6npGqiyw8uq/rNuokDU9J11iDsc0p5sk=; b=BgvkwUqGfGa5PFIHTQZPwAbujNP/86fWENewjdgCvLW2qbEEON1oOLLQbtLFSqgPhi ngqaidkjMbPqgQCVZe9flB3hGsVZMynJXb1b8AQ//CD9RhJ0QKWs+uSEgGlVZBG7+59J ZXP18evdD8/fCYaDnVuYyoqGplwx8mFoejHPGi2x6VQh+bGBBYQE8FgbUrcoyOVpEW/S RlpV/x1YidJfdubppXKp0Eos07r/7hp4tDR1RIfDKI0Ro44gpEnT6JvyzRzRFLTO5k6U ixolpYcVyBHrr0aTjYbZTb0IWMsX66WWYLr+8bGnxmpykmFstkN2icl6cPKbNPXy/37A EJiA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NEk6Pwrp; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id g9-20020ac85d49000000b003e6325794bcsi925301qtx.551.2023.05.25.17.27.07 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 25 May 2023 17:27:07 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NEk6Pwrp; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2LFV-0002im-4X; Thu, 25 May 2023 20:24:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2LFT-0002gn-37 for qemu-devel@nongnu.org; Thu, 25 May 2023 20:24:03 -0400 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q2LFG-0002DI-GP for qemu-devel@nongnu.org; Thu, 25 May 2023 20:24:02 -0400 Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-64d44b198baso265041b3a.0 for ; Thu, 25 May 2023 17:23:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1685060629; x=1687652629; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=S6NQFZiOtbz6npGqiyw8uq/rNuokDU9J11iDsc0p5sk=; b=NEk6PwrpN1tWfEBYxVopvFRF1iSf5H5UL3yV8vltA+oSrwiIfIcNoWKJf/aGJrB2eZ 3i68wjSdFNpjBd93m4oxq9hN//7Q2H0Uu4CIJ8DRGlLMcA/okrI985niyK6JqR1cTVJg maX4aABJhCnnAMk0n9DNMkwb7arE7EXUrD4He+4kg+jmlA9PBkvrMyLynII85yiMizzU G4TZwnmlGEQxmEeh5w2mAb0d0prYUZ+JWT/EVhBBTWjaq+Nb1nWiOflRsarhNeP9rLWE C0D0gMFgI2OjJtK9aK/CsO+TCa4D95hChq9xywyX8/FpTnmXhUBpiovE43dItN8uv1zB 3EVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685060629; x=1687652629; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S6NQFZiOtbz6npGqiyw8uq/rNuokDU9J11iDsc0p5sk=; b=dAfsAmk0GHYx9T+3+u5FHFbY0g84Vw9KX8YUA+cavYcX9OP5kaUw0g3niIiKcuTPRF PcAeSsGuKqGejwZ4l1xfSKZJX7tc8x51UUpaEG8YPnlIimLnBGFQPl+TZHTKE0DG5xCd TFpSk+yHjAvcBjwG5puV5fDMNHYQv+iQOZKGJS4tCjDzAJ8QyuEdP5+yxSaDO0lTYYqc GAukSAe8aUQzkb7roaEMs3XdRo3zz4kSdWsNJBEtLMruflRpVaNFudRGS7K/jCRpxO+Y pqwp1uALfEL1L3HPiyi1jkxTC0/Vo1ui7yzN68AHqYOaQFv2PeKqcsU22qvtN9tYOuMO 7TLQ== X-Gm-Message-State: AC+VfDxj5BGvBN7Hyd81RgiaeD/MDSDzu+vih9DIs1/f3Hf5NBJkM/s/ a7rVUMZC36vNt4V8+FQp1ONDi7ZRDUzolIhVN1c= X-Received: by 2002:aa7:8891:0:b0:63b:54e4:871b with SMTP id z17-20020aa78891000000b0063b54e4871bmr3705pfe.8.1685060628738; Thu, 25 May 2023 17:23:48 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:7ac5:31cc:3997:3a16]) by smtp.gmail.com with ESMTPSA id s188-20020a635ec5000000b0053491d92b65sm1675593pgb.84.2023.05.25.17.23.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 May 2023 17:23:48 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH v4 16/16] accel/tcg: Add aarch64 store_atom_insert_al16 Date: Thu, 25 May 2023 17:23:34 -0700 Message-Id: <20230526002334.1760495-17-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230526002334.1760495-1-richard.henderson@linaro.org> References: <20230526002334.1760495-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::432; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x432.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Peter Maydell --- host/include/aarch64/host/store-insert-al16.h | 47 +++++++++++++++++++ 1 file changed, 47 insertions(+) create mode 100644 host/include/aarch64/host/store-insert-al16.h diff --git a/host/include/aarch64/host/store-insert-al16.h b/host/include/aarch64/host/store-insert-al16.h new file mode 100644 index 0000000000..1943155bc6 --- /dev/null +++ b/host/include/aarch64/host/store-insert-al16.h @@ -0,0 +1,47 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Atomic store insert into 128-bit, AArch64 version. + * + * Copyright (C) 2023 Linaro, Ltd. + */ + +#ifndef AARCH64_STORE_INSERT_AL16_H +#define AARCH64_STORE_INSERT_AL16_H + +/** + * store_atom_insert_al16: + * @p: host address + * @val: shifted value to store + * @msk: mask for value to store + * + * Atomically store @val to @p masked by @msk. + */ +static inline void ATTRIBUTE_ATOMIC128_OPT +store_atom_insert_al16(Int128 *ps, Int128 val, Int128 msk) +{ + /* + * GCC only implements __sync* primitives for int128 on aarch64. + * We can do better without the barriers, and integrating the + * arithmetic into the load-exclusive/store-conditional pair. + */ + uint64_t tl, th, vl, vh, ml, mh; + uint32_t fail; + + qemu_build_assert(!HOST_BIG_ENDIAN); + vl = int128_getlo(val); + vh = int128_gethi(val); + ml = int128_getlo(msk); + mh = int128_gethi(msk); + + asm("0: ldxp %[l], %[h], %[mem]\n\t" + "bic %[l], %[l], %[ml]\n\t" + "bic %[h], %[h], %[mh]\n\t" + "orr %[l], %[l], %[vl]\n\t" + "orr %[h], %[h], %[vh]\n\t" + "stxp %w[f], %[l], %[h], %[mem]\n\t" + "cbnz %w[f], 0b\n" + : [mem] "+Q"(*ps), [f] "=&r"(fail), [l] "=&r"(tl), [h] "=&r"(th) + : [vl] "r"(vl), [vh] "r"(vh), [ml] "r"(ml), [mh] "r"(mh)); +} + +#endif /* AARCH64_STORE_INSERT_AL16_H */